2024-04-24T23:27:57.762 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-24T23:27:57.762 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672205 --owner scheduled_teuthology@teuthology --description fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu/{overrides ubuntu_latest}} kclient-overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health} tasks/kernel_cfuse_workunits_dbench_iozone} --name teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi --verbose -- /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672205/orig.config.yaml 2024-04-24T23:27:57.810 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-24T23:27:57.897 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672205 branch: squid description: fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu/{overrides ubuntu_latest}} kclient-overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health} tasks/kernel_cfuse_workunits_dbench_iozone} email: ceph-qa@ceph.com first_in_suite: false job_id: '7672205' kclient-workload: sequential: - kclient: - client.1 - workunit: clients: client.1: - suites/dbench.sh kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 valgrind: exit_on_first_error: false ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 seed: 8528 sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 sleep_before_teardown: 0 subset: 122/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi157.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHAfRS1+kKdGxkV0ecDPkMU4fbI8Zk7hbDuWL3iDXy2xV3OMdaboLf/Y1BX9iPjRBJLx5eWgCssb0hZqOoSmMlI= smithi169.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGUNF8X44qTR0U8iy8BEnPwE+8Z0hrRptOpkJu67oUJaC9XZZroz6yT9WgSl99I8qyu5Rov21j+zdmWFZ0gcnvI= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - parallel: - user-workload - kclient-workload teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-24_21:24:02 tube: smithi user: teuthology user-workload: sequential: - ceph-fuse: - client.0 - workunit: clients: client.0: - suites/iozone.sh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-24T23:27:57.897 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa; will attempt to use it 2024-04-24T23:27:57.898 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks 2024-04-24T23:27:57.898 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-24T23:27:57.899 INFO:teuthology.task.internal:Checking packages... 2024-04-24T23:27:57.923 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd2c45d977142e024a59b68a8f1dce8f0b9840551' 2024-04-24T23:27:57.924 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-24T23:27:57.924 INFO:teuthology.packaging:ref: None 2024-04-24T23:27:57.924 INFO:teuthology.packaging:tag: None 2024-04-24T23:27:57.924 INFO:teuthology.packaging:branch: squid 2024-04-24T23:27:57.924 INFO:teuthology.packaging:sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:27:57.924 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-04-24T23:27:58.215 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2445.gd2c45d97 2024-04-24T23:27:58.216 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-24T23:27:58.223 INFO:teuthology.task.internal:no buildpackages task found 2024-04-24T23:27:58.224 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-24T23:27:58.255 INFO:teuthology.task.internal:Saving configuration 2024-04-24T23:27:58.272 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-24T23:27:58.305 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-24T23:27:58.328 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi157.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672205', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 23:22:11.271807', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHAfRS1+kKdGxkV0ecDPkMU4fbI8Zk7hbDuWL3iDXy2xV3OMdaboLf/Y1BX9iPjRBJLx5eWgCssb0hZqOoSmMlI='} 2024-04-24T23:27:58.354 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi169.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672205', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 23:22:11.270314', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGUNF8X44qTR0U8iy8BEnPwE+8Z0hrRptOpkJu67oUJaC9XZZroz6yT9WgSl99I8qyu5Rov21j+zdmWFZ0gcnvI='} 2024-04-24T23:27:58.354 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-24T23:27:58.361 INFO:teuthology.task.internal:roles: ubuntu@smithi157.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2024-04-24T23:27:58.361 INFO:teuthology.task.internal:roles: ubuntu@smithi169.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1'] 2024-04-24T23:27:58.361 INFO:teuthology.run_tasks:Running task console_log... 2024-04-24T23:27:58.490 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f946815b040>, signals=[15]) 2024-04-24T23:27:58.490 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-24T23:27:58.497 INFO:teuthology.task.internal:Opening connections... 2024-04-24T23:27:58.497 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi157.front.sepia.ceph.com 2024-04-24T23:27:58.499 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T23:27:58.578 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi169.front.sepia.ceph.com 2024-04-24T23:27:58.579 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T23:27:58.653 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-24T23:27:58.659 DEBUG:teuthology.orchestra.run.smithi157:> uname -m 2024-04-24T23:27:58.678 INFO:teuthology.orchestra.run.smithi157.stdout:x86_64 2024-04-24T23:27:58.678 DEBUG:teuthology.orchestra.run.smithi157:> cat /etc/os-release 2024-04-24T23:27:58.736 INFO:teuthology.orchestra.run.smithi157.stdout:NAME="CentOS Stream" 2024-04-24T23:27:58.736 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION="9" 2024-04-24T23:27:58.736 INFO:teuthology.orchestra.run.smithi157.stdout:ID="centos" 2024-04-24T23:27:58.736 INFO:teuthology.orchestra.run.smithi157.stdout:ID_LIKE="rhel fedora" 2024-04-24T23:27:58.736 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION_ID="9" 2024-04-24T23:27:58.736 INFO:teuthology.orchestra.run.smithi157.stdout:PLATFORM_ID="platform:el9" 2024-04-24T23:27:58.736 INFO:teuthology.orchestra.run.smithi157.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-24T23:27:58.736 INFO:teuthology.orchestra.run.smithi157.stdout:ANSI_COLOR="0;31" 2024-04-24T23:27:58.736 INFO:teuthology.orchestra.run.smithi157.stdout:LOGO="fedora-logo-icon" 2024-04-24T23:27:58.737 INFO:teuthology.orchestra.run.smithi157.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-24T23:27:58.737 INFO:teuthology.orchestra.run.smithi157.stdout:HOME_URL="https://centos.org/" 2024-04-24T23:27:58.737 INFO:teuthology.orchestra.run.smithi157.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T23:27:58.737 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-24T23:27:58.737 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-24T23:27:58.737 INFO:teuthology.lock.ops:Updating smithi157.front.sepia.ceph.com on lock server 2024-04-24T23:27:58.762 DEBUG:teuthology.orchestra.run.smithi169:> uname -m 2024-04-24T23:27:58.780 INFO:teuthology.orchestra.run.smithi169.stdout:x86_64 2024-04-24T23:27:58.780 DEBUG:teuthology.orchestra.run.smithi169:> cat /etc/os-release 2024-04-24T23:27:58.838 INFO:teuthology.orchestra.run.smithi169.stdout:NAME="CentOS Stream" 2024-04-24T23:27:58.838 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION="9" 2024-04-24T23:27:58.838 INFO:teuthology.orchestra.run.smithi169.stdout:ID="centos" 2024-04-24T23:27:58.838 INFO:teuthology.orchestra.run.smithi169.stdout:ID_LIKE="rhel fedora" 2024-04-24T23:27:58.838 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION_ID="9" 2024-04-24T23:27:58.838 INFO:teuthology.orchestra.run.smithi169.stdout:PLATFORM_ID="platform:el9" 2024-04-24T23:27:58.838 INFO:teuthology.orchestra.run.smithi169.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-24T23:27:58.838 INFO:teuthology.orchestra.run.smithi169.stdout:ANSI_COLOR="0;31" 2024-04-24T23:27:58.838 INFO:teuthology.orchestra.run.smithi169.stdout:LOGO="fedora-logo-icon" 2024-04-24T23:27:58.838 INFO:teuthology.orchestra.run.smithi169.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-24T23:27:58.839 INFO:teuthology.orchestra.run.smithi169.stdout:HOME_URL="https://centos.org/" 2024-04-24T23:27:58.839 INFO:teuthology.orchestra.run.smithi169.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T23:27:58.839 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-24T23:27:58.839 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-24T23:27:58.839 INFO:teuthology.lock.ops:Updating smithi169.front.sepia.ceph.com on lock server 2024-04-24T23:27:58.862 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-24T23:27:58.871 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-24T23:27:58.897 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-24T23:27:58.897 DEBUG:teuthology.orchestra.run.smithi157:> test '!' -e /home/ubuntu/cephtest 2024-04-24T23:27:58.900 DEBUG:teuthology.orchestra.run.smithi169:> test '!' -e /home/ubuntu/cephtest 2024-04-24T23:27:58.916 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-24T23:27:58.930 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-24T23:27:58.930 DEBUG:teuthology.orchestra.run.smithi157:> test -z $(ls -A /var/lib/ceph) 2024-04-24T23:27:58.957 DEBUG:teuthology.orchestra.run.smithi169:> test -z $(ls -A /var/lib/ceph) 2024-04-24T23:27:59.059 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-24T23:27:59.116 INFO:teuthology.run_tasks:Running task kernel... 2024-04-24T23:27:59.130 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-04-24T23:27:59.130 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}} 2024-04-24T23:27:59.130 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-04-24T23:27:59.131 DEBUG:teuthology.orchestra.run.smithi157:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T23:27:59.131 DEBUG:teuthology.orchestra.run.smithi169:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T23:27:59.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:27:59.148 DEBUG:teuthology.orchestra.run.smithi157:> uname -r 2024-04-24T23:27:59.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:27:59.149 DEBUG:teuthology.orchestra.run.smithi169:> uname -r 2024-04-24T23:27:59.206 INFO:teuthology.orchestra.run.smithi157.stdout:5.14.0-437.el9.x86_64 2024-04-24T23:27:59.207 INFO:teuthology.task.kernel:Running kernel on smithi157: 5.14.0-437.el9.x86_64 2024-04-24T23:27:59.207 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum install -y kernel 2024-04-24T23:27:59.208 INFO:teuthology.orchestra.run.smithi169.stdout:5.14.0-437.el9.x86_64 2024-04-24T23:27:59.208 INFO:teuthology.task.kernel:Running kernel on smithi169: 5.14.0-437.el9.x86_64 2024-04-24T23:27:59.209 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum install -y kernel 2024-04-24T23:28:01.875 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - BaseOS 7.7 MB/s | 8.0 MB 00:01 2024-04-24T23:28:04.476 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - AppStream 12 MB/s | 19 MB 00:01 2024-04-24T23:28:04.949 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - BaseOS 2.0 MB/s | 8.0 MB 00:04 2024-04-24T23:28:08.485 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.2 MB 00:00 2024-04-24T23:28:09.841 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - AppStream 5.0 MB/s | 19 MB 00:03 2024-04-24T23:28:10.410 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - Extras packages 29 kB/s | 16 kB 00:00 2024-04-24T23:28:15.050 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.2 MB 00:01 2024-04-24T23:28:16.659 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 16 kB 00:00 2024-04-24T23:28:18.394 INFO:teuthology.orchestra.run.smithi157.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-24T23:28:23.435 INFO:teuthology.orchestra.run.smithi157.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-04-24T23:28:25.283 INFO:teuthology.orchestra.run.smithi157.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-24T23:28:25.283 INFO:teuthology.orchestra.run.smithi157.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-24T23:28:25.376 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-04-24T23:28:25.377 INFO:teuthology.orchestra.run.smithi157.stdout:Nothing to do. 2024-04-24T23:28:25.377 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-04-24T23:28:25.490 DEBUG:teuthology.orchestra.run.smithi157:> echo no | sudo yum reinstall kernel || true 2024-04-24T23:28:25.986 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:00:02 ago on Wed 24 Apr 2024 11:28:23 PM UTC. 2024-04-24T23:28:26.121 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-04-24T23:28:26.138 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-04-24T23:28:26.138 INFO:teuthology.orchestra.run.smithi157.stdout: Package Architecture Version Repository Size 2024-04-24T23:28:26.138 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-04-24T23:28:26.138 INFO:teuthology.orchestra.run.smithi157.stdout:Reinstalling: 2024-04-24T23:28:26.138 INFO:teuthology.orchestra.run.smithi157.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-24T23:28:26.138 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:28:26.139 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-04-24T23:28:26.139 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-04-24T23:28:26.139 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:28:26.139 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 255 k 2024-04-24T23:28:26.139 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 0 2024-04-24T23:28:26.139 INFO:teuthology.orchestra.run.smithi157.stderr:Operation aborted. 2024-04-24T23:28:26.200 INFO:teuthology.orchestra.run.smithi157.stdout:Is this ok [y/N]: 2024-04-24T23:28:26.200 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum reinstall -y kernel || true 2024-04-24T23:28:26.697 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:00:03 ago on Wed 24 Apr 2024 11:28:23 PM UTC. 2024-04-24T23:28:26.813 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-04-24T23:28:26.814 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-04-24T23:28:26.814 INFO:teuthology.orchestra.run.smithi157.stdout: Package Architecture Version Repository Size 2024-04-24T23:28:26.815 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-04-24T23:28:26.815 INFO:teuthology.orchestra.run.smithi157.stdout:Reinstalling: 2024-04-24T23:28:26.815 INFO:teuthology.orchestra.run.smithi157.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-24T23:28:26.815 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:28:26.816 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-04-24T23:28:26.816 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-04-24T23:28:26.816 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:28:26.816 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 255 k 2024-04-24T23:28:26.816 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 0 2024-04-24T23:28:26.816 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-04-24T23:28:27.320 INFO:teuthology.orchestra.run.smithi157.stdout:kernel-5.14.0-437.el9.x86_64.rpm 674 kB/s | 255 kB 00:00 2024-04-24T23:28:27.320 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:28:27.321 INFO:teuthology.orchestra.run.smithi157.stdout:Total 504 kB/s | 255 kB 00:00 2024-04-24T23:28:27.339 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-04-24T23:28:27.536 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-04-24T23:28:27.536 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-04-24T23:28:27.541 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-04-24T23:28:27.541 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-04-24T23:28:27.679 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-04-24T23:28:27.746 INFO:teuthology.orchestra.run.smithi157.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-24T23:28:27.746 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T23:28:28.453 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T23:28:28.454 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-24T23:28:29.161 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T23:28:29.162 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:28:29.162 INFO:teuthology.orchestra.run.smithi157.stdout:Reinstalled: 2024-04-24T23:28:29.162 INFO:teuthology.orchestra.run.smithi157.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-24T23:28:29.162 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:28:29.162 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-04-24T23:28:29.332 DEBUG:teuthology.orchestra.run.smithi157:> rpm -q kernel | sort -rV | head -n 1 2024-04-24T23:28:29.379 INFO:teuthology.orchestra.run.smithi157.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-24T23:28:29.380 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-24T23:28:29.380 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-24T23:28:29.380 DEBUG:teuthology.parallel:result is None 2024-04-24T23:28:29.554 INFO:teuthology.orchestra.run.smithi169.stdout:Extra Packages for Enterprise Linux 1.1 MB/s | 21 MB 00:19 2024-04-24T23:28:34.559 INFO:teuthology.orchestra.run.smithi169.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-04-24T23:28:36.415 INFO:teuthology.orchestra.run.smithi169.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-24T23:28:36.415 INFO:teuthology.orchestra.run.smithi169.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-24T23:28:36.505 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-04-24T23:28:36.506 INFO:teuthology.orchestra.run.smithi169.stdout:Nothing to do. 2024-04-24T23:28:36.506 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-04-24T23:28:36.621 DEBUG:teuthology.orchestra.run.smithi169:> echo no | sudo yum reinstall kernel || true 2024-04-24T23:28:37.123 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:00:03 ago on Wed 24 Apr 2024 11:28:34 PM UTC. 2024-04-24T23:28:37.259 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-04-24T23:28:37.277 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-04-24T23:28:37.277 INFO:teuthology.orchestra.run.smithi169.stdout: Package Architecture Version Repository Size 2024-04-24T23:28:37.277 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-04-24T23:28:37.277 INFO:teuthology.orchestra.run.smithi169.stdout:Reinstalling: 2024-04-24T23:28:37.277 INFO:teuthology.orchestra.run.smithi169.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-24T23:28:37.277 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-24T23:28:37.277 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-04-24T23:28:37.278 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-04-24T23:28:37.278 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-24T23:28:37.278 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 255 k 2024-04-24T23:28:37.278 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 0 2024-04-24T23:28:37.278 INFO:teuthology.orchestra.run.smithi169.stderr:Operation aborted. 2024-04-24T23:28:37.330 INFO:teuthology.orchestra.run.smithi169.stdout:Is this ok [y/N]: 2024-04-24T23:28:37.330 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum reinstall -y kernel || true 2024-04-24T23:28:37.828 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:00:03 ago on Wed 24 Apr 2024 11:28:34 PM UTC. 2024-04-24T23:28:37.945 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-04-24T23:28:37.946 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-04-24T23:28:37.946 INFO:teuthology.orchestra.run.smithi169.stdout: Package Architecture Version Repository Size 2024-04-24T23:28:37.946 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-04-24T23:28:37.946 INFO:teuthology.orchestra.run.smithi169.stdout:Reinstalling: 2024-04-24T23:28:37.946 INFO:teuthology.orchestra.run.smithi169.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-24T23:28:37.946 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-24T23:28:37.946 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-04-24T23:28:37.946 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-04-24T23:28:37.946 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-24T23:28:37.946 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 255 k 2024-04-24T23:28:37.946 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 0 2024-04-24T23:28:37.946 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-04-24T23:28:38.394 INFO:teuthology.orchestra.run.smithi169.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.6 MB/s | 255 kB 00:00 2024-04-24T23:28:38.395 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:28:38.395 INFO:teuthology.orchestra.run.smithi169.stdout:Total 568 kB/s | 255 kB 00:00 2024-04-24T23:28:38.413 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-04-24T23:28:38.608 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-04-24T23:28:38.608 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-04-24T23:28:38.613 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-04-24T23:28:38.613 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-04-24T23:28:38.751 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-04-24T23:28:38.829 INFO:teuthology.orchestra.run.smithi169.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-24T23:28:38.829 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T23:28:39.485 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T23:28:39.485 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-24T23:28:40.205 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T23:28:40.205 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-24T23:28:40.205 INFO:teuthology.orchestra.run.smithi169.stdout:Reinstalled: 2024-04-24T23:28:40.205 INFO:teuthology.orchestra.run.smithi169.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-24T23:28:40.205 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-24T23:28:40.205 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-04-24T23:28:40.364 DEBUG:teuthology.orchestra.run.smithi169:> rpm -q kernel | sort -rV | head -n 1 2024-04-24T23:28:40.411 INFO:teuthology.orchestra.run.smithi169.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-24T23:28:40.411 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-24T23:28:40.411 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-24T23:28:40.411 DEBUG:teuthology.parallel:result is None 2024-04-24T23:28:40.412 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-24T23:28:40.419 INFO:teuthology.task.internal:Creating test directory... 2024-04-24T23:28:40.419 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T23:28:40.422 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T23:28:40.438 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-24T23:28:40.465 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-24T23:28:40.516 INFO:teuthology.task.internal:Creating archive directory... 2024-04-24T23:28:40.516 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T23:28:40.518 DEBUG:teuthology.orchestra.run.smithi169:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T23:28:40.548 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-24T23:28:40.565 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-24T23:28:40.566 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T23:28:40.584 DEBUG:teuthology.orchestra.run.smithi169:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T23:28:40.617 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T23:28:40.623 INFO:teuthology.orchestra.run.smithi169.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T23:28:40.631 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T23:28:40.643 INFO:teuthology.orchestra.run.smithi169.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T23:28:40.644 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-24T23:28:40.653 INFO:teuthology.task.internal:Configuring sudo... 2024-04-24T23:28:40.653 DEBUG:teuthology.orchestra.run.smithi157:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T23:28:40.675 DEBUG:teuthology.orchestra.run.smithi169:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T23:28:40.708 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-24T23:28:40.717 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-24T23:28:40.717 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T23:28:40.742 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T23:28:40.763 DEBUG:teuthology.orchestra.run.smithi157:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T23:28:40.839 DEBUG:teuthology.orchestra.run.smithi157:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T23:28:40.882 DEBUG:teuthology.orchestra.run.smithi157:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T23:28:40.962 DEBUG:teuthology.orchestra.run.smithi157:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T23:28:41.004 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-24T23:28:41.004 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T23:28:41.068 DEBUG:teuthology.orchestra.run.smithi169:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T23:28:41.107 DEBUG:teuthology.orchestra.run.smithi169:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T23:28:41.175 DEBUG:teuthology.orchestra.run.smithi169:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T23:28:41.217 DEBUG:teuthology.orchestra.run.smithi169:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T23:28:41.256 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-24T23:28:41.256 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T23:28:41.320 DEBUG:teuthology.orchestra.run.smithi157:> sudo service rsyslog restart 2024-04-24T23:28:41.322 DEBUG:teuthology.orchestra.run.smithi169:> sudo service rsyslog restart 2024-04-24T23:28:41.354 INFO:teuthology.orchestra.run.smithi157.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T23:28:41.394 INFO:teuthology.orchestra.run.smithi169.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T23:28:41.831 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-24T23:28:41.839 INFO:teuthology.task.internal:Starting timer... 2024-04-24T23:28:41.839 INFO:teuthology.run_tasks:Running task pcp... 2024-04-24T23:28:41.892 INFO:teuthology.run_tasks:Running task selinux... 2024-04-24T23:28:41.934 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-04-24T23:28:41.935 DEBUG:teuthology.orchestra.run.smithi157:> sudo service auditd rotate 2024-04-24T23:28:41.991 INFO:teuthology.orchestra.run.smithi157.stdout:Rotating logs: 2024-04-24T23:28:41.993 DEBUG:teuthology.orchestra.run.smithi169:> sudo service auditd rotate 2024-04-24T23:28:42.050 INFO:teuthology.orchestra.run.smithi169.stdout:Rotating logs: 2024-04-24T23:28:42.052 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-24T23:28:42.052 DEBUG:teuthology.orchestra.run.smithi157:> /usr/sbin/getenforce 2024-04-24T23:28:42.078 INFO:teuthology.orchestra.run.smithi157.stdout:Permissive 2024-04-24T23:28:42.078 DEBUG:teuthology.orchestra.run.smithi169:> /usr/sbin/getenforce 2024-04-24T23:28:42.099 INFO:teuthology.orchestra.run.smithi169.stdout:Permissive 2024-04-24T23:28:42.099 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi157.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi169.front.sepia.ceph.com': 'permissive'} 2024-04-24T23:28:42.100 DEBUG:teuthology.orchestra.run.smithi157:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T23:28:42.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:28:42.144 DEBUG:teuthology.orchestra.run.smithi169:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T23:28:42.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:28:42.169 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-24T23:28:42.170 DEBUG:teuthology.orchestra.run.smithi157:> sudo /usr/sbin/setenforce permissive 2024-04-24T23:28:42.208 DEBUG:teuthology.orchestra.run.smithi169:> sudo /usr/sbin/setenforce permissive 2024-04-24T23:28:42.235 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-24T23:28:42.244 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-24T23:28:42.249 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-24T23:28:42.413 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-24T23:28:42.433 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-24T23:28:42.435 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi157.front.sepia.ceph.com,smithi169.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-24T23:32:14.045 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi157.front.sepia.ceph.com'), Remote(name='ubuntu@smithi169.front.sepia.ceph.com')] 2024-04-24T23:32:14.047 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-04-24T23:32:14.048 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T23:32:14.124 DEBUG:teuthology.orchestra.run.smithi157:> true 2024-04-24T23:32:14.205 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-04-24T23:32:14.206 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-04-24T23:32:14.206 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T23:32:14.283 DEBUG:teuthology.orchestra.run.smithi169:> true 2024-04-24T23:32:14.366 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-04-24T23:32:14.366 INFO:teuthology.run_tasks:Running task clock... 2024-04-24T23:32:14.769 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-24T23:32:14.769 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-24T23:32:14.770 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T23:32:14.773 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-24T23:32:14.773 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T23:32:14.804 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-24T23:32:14.806 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-24T23:32:14.821 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-24T23:32:14.821 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-24T23:32:14.844 INFO:teuthology.orchestra.run.smithi157.stderr:sudo: ntpd: command not found 2024-04-24T23:32:14.844 INFO:teuthology.orchestra.run.smithi169.stderr:sudo: ntpd: command not found 2024-04-24T23:32:14.856 INFO:teuthology.orchestra.run.smithi157.stdout:506 Cannot talk to daemon 2024-04-24T23:32:14.856 INFO:teuthology.orchestra.run.smithi169.stdout:506 Cannot talk to daemon 2024-04-24T23:32:14.870 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-24T23:32:14.871 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-24T23:32:14.883 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-24T23:32:14.883 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-24T23:32:14.935 INFO:teuthology.orchestra.run.smithi169.stderr:bash: line 1: ntpq: command not found 2024-04-24T23:32:14.939 INFO:teuthology.orchestra.run.smithi169.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T23:32:14.939 INFO:teuthology.orchestra.run.smithi169.stdout:=============================================================================== 2024-04-24T23:32:14.940 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:32:14.940 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:32:14.940 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:32:14.940 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:32:14.949 INFO:teuthology.orchestra.run.smithi157.stderr:bash: line 1: ntpq: command not found 2024-04-24T23:32:14.954 INFO:teuthology.orchestra.run.smithi157.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T23:32:14.954 INFO:teuthology.orchestra.run.smithi157.stdout:=============================================================================== 2024-04-24T23:32:14.954 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:32:14.954 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:32:14.955 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:32:14.955 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:32:14.955 INFO:teuthology.run_tasks:Running task install... 2024-04-24T23:32:14.978 DEBUG:teuthology.task.install:project ceph 2024-04-24T23:32:14.979 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}} 2024-04-24T23:32:14.979 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'} 2024-04-24T23:32:14.979 INFO:teuthology.task.install:Using flavor: default 2024-04-24T23:32:14.986 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-24T23:32:14.987 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-24T23:32:14.987 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'tag': None, 'wait_for_package': False} 2024-04-24T23:32:14.988 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:32:14.991 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'tag': None, 'wait_for_package': False} 2024-04-24T23:32:14.991 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:32:15.151 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/ 2024-04-24T23:32:15.151 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2445.gd2c45d97 2024-04-24T23:32:15.262 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-24T23:32:15.263 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-24T23:32:15.263 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-24T23:32:15.288 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-24T23:32:15.288 DEBUG:teuthology.orchestra.run.smithi169:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2c45d977142e024a59b68a8f1dce8f0b9840551/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-24T23:32:15.290 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/ 2024-04-24T23:32:15.290 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2445.gd2c45d97 2024-04-24T23:32:15.353 DEBUG:teuthology.orchestra.run.smithi169:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-24T23:32:15.410 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-24T23:32:15.410 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-24T23:32:15.410 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-24T23:32:15.427 DEBUG:teuthology.orchestra.run.smithi169:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-24T23:32:15.437 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-24T23:32:15.437 DEBUG:teuthology.orchestra.run.smithi157:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2c45d977142e024a59b68a8f1dce8f0b9840551/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-24T23:32:15.492 INFO:teuthology.orchestra.run.smithi169.stdout:check_obsoletes = 1 2024-04-24T23:32:15.493 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum clean all 2024-04-24T23:32:15.507 DEBUG:teuthology.orchestra.run.smithi157:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-24T23:32:15.588 DEBUG:teuthology.orchestra.run.smithi157:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-24T23:32:15.655 INFO:teuthology.orchestra.run.smithi157.stdout:check_obsoletes = 1 2024-04-24T23:32:15.657 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum clean all 2024-04-24T23:32:15.721 INFO:teuthology.orchestra.run.smithi169.stdout:48 files removed 2024-04-24T23:32:15.749 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-24T23:32:15.886 INFO:teuthology.orchestra.run.smithi157.stdout:48 files removed 2024-04-24T23:32:15.913 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-24T23:32:16.637 INFO:teuthology.orchestra.run.smithi169.stdout:ceph packages for x86_64 133 kB/s | 83 kB 00:00 2024-04-24T23:32:17.065 INFO:teuthology.orchestra.run.smithi157.stdout:ceph packages for x86_64 93 kB/s | 83 kB 00:00 2024-04-24T23:32:17.455 INFO:teuthology.orchestra.run.smithi169.stdout:ceph noarch packages 15 kB/s | 11 kB 00:00 2024-04-24T23:32:17.774 INFO:teuthology.orchestra.run.smithi157.stdout:ceph noarch packages 17 kB/s | 11 kB 00:00 2024-04-24T23:32:18.355 INFO:teuthology.orchestra.run.smithi169.stdout:ceph source packages 2.2 kB/s | 1.9 kB 00:00 2024-04-24T23:32:18.541 INFO:teuthology.orchestra.run.smithi157.stdout:ceph source packages 2.6 kB/s | 1.9 kB 00:00 2024-04-24T23:32:19.408 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - BaseOS 9.7 MB/s | 8.0 MB 00:00 2024-04-24T23:32:21.267 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-04-24T23:32:22.730 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - BaseOS 1.9 MB/s | 8.0 MB 00:04 2024-04-24T23:32:24.673 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-04-24T23:32:26.234 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - CRB 4.4 MB/s | 6.2 MB 00:01 2024-04-24T23:32:27.727 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - Extras packages 48 kB/s | 16 kB 00:00 2024-04-24T23:32:28.895 INFO:teuthology.orchestra.run.smithi157.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-04-24T23:32:29.540 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - CRB 4.7 MB/s | 6.2 MB 00:01 2024-04-24T23:32:31.166 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - Extras packages 35 kB/s | 16 kB 00:00 2024-04-24T23:32:33.669 INFO:teuthology.orchestra.run.smithi169.stdout:Extra Packages for Enterprise Linux 8.8 MB/s | 21 MB 00:02 2024-04-24T23:32:34.028 INFO:teuthology.orchestra.run.smithi157.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-04-24T23:32:36.163 INFO:teuthology.orchestra.run.smithi157.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T23:32:36.164 INFO:teuthology.orchestra.run.smithi157.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T23:32:36.175 INFO:teuthology.orchestra.run.smithi157.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-24T23:32:36.175 INFO:teuthology.orchestra.run.smithi157.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-24T23:32:36.176 INFO:teuthology.orchestra.run.smithi157.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-24T23:32:36.177 INFO:teuthology.orchestra.run.smithi157.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-24T23:32:36.179 INFO:teuthology.orchestra.run.smithi157.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-24T23:32:36.181 INFO:teuthology.orchestra.run.smithi157.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-24T23:32:36.182 INFO:teuthology.orchestra.run.smithi157.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-24T23:32:36.244 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-04-24T23:32:36.253 INFO:teuthology.orchestra.run.smithi157.stdout:======================================================================================= 2024-04-24T23:32:36.253 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2024-04-24T23:32:36.254 INFO:teuthology.orchestra.run.smithi157.stdout:======================================================================================= 2024-04-24T23:32:36.254 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-04-24T23:32:36.254 INFO:teuthology.orchestra.run.smithi157.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-24T23:32:36.254 INFO:teuthology.orchestra.run.smithi157.stdout: ceph x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 6.6 k 2024-04-24T23:32:36.254 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-base x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.4 M 2024-04-24T23:32:36.254 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 890 k 2024-04-24T23:32:36.254 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 145 k 2024-04-24T23:32:36.254 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 1.5 M 2024-04-24T23:32:36.254 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 141 k 2024-04-24T23:32:36.254 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 3.5 M 2024-04-24T23:32:36.254 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 7.4 M 2024-04-24T23:32:36.254 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-rook noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 49 k 2024-04-24T23:32:36.255 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-radosgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 10 M 2024-04-24T23:32:36.255 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-test x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 48 M 2024-04-24T23:32:36.255 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-volume noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 264 k 2024-04-24T23:32:36.255 INFO:teuthology.orchestra.run.smithi157.stdout: cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 762 k 2024-04-24T23:32:36.255 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 222 k 2024-04-24T23:32:36.255 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-shell x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 27 k 2024-04-24T23:32:36.255 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-top noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 18 k 2024-04-24T23:32:36.255 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-24T23:32:36.255 INFO:teuthology.orchestra.run.smithi157.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-24T23:32:36.255 INFO:teuthology.orchestra.run.smithi157.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-24T23:32:36.255 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 32 k 2024-04-24T23:32:36.255 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 745 k 2024-04-24T23:32:36.255 INFO:teuthology.orchestra.run.smithi157.stdout: librados-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 127 k 2024-04-24T23:32:36.256 INFO:teuthology.orchestra.run.smithi157.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-24T23:32:36.256 INFO:teuthology.orchestra.run.smithi157.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-24T23:32:36.256 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-24T23:32:36.256 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-24T23:32:36.256 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-24T23:32:36.256 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-24T23:32:36.256 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cephfs x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 162 k 2024-04-24T23:32:36.256 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rados x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 321 k 2024-04-24T23:32:36.256 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 299 k 2024-04-24T23:32:36.256 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 99 k 2024-04-24T23:32:36.256 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 85 k 2024-04-24T23:32:36.257 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-24T23:32:36.257 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-nbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 171 k 2024-04-24T23:32:36.257 INFO:teuthology.orchestra.run.smithi157.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-24T23:32:36.257 INFO:teuthology.orchestra.run.smithi157.stdout:Upgrading: 2024-04-24T23:32:36.257 INFO:teuthology.orchestra.run.smithi157.stdout: librados2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.4 M 2024-04-24T23:32:36.257 INFO:teuthology.orchestra.run.smithi157.stdout: librbd1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-24T23:32:36.257 INFO:teuthology.orchestra.run.smithi157.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-24T23:32:36.257 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-24T23:32:36.257 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2024-04-24T23:32:36.257 INFO:teuthology.orchestra.run.smithi157.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-24T23:32:36.257 INFO:teuthology.orchestra.run.smithi157.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-24T23:32:36.257 INFO:teuthology.orchestra.run.smithi157.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-24T23:32:36.257 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 21 M 2024-04-24T23:32:36.257 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 32 k 2024-04-24T23:32:36.258 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mds x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 2.4 M 2024-04-24T23:32:36.258 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 249 k 2024-04-24T23:32:36.258 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mon x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 4.7 M 2024-04-24T23:32:36.258 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-osd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 17 M 2024-04-24T23:32:36.258 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 15 k 2024-04-24T23:32:36.258 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-selinux x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 25 k 2024-04-24T23:32:36.258 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-24T23:32:36.258 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-24T23:32:36.258 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-24T23:32:36.258 INFO:teuthology.orchestra.run.smithi157.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-24T23:32:36.258 INFO:teuthology.orchestra.run.smithi157.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-24T23:32:36.258 INFO:teuthology.orchestra.run.smithi157.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-24T23:32:36.258 INFO:teuthology.orchestra.run.smithi157.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-24T23:32:36.259 INFO:teuthology.orchestra.run.smithi157.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-24T23:32:36.259 INFO:teuthology.orchestra.run.smithi157.stdout: libcephsqlite x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 163 k 2024-04-24T23:32:36.259 INFO:teuthology.orchestra.run.smithi157.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-24T23:32:36.259 INFO:teuthology.orchestra.run.smithi157.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-24T23:32:36.259 INFO:teuthology.orchestra.run.smithi157.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-24T23:32:36.259 INFO:teuthology.orchestra.run.smithi157.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-24T23:32:36.259 INFO:teuthology.orchestra.run.smithi157.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-24T23:32:36.259 INFO:teuthology.orchestra.run.smithi157.stdout: libradosstriper1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 501 k 2024-04-24T23:32:36.259 INFO:teuthology.orchestra.run.smithi157.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-24T23:32:36.259 INFO:teuthology.orchestra.run.smithi157.stdout: librgw2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.3 M 2024-04-24T23:32:36.259 INFO:teuthology.orchestra.run.smithi157.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-24T23:32:36.259 INFO:teuthology.orchestra.run.smithi157.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-24T23:32:36.259 INFO:teuthology.orchestra.run.smithi157.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-24T23:32:36.260 INFO:teuthology.orchestra.run.smithi157.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-24T23:32:36.260 INFO:teuthology.orchestra.run.smithi157.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-24T23:32:36.260 INFO:teuthology.orchestra.run.smithi157.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-24T23:32:36.260 INFO:teuthology.orchestra.run.smithi157.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-24T23:32:36.260 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-24T23:32:36.260 INFO:teuthology.orchestra.run.smithi157.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-24T23:32:36.260 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-24T23:32:36.260 INFO:teuthology.orchestra.run.smithi157.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-24T23:32:36.260 INFO:teuthology.orchestra.run.smithi157.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-24T23:32:36.260 INFO:teuthology.orchestra.run.smithi157.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-24T23:32:36.260 INFO:teuthology.orchestra.run.smithi157.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-24T23:32:36.260 INFO:teuthology.orchestra.run.smithi157.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-24T23:32:36.260 INFO:teuthology.orchestra.run.smithi157.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-24T23:32:36.261 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-24T23:32:36.261 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-argparse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 45 k 2024-04-24T23:32:36.261 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 133 k 2024-04-24T23:32:36.261 INFO:teuthology.orchestra.run.smithi157.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-24T23:32:36.261 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-24T23:32:36.261 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-24T23:32:36.261 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-24T23:32:36.261 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-24T23:32:36.261 INFO:teuthology.orchestra.run.smithi157.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-24T23:32:36.261 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-24T23:32:36.261 INFO:teuthology.orchestra.run.smithi157.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-24T23:32:36.261 INFO:teuthology.orchestra.run.smithi157.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-24T23:32:36.261 INFO:teuthology.orchestra.run.smithi157.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-24T23:32:36.261 INFO:teuthology.orchestra.run.smithi157.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-24T23:32:36.262 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-24T23:32:36.262 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-24T23:32:36.262 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-24T23:32:36.262 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-24T23:32:36.262 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-24T23:32:36.262 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-24T23:32:36.262 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-24T23:32:36.262 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-24T23:32:36.262 INFO:teuthology.orchestra.run.smithi157.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-24T23:32:36.262 INFO:teuthology.orchestra.run.smithi157.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-24T23:32:36.262 INFO:teuthology.orchestra.run.smithi157.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-24T23:32:36.262 INFO:teuthology.orchestra.run.smithi157.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-24T23:32:36.262 INFO:teuthology.orchestra.run.smithi157.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-24T23:32:36.263 INFO:teuthology.orchestra.run.smithi157.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-24T23:32:36.263 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-24T23:32:36.263 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-24T23:32:36.263 INFO:teuthology.orchestra.run.smithi157.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-24T23:32:36.263 INFO:teuthology.orchestra.run.smithi157.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-24T23:32:36.263 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-24T23:32:36.263 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-24T23:32:36.263 INFO:teuthology.orchestra.run.smithi157.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-24T23:32:36.263 INFO:teuthology.orchestra.run.smithi157.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-24T23:32:36.263 INFO:teuthology.orchestra.run.smithi157.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-24T23:32:36.263 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-24T23:32:36.263 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-24T23:32:36.263 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-24T23:32:36.264 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-24T23:32:36.264 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-24T23:32:36.264 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-24T23:32:36.264 INFO:teuthology.orchestra.run.smithi157.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-24T23:32:36.264 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-24T23:32:36.264 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-24T23:32:36.264 INFO:teuthology.orchestra.run.smithi157.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-24T23:32:36.264 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-24T23:32:36.264 INFO:teuthology.orchestra.run.smithi157.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-24T23:32:36.264 INFO:teuthology.orchestra.run.smithi157.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-24T23:32:36.264 INFO:teuthology.orchestra.run.smithi157.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-24T23:32:36.264 INFO:teuthology.orchestra.run.smithi157.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-24T23:32:36.265 INFO:teuthology.orchestra.run.smithi157.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-24T23:32:36.265 INFO:teuthology.orchestra.run.smithi157.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-24T23:32:36.265 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-24T23:32:36.265 INFO:teuthology.orchestra.run.smithi157.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-24T23:32:36.265 INFO:teuthology.orchestra.run.smithi157.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-24T23:32:36.265 INFO:teuthology.orchestra.run.smithi157.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-24T23:32:36.265 INFO:teuthology.orchestra.run.smithi157.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-24T23:32:36.265 INFO:teuthology.orchestra.run.smithi157.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-24T23:32:36.265 INFO:teuthology.orchestra.run.smithi157.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-24T23:32:36.265 INFO:teuthology.orchestra.run.smithi157.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-24T23:32:36.265 INFO:teuthology.orchestra.run.smithi157.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-24T23:32:36.265 INFO:teuthology.orchestra.run.smithi157.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-24T23:32:36.265 INFO:teuthology.orchestra.run.smithi157.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-24T23:32:36.265 INFO:teuthology.orchestra.run.smithi157.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-24T23:32:36.266 INFO:teuthology.orchestra.run.smithi157.stdout:Installing weak dependencies: 2024-04-24T23:32:36.266 INFO:teuthology.orchestra.run.smithi157.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-24T23:32:36.266 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:32:36.266 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-04-24T23:32:36.266 INFO:teuthology.orchestra.run.smithi157.stdout:======================================================================================= 2024-04-24T23:32:36.266 INFO:teuthology.orchestra.run.smithi157.stdout:Install 148 Packages 2024-04-24T23:32:36.266 INFO:teuthology.orchestra.run.smithi157.stdout:Upgrade 4 Packages 2024-04-24T23:32:36.266 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:32:36.267 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 220 M 2024-04-24T23:32:36.267 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-04-24T23:32:37.554 INFO:teuthology.orchestra.run.smithi157.stdout:(1/152): ceph-19.0.0-2445.gd2c45d97.el9.x86_64. 14 kB/s | 6.6 kB 00:00 2024-04-24T23:32:37.929 INFO:teuthology.orchestra.run.smithi157.stdout:(2/152): ceph-fuse-19.0.0-2445.gd2c45d97.el9.x8 2.3 MB/s | 890 kB 00:00 2024-04-24T23:32:38.055 INFO:teuthology.orchestra.run.smithi157.stdout:(3/152): ceph-immutable-object-cache-19.0.0-244 1.1 MB/s | 145 kB 00:00 2024-04-24T23:32:38.622 INFO:teuthology.orchestra.run.smithi157.stdout:(4/152): ceph-base-19.0.0-2445.gd2c45d97.el9.x8 3.5 MB/s | 5.4 MB 00:01 2024-04-24T23:32:38.672 INFO:teuthology.orchestra.run.smithi157.stdout:(5/152): ceph-mds-19.0.0-2445.gd2c45d97.el9.x86 3.8 MB/s | 2.4 MB 00:00 2024-04-24T23:32:38.817 INFO:teuthology.orchestra.run.smithi169.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-04-24T23:32:38.914 INFO:teuthology.orchestra.run.smithi157.stdout:(6/152): ceph-common-19.0.0-2445.gd2c45d97.el9. 12 MB/s | 21 MB 00:01 2024-04-24T23:32:38.956 INFO:teuthology.orchestra.run.smithi157.stdout:(7/152): ceph-mgr-19.0.0-2445.gd2c45d97.el9.x86 4.5 MB/s | 1.5 MB 00:00 2024-04-24T23:32:39.664 INFO:teuthology.orchestra.run.smithi157.stdout:(8/152): ceph-mon-19.0.0-2445.gd2c45d97.el9.x86 4.7 MB/s | 4.7 MB 00:00 2024-04-24T23:32:39.748 INFO:teuthology.orchestra.run.smithi157.stdout:(9/152): ceph-selinux-19.0.0-2445.gd2c45d97.el9 303 kB/s | 25 kB 00:00 2024-04-24T23:32:40.040 INFO:teuthology.orchestra.run.smithi157.stdout:(10/152): ceph-osd-19.0.0-2445.gd2c45d97.el9.x8 15 MB/s | 17 MB 00:01 2024-04-24T23:32:40.099 INFO:teuthology.orchestra.run.smithi157.stdout:(11/152): cephfs-mirror-19.0.0-2445.gd2c45d97.e 3.7 MB/s | 222 kB 00:00 2024-04-24T23:32:40.157 INFO:teuthology.orchestra.run.smithi157.stdout:(12/152): cephfs-shell-19.0.0-2445.gd2c45d97.el 465 kB/s | 27 kB 00:00 2024-04-24T23:32:40.224 INFO:teuthology.orchestra.run.smithi157.stdout:(13/152): libcephfs-devel-19.0.0-2445.gd2c45d97 476 kB/s | 32 kB 00:00 2024-04-24T23:32:40.383 INFO:teuthology.orchestra.run.smithi157.stdout:(14/152): ceph-radosgw-19.0.0-2445.gd2c45d97.el 7.2 MB/s | 10 MB 00:01 2024-04-24T23:32:40.408 INFO:teuthology.orchestra.run.smithi157.stdout:(15/152): libcephfs2-19.0.0-2445.gd2c45d97.el9. 4.0 MB/s | 745 kB 00:00 2024-04-24T23:32:40.450 INFO:teuthology.orchestra.run.smithi157.stdout:(16/152): libcephsqlite-19.0.0-2445.gd2c45d97.e 2.4 MB/s | 163 kB 00:00 2024-04-24T23:32:40.475 INFO:teuthology.orchestra.run.smithi157.stdout:(17/152): librados-devel-19.0.0-2445.gd2c45d97. 1.8 MB/s | 127 kB 00:00 2024-04-24T23:32:40.534 INFO:teuthology.orchestra.run.smithi157.stdout:(18/152): libradosstriper1-19.0.0-2445.gd2c45d9 5.8 MB/s | 501 kB 00:00 2024-04-24T23:32:40.617 INFO:teuthology.orchestra.run.smithi157.stdout:(19/152): python3-ceph-argparse-19.0.0-2445.gd2 543 kB/s | 45 kB 00:00 2024-04-24T23:32:40.718 INFO:teuthology.orchestra.run.smithi157.stdout:(20/152): python3-ceph-common-19.0.0-2445.gd2c4 1.3 MB/s | 133 kB 00:00 2024-04-24T23:32:40.818 INFO:teuthology.orchestra.run.smithi157.stdout:(21/152): librgw2-19.0.0-2445.gd2c45d97.el9.x86 16 MB/s | 5.3 MB 00:00 2024-04-24T23:32:40.843 INFO:teuthology.orchestra.run.smithi157.stdout:(22/152): python3-cephfs-19.0.0-2445.gd2c45d97. 1.3 MB/s | 162 kB 00:00 2024-04-24T23:32:40.877 INFO:teuthology.orchestra.run.smithi157.stdout:(23/152): python3-rados-19.0.0-2445.gd2c45d97.e 5.3 MB/s | 321 kB 00:00 2024-04-24T23:32:40.919 INFO:teuthology.orchestra.run.smithi157.stdout:(24/152): python3-rbd-19.0.0-2445.gd2c45d97.el9 3.9 MB/s | 299 kB 00:00 2024-04-24T23:32:40.944 INFO:teuthology.orchestra.run.smithi157.stdout:(25/152): python3-rgw-19.0.0-2445.gd2c45d97.el9 1.5 MB/s | 99 kB 00:00 2024-04-24T23:32:40.968 INFO:teuthology.orchestra.run.smithi169.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T23:32:40.968 INFO:teuthology.orchestra.run.smithi169.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T23:32:40.979 INFO:teuthology.orchestra.run.smithi169.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-24T23:32:40.979 INFO:teuthology.orchestra.run.smithi169.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-24T23:32:40.981 INFO:teuthology.orchestra.run.smithi169.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-24T23:32:40.981 INFO:teuthology.orchestra.run.smithi169.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-24T23:32:40.983 INFO:teuthology.orchestra.run.smithi169.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-24T23:32:40.985 INFO:teuthology.orchestra.run.smithi169.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-24T23:32:40.986 INFO:teuthology.orchestra.run.smithi169.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-24T23:32:40.986 INFO:teuthology.orchestra.run.smithi157.stdout:(26/152): rbd-fuse-19.0.0-2445.gd2c45d97.el9.x8 1.2 MB/s | 85 kB 00:00 2024-04-24T23:32:41.046 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-04-24T23:32:41.055 INFO:teuthology.orchestra.run.smithi169.stdout:======================================================================================= 2024-04-24T23:32:41.055 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repository Size 2024-04-24T23:32:41.055 INFO:teuthology.orchestra.run.smithi169.stdout:======================================================================================= 2024-04-24T23:32:41.055 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-04-24T23:32:41.056 INFO:teuthology.orchestra.run.smithi169.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-24T23:32:41.056 INFO:teuthology.orchestra.run.smithi169.stdout: ceph x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 6.6 k 2024-04-24T23:32:41.056 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-base x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.4 M 2024-04-24T23:32:41.056 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 890 k 2024-04-24T23:32:41.056 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 145 k 2024-04-24T23:32:41.056 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 1.5 M 2024-04-24T23:32:41.056 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 141 k 2024-04-24T23:32:41.056 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 3.5 M 2024-04-24T23:32:41.056 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 7.4 M 2024-04-24T23:32:41.056 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-rook noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 49 k 2024-04-24T23:32:41.056 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-radosgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 10 M 2024-04-24T23:32:41.057 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-test x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 48 M 2024-04-24T23:32:41.057 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-volume noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 264 k 2024-04-24T23:32:41.057 INFO:teuthology.orchestra.run.smithi169.stdout: cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 762 k 2024-04-24T23:32:41.057 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 222 k 2024-04-24T23:32:41.057 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-shell x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 27 k 2024-04-24T23:32:41.057 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-top noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 18 k 2024-04-24T23:32:41.057 INFO:teuthology.orchestra.run.smithi169.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-24T23:32:41.057 INFO:teuthology.orchestra.run.smithi169.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-24T23:32:41.057 INFO:teuthology.orchestra.run.smithi169.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-24T23:32:41.057 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 32 k 2024-04-24T23:32:41.057 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 745 k 2024-04-24T23:32:41.057 INFO:teuthology.orchestra.run.smithi169.stdout: librados-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 127 k 2024-04-24T23:32:41.057 INFO:teuthology.orchestra.run.smithi169.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-24T23:32:41.057 INFO:teuthology.orchestra.run.smithi169.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-24T23:32:41.058 INFO:teuthology.orchestra.run.smithi169.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-24T23:32:41.058 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-24T23:32:41.058 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-24T23:32:41.058 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-24T23:32:41.058 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cephfs x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 162 k 2024-04-24T23:32:41.058 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rados x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 321 k 2024-04-24T23:32:41.058 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 299 k 2024-04-24T23:32:41.058 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 99 k 2024-04-24T23:32:41.058 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 85 k 2024-04-24T23:32:41.058 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-24T23:32:41.058 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-nbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 171 k 2024-04-24T23:32:41.058 INFO:teuthology.orchestra.run.smithi169.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-24T23:32:41.058 INFO:teuthology.orchestra.run.smithi169.stdout:Upgrading: 2024-04-24T23:32:41.059 INFO:teuthology.orchestra.run.smithi169.stdout: librados2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.4 M 2024-04-24T23:32:41.059 INFO:teuthology.orchestra.run.smithi169.stdout: librbd1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-24T23:32:41.059 INFO:teuthology.orchestra.run.smithi169.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-24T23:32:41.059 INFO:teuthology.orchestra.run.smithi169.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-24T23:32:41.059 INFO:teuthology.orchestra.run.smithi169.stdout:Installing dependencies: 2024-04-24T23:32:41.059 INFO:teuthology.orchestra.run.smithi169.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-24T23:32:41.059 INFO:teuthology.orchestra.run.smithi169.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-24T23:32:41.059 INFO:teuthology.orchestra.run.smithi169.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-24T23:32:41.059 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 21 M 2024-04-24T23:32:41.059 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 32 k 2024-04-24T23:32:41.059 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mds x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 2.4 M 2024-04-24T23:32:41.059 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 249 k 2024-04-24T23:32:41.059 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mon x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 4.7 M 2024-04-24T23:32:41.060 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-osd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 17 M 2024-04-24T23:32:41.060 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 15 k 2024-04-24T23:32:41.060 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-selinux x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 25 k 2024-04-24T23:32:41.060 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-24T23:32:41.060 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-24T23:32:41.060 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-24T23:32:41.060 INFO:teuthology.orchestra.run.smithi169.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-24T23:32:41.060 INFO:teuthology.orchestra.run.smithi169.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-24T23:32:41.060 INFO:teuthology.orchestra.run.smithi169.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-24T23:32:41.060 INFO:teuthology.orchestra.run.smithi169.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-24T23:32:41.060 INFO:teuthology.orchestra.run.smithi169.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-24T23:32:41.060 INFO:teuthology.orchestra.run.smithi169.stdout: libcephsqlite x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 163 k 2024-04-24T23:32:41.060 INFO:teuthology.orchestra.run.smithi169.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-24T23:32:41.060 INFO:teuthology.orchestra.run.smithi169.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-24T23:32:41.061 INFO:teuthology.orchestra.run.smithi169.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-24T23:32:41.061 INFO:teuthology.orchestra.run.smithi169.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-24T23:32:41.061 INFO:teuthology.orchestra.run.smithi169.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-24T23:32:41.061 INFO:teuthology.orchestra.run.smithi169.stdout: libradosstriper1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 501 k 2024-04-24T23:32:41.061 INFO:teuthology.orchestra.run.smithi169.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-24T23:32:41.061 INFO:teuthology.orchestra.run.smithi169.stdout: librgw2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.3 M 2024-04-24T23:32:41.061 INFO:teuthology.orchestra.run.smithi169.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-24T23:32:41.061 INFO:teuthology.orchestra.run.smithi169.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-24T23:32:41.061 INFO:teuthology.orchestra.run.smithi169.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-24T23:32:41.061 INFO:teuthology.orchestra.run.smithi169.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-24T23:32:41.061 INFO:teuthology.orchestra.run.smithi169.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-24T23:32:41.061 INFO:teuthology.orchestra.run.smithi169.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-24T23:32:41.061 INFO:teuthology.orchestra.run.smithi169.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-24T23:32:41.062 INFO:teuthology.orchestra.run.smithi169.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-24T23:32:41.062 INFO:teuthology.orchestra.run.smithi169.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-24T23:32:41.062 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-24T23:32:41.062 INFO:teuthology.orchestra.run.smithi169.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-24T23:32:41.062 INFO:teuthology.orchestra.run.smithi169.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-24T23:32:41.062 INFO:teuthology.orchestra.run.smithi169.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-24T23:32:41.062 INFO:teuthology.orchestra.run.smithi169.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-24T23:32:41.062 INFO:teuthology.orchestra.run.smithi169.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-24T23:32:41.062 INFO:teuthology.orchestra.run.smithi169.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-24T23:32:41.062 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-24T23:32:41.062 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-argparse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 45 k 2024-04-24T23:32:41.062 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 133 k 2024-04-24T23:32:41.062 INFO:teuthology.orchestra.run.smithi169.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-24T23:32:41.063 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-24T23:32:41.063 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-24T23:32:41.063 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-24T23:32:41.063 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-24T23:32:41.063 INFO:teuthology.orchestra.run.smithi169.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-24T23:32:41.063 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-24T23:32:41.063 INFO:teuthology.orchestra.run.smithi169.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-24T23:32:41.063 INFO:teuthology.orchestra.run.smithi169.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-24T23:32:41.063 INFO:teuthology.orchestra.run.smithi169.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-24T23:32:41.063 INFO:teuthology.orchestra.run.smithi169.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-24T23:32:41.063 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-24T23:32:41.063 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-24T23:32:41.063 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-24T23:32:41.064 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-24T23:32:41.064 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-24T23:32:41.064 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-24T23:32:41.064 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-24T23:32:41.064 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-24T23:32:41.064 INFO:teuthology.orchestra.run.smithi169.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-24T23:32:41.064 INFO:teuthology.orchestra.run.smithi169.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-24T23:32:41.064 INFO:teuthology.orchestra.run.smithi169.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-24T23:32:41.064 INFO:teuthology.orchestra.run.smithi169.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-24T23:32:41.064 INFO:teuthology.orchestra.run.smithi169.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-24T23:32:41.064 INFO:teuthology.orchestra.run.smithi169.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-24T23:32:41.064 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-24T23:32:41.064 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-24T23:32:41.064 INFO:teuthology.orchestra.run.smithi169.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-24T23:32:41.065 INFO:teuthology.orchestra.run.smithi169.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-24T23:32:41.065 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-24T23:32:41.065 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-24T23:32:41.065 INFO:teuthology.orchestra.run.smithi169.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-24T23:32:41.065 INFO:teuthology.orchestra.run.smithi169.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-24T23:32:41.065 INFO:teuthology.orchestra.run.smithi169.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-24T23:32:41.065 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-24T23:32:41.065 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-24T23:32:41.065 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-24T23:32:41.065 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-24T23:32:41.065 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-24T23:32:41.065 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-24T23:32:41.065 INFO:teuthology.orchestra.run.smithi169.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-24T23:32:41.066 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-24T23:32:41.066 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-24T23:32:41.066 INFO:teuthology.orchestra.run.smithi169.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-24T23:32:41.066 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-24T23:32:41.066 INFO:teuthology.orchestra.run.smithi169.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-24T23:32:41.066 INFO:teuthology.orchestra.run.smithi169.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-24T23:32:41.066 INFO:teuthology.orchestra.run.smithi169.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-24T23:32:41.066 INFO:teuthology.orchestra.run.smithi169.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-24T23:32:41.066 INFO:teuthology.orchestra.run.smithi169.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-24T23:32:41.066 INFO:teuthology.orchestra.run.smithi169.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-24T23:32:41.066 INFO:teuthology.orchestra.run.smithi169.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-24T23:32:41.066 INFO:teuthology.orchestra.run.smithi169.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-24T23:32:41.066 INFO:teuthology.orchestra.run.smithi169.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-24T23:32:41.067 INFO:teuthology.orchestra.run.smithi169.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-24T23:32:41.067 INFO:teuthology.orchestra.run.smithi169.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-24T23:32:41.067 INFO:teuthology.orchestra.run.smithi169.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-24T23:32:41.067 INFO:teuthology.orchestra.run.smithi169.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-24T23:32:41.067 INFO:teuthology.orchestra.run.smithi169.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-24T23:32:41.067 INFO:teuthology.orchestra.run.smithi169.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-24T23:32:41.067 INFO:teuthology.orchestra.run.smithi169.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-24T23:32:41.067 INFO:teuthology.orchestra.run.smithi169.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-24T23:32:41.067 INFO:teuthology.orchestra.run.smithi169.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-24T23:32:41.067 INFO:teuthology.orchestra.run.smithi169.stdout:Installing weak dependencies: 2024-04-24T23:32:41.067 INFO:teuthology.orchestra.run.smithi169.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-24T23:32:41.067 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-24T23:32:41.067 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-04-24T23:32:41.067 INFO:teuthology.orchestra.run.smithi169.stdout:======================================================================================= 2024-04-24T23:32:41.068 INFO:teuthology.orchestra.run.smithi169.stdout:Install 148 Packages 2024-04-24T23:32:41.068 INFO:teuthology.orchestra.run.smithi169.stdout:Upgrade 4 Packages 2024-04-24T23:32:41.068 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-24T23:32:41.068 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 220 M 2024-04-24T23:32:41.068 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-04-24T23:32:41.086 INFO:teuthology.orchestra.run.smithi157.stdout:(27/152): rbd-nbd-19.0.0-2445.gd2c45d97.el9.x86 1.7 MB/s | 171 kB 00:00 2024-04-24T23:32:41.154 INFO:teuthology.orchestra.run.smithi157.stdout:(28/152): rbd-mirror-19.0.0-2445.gd2c45d97.el9. 15 MB/s | 3.1 MB 00:00 2024-04-24T23:32:41.171 INFO:teuthology.orchestra.run.smithi157.stdout:(29/152): ceph-grafana-dashboards-19.0.0-2445.g 377 kB/s | 32 kB 00:00 2024-04-24T23:32:41.213 INFO:teuthology.orchestra.run.smithi157.stdout:(30/152): ceph-mgr-cephadm-19.0.0-2445.gd2c45d9 2.4 MB/s | 141 kB 00:00 2024-04-24T23:32:41.722 INFO:teuthology.orchestra.run.smithi157.stdout:(31/152): ceph-mgr-dashboard-19.0.0-2445.gd2c45 6.4 MB/s | 3.5 MB 00:00 2024-04-24T23:32:41.813 INFO:teuthology.orchestra.run.smithi157.stdout:(32/152): ceph-mgr-diskprediction-local-19.0.0- 12 MB/s | 7.4 MB 00:00 2024-04-24T23:32:41.838 INFO:teuthology.orchestra.run.smithi157.stdout:(33/152): ceph-mgr-modules-core-19.0.0-2445.gd2 2.1 MB/s | 249 kB 00:00 2024-04-24T23:32:41.864 INFO:teuthology.orchestra.run.smithi157.stdout:(34/152): ceph-mgr-rook-19.0.0-2445.gd2c45d97.e 985 kB/s | 49 kB 00:00 2024-04-24T23:32:41.906 INFO:teuthology.orchestra.run.smithi157.stdout:(35/152): ceph-prometheus-alerts-19.0.0-2445.gd 229 kB/s | 15 kB 00:00 2024-04-24T23:32:41.931 INFO:teuthology.orchestra.run.smithi157.stdout:(36/152): ceph-volume-19.0.0-2445.gd2c45d97.el9 3.8 MB/s | 264 kB 00:00 2024-04-24T23:32:41.982 INFO:teuthology.orchestra.run.smithi157.stdout:(37/152): cephfs-top-19.0.0-2445.gd2c45d97.el9. 366 kB/s | 18 kB 00:00 2024-04-24T23:32:42.015 INFO:teuthology.orchestra.run.smithi157.stdout:(38/152): cephadm-19.0.0-2445.gd2c45d97.el9.noa 6.8 MB/s | 762 kB 00:00 2024-04-24T23:32:42.124 INFO:teuthology.orchestra.run.smithi157.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 567 kB/s | 80 kB 00:00 2024-04-24T23:32:42.240 INFO:teuthology.orchestra.run.smithi157.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.6 MB/s | 195 kB 00:00 2024-04-24T23:32:42.302 INFO:teuthology.orchestra.run.smithi169.stdout:(1/152): ceph-19.0.0-2445.gd2c45d97.el9.x86_64. 15 kB/s | 6.6 kB 00:00 2024-04-24T23:32:42.324 INFO:teuthology.orchestra.run.smithi157.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-04-24T23:32:42.349 INFO:teuthology.orchestra.run.smithi157.stdout:(42/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 803 kB 00:00 2024-04-24T23:32:42.408 INFO:teuthology.orchestra.run.smithi157.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-04-24T23:32:42.458 INFO:teuthology.orchestra.run.smithi157.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 2.7 MB/s | 135 kB 00:00 2024-04-24T23:32:42.509 INFO:teuthology.orchestra.run.smithi157.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-24T23:32:42.518 INFO:teuthology.orchestra.run.smithi169.stdout:(2/152): ceph-fuse-19.0.0-2445.gd2c45d97.el9.x8 4.0 MB/s | 890 kB 00:00 2024-04-24T23:32:42.569 INFO:teuthology.orchestra.run.smithi169.stdout:(3/152): ceph-immutable-object-cache-19.0.0-244 2.8 MB/s | 145 kB 00:00 2024-04-24T23:32:42.609 INFO:teuthology.orchestra.run.smithi157.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.1 MB/s | 215 kB 00:00 2024-04-24T23:32:42.651 INFO:teuthology.orchestra.run.smithi157.stdout:(47/152): python3-cryptography-36.0.1-4.el9.x86 3.8 MB/s | 1.2 MB 00:00 2024-04-24T23:32:42.759 INFO:teuthology.orchestra.run.smithi157.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 4.5 MB/s | 689 kB 00:00 2024-04-24T23:32:42.820 INFO:teuthology.orchestra.run.smithi157.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 1.7 MB/s | 107 kB 00:00 2024-04-24T23:32:42.845 INFO:teuthology.orchestra.run.smithi157.stdout:(50/152): bison-3.7.4-5.el9.x86_64.rpm 4.7 MB/s | 943 kB 00:00 2024-04-24T23:32:42.862 INFO:teuthology.orchestra.run.smithi157.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 516 kB/s | 21 kB 00:00 2024-04-24T23:32:42.886 INFO:teuthology.orchestra.run.smithi169.stdout:(4/152): ceph-mds-19.0.0-2445.gd2c45d97.el9.x86 7.5 MB/s | 2.4 MB 00:00 2024-04-24T23:32:42.896 INFO:teuthology.orchestra.run.smithi157.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 980 kB/s | 33 kB 00:00 2024-04-24T23:32:42.922 INFO:teuthology.orchestra.run.smithi157.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 4.1 MB/s | 313 kB 00:00 2024-04-24T23:32:42.964 INFO:teuthology.orchestra.run.smithi157.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 438 kB/s | 18 kB 00:00 2024-04-24T23:32:43.005 INFO:teuthology.orchestra.run.smithi157.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 268 kB/s | 11 kB 00:00 2024-04-24T23:32:43.056 INFO:teuthology.orchestra.run.smithi157.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-04-24T23:32:43.106 INFO:teuthology.orchestra.run.smithi157.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 906 kB/s | 45 kB 00:00 2024-04-24T23:32:43.144 INFO:teuthology.orchestra.run.smithi169.stdout:(5/152): ceph-mgr-19.0.0-2445.gd2c45d97.el9.x86 5.8 MB/s | 1.5 MB 00:00 2024-04-24T23:32:43.231 INFO:teuthology.orchestra.run.smithi157.stdout:(58/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 8.9 MB/s | 3.0 MB 00:00 2024-04-24T23:32:43.256 INFO:teuthology.orchestra.run.smithi157.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 4.3 MB/s | 662 kB 00:00 2024-04-24T23:32:43.307 INFO:teuthology.orchestra.run.smithi157.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 7.6 MB/s | 582 kB 00:00 2024-04-24T23:32:43.332 INFO:teuthology.orchestra.run.smithi157.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 338 kB/s | 25 kB 00:00 2024-04-24T23:32:43.357 INFO:teuthology.orchestra.run.smithi157.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 947 kB/s | 47 kB 00:00 2024-04-24T23:32:43.416 INFO:teuthology.orchestra.run.smithi157.stdout:(63/152): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 188 kB 00:00 2024-04-24T23:32:43.441 INFO:teuthology.orchestra.run.smithi157.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-04-24T23:32:43.475 INFO:teuthology.orchestra.run.smithi157.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 605 kB/s | 35 kB 00:00 2024-04-24T23:32:43.742 INFO:teuthology.orchestra.run.smithi157.stdout:(66/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-04-24T23:32:43.786 INFO:teuthology.orchestra.run.smithi169.stdout:(6/152): ceph-mon-19.0.0-2445.gd2c45d97.el9.x86 7.3 MB/s | 4.7 MB 00:00 2024-04-24T23:32:43.875 INFO:teuthology.orchestra.run.smithi157.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 11 MB/s | 4.7 MB 00:00 2024-04-24T23:32:43.909 INFO:teuthology.orchestra.run.smithi157.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 9.5 MB/s | 1.6 MB 00:00 2024-04-24T23:32:43.951 INFO:teuthology.orchestra.run.smithi157.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 3.1 MB/s | 133 kB 00:00 2024-04-24T23:32:43.976 INFO:teuthology.orchestra.run.smithi157.stdout:(70/152): postgresql-contrib-13.14-1.el9.x86_64 8.9 MB/s | 911 kB 00:00 2024-04-24T23:32:44.068 INFO:teuthology.orchestra.run.smithi157.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 11 MB/s | 1.0 MB 00:00 2024-04-24T23:32:44.119 INFO:teuthology.orchestra.run.smithi157.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-04-24T23:32:44.195 INFO:teuthology.orchestra.run.smithi169.stdout:(7/152): ceph-base-19.0.0-2445.gd2c45d97.el9.x8 2.3 MB/s | 5.4 MB 00:02 2024-04-24T23:32:44.235 INFO:teuthology.orchestra.run.smithi157.stdout:(73/152): postgresql-server-13.14-1.el9.x86_64. 20 MB/s | 5.8 MB 00:00 2024-04-24T23:32:44.286 INFO:teuthology.orchestra.run.smithi157.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 4.8 MB/s | 244 kB 00:00 2024-04-24T23:32:44.327 INFO:teuthology.orchestra.run.smithi157.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 5.8 MB/s | 248 kB 00:00 2024-04-24T23:32:44.369 INFO:teuthology.orchestra.run.smithi157.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 614 kB/s | 26 kB 00:00 2024-04-24T23:32:44.412 INFO:teuthology.orchestra.run.smithi157.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 457 kB/s | 19 kB 00:00 2024-04-24T23:32:44.454 INFO:teuthology.orchestra.run.smithi157.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 4.0 MB/s | 172 kB 00:00 2024-04-24T23:32:44.524 INFO:teuthology.orchestra.run.smithi157.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 492 kB/s | 35 kB 00:00 2024-04-24T23:32:44.596 INFO:teuthology.orchestra.run.smithi157.stdout:(80/152): python3-babel-2.9.1-2.el9.noarch.rpm 12 MB/s | 6.0 MB 00:00 2024-04-24T23:32:44.646 INFO:teuthology.orchestra.run.smithi157.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-04-24T23:32:44.705 INFO:teuthology.orchestra.run.smithi157.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-04-24T23:32:44.780 INFO:teuthology.orchestra.run.smithi157.stdout:(83/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 20 MB/s | 5.1 MB 00:00 2024-04-24T23:32:44.805 INFO:teuthology.orchestra.run.smithi157.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 770 kB/s | 77 kB 00:00 2024-04-24T23:32:44.831 INFO:teuthology.orchestra.run.smithi157.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 842 kB/s | 42 kB 00:00 2024-04-24T23:32:44.856 INFO:teuthology.orchestra.run.smithi157.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-04-24T23:32:44.890 INFO:teuthology.orchestra.run.smithi157.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 159 kB 00:00 2024-04-24T23:32:44.915 INFO:teuthology.orchestra.run.smithi157.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 4.7 MB/s | 279 kB 00:00 2024-04-24T23:32:44.940 INFO:teuthology.orchestra.run.smithi157.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-24T23:32:44.965 INFO:teuthology.orchestra.run.smithi157.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-24T23:32:45.015 INFO:teuthology.orchestra.run.smithi157.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 832 kB/s | 42 kB 00:00 2024-04-24T23:32:45.066 INFO:teuthology.orchestra.run.smithi157.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 880 kB/s | 44 kB 00:00 2024-04-24T23:32:45.124 INFO:teuthology.orchestra.run.smithi157.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-04-24T23:32:45.175 INFO:teuthology.orchestra.run.smithi157.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 714 kB/s | 36 kB 00:00 2024-04-24T23:32:45.225 INFO:teuthology.orchestra.run.smithi157.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 950 kB/s | 47 kB 00:00 2024-04-24T23:32:45.284 INFO:teuthology.orchestra.run.smithi157.stdout:(96/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.1 MB/s | 305 kB 00:00 2024-04-24T23:32:45.337 INFO:teuthology.orchestra.run.smithi169.stdout:(8/152): ceph-osd-19.0.0-2445.gd2c45d97.el9.x86 11 MB/s | 17 MB 00:01 2024-04-24T23:32:45.395 INFO:teuthology.orchestra.run.smithi169.stdout:(9/152): ceph-selinux-19.0.0-2445.gd2c45d97.el9 432 kB/s | 25 kB 00:00 2024-04-24T23:32:45.468 INFO:teuthology.orchestra.run.smithi157.stdout:(97/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 31 MB/s | 16 MB 00:00 2024-04-24T23:32:45.493 INFO:teuthology.orchestra.run.smithi157.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 277 kB/s | 58 kB 00:00 2024-04-24T23:32:45.524 INFO:teuthology.orchestra.run.smithi157.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-04-24T23:32:45.543 INFO:teuthology.orchestra.run.smithi157.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-24T23:32:45.569 INFO:teuthology.orchestra.run.smithi157.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 890 kB/s | 45 kB 00:00 2024-04-24T23:32:45.771 INFO:teuthology.orchestra.run.smithi169.stdout:(10/152): ceph-radosgw-19.0.0-2445.gd2c45d97.el 6.5 MB/s | 10 MB 00:01 2024-04-24T23:32:45.802 INFO:teuthology.orchestra.run.smithi157.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 86 kB/s | 22 kB 00:00 2024-04-24T23:32:45.854 INFO:teuthology.orchestra.run.smithi169.stdout:(11/152): cephfs-mirror-19.0.0-2445.gd2c45d97.e 2.6 MB/s | 222 kB 00:00 2024-04-24T23:32:45.979 INFO:teuthology.orchestra.run.smithi169.stdout:(12/152): cephfs-shell-19.0.0-2445.gd2c45d97.el 216 kB/s | 27 kB 00:00 2024-04-24T23:32:46.003 INFO:teuthology.orchestra.run.smithi157.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 1.9 MB/s | 863 kB 00:00 2024-04-24T23:32:46.063 INFO:teuthology.orchestra.run.smithi169.stdout:(13/152): libcephfs-devel-19.0.0-2445.gd2c45d97 380 kB/s | 32 kB 00:00 2024-04-24T23:32:46.364 INFO:teuthology.orchestra.run.smithi169.stdout:(14/152): ceph-common-19.0.0-2445.gd2c45d97.el9 4.7 MB/s | 21 MB 00:04 2024-04-24T23:32:46.397 INFO:teuthology.orchestra.run.smithi169.stdout:(15/152): libcephfs2-19.0.0-2445.gd2c45d97.el9. 2.2 MB/s | 745 kB 00:00 2024-04-24T23:32:46.439 INFO:teuthology.orchestra.run.smithi169.stdout:(16/152): libcephsqlite-19.0.0-2445.gd2c45d97.e 2.1 MB/s | 163 kB 00:00 2024-04-24T23:32:46.472 INFO:teuthology.orchestra.run.smithi169.stdout:(17/152): librados-devel-19.0.0-2445.gd2c45d97. 1.6 MB/s | 127 kB 00:00 2024-04-24T23:32:46.589 INFO:teuthology.orchestra.run.smithi169.stdout:(18/152): libradosstriper1-19.0.0-2445.gd2c45d9 3.3 MB/s | 501 kB 00:00 2024-04-24T23:32:46.690 INFO:teuthology.orchestra.run.smithi169.stdout:(19/152): python3-ceph-argparse-19.0.0-2445.gd2 450 kB/s | 45 kB 00:00 2024-04-24T23:32:46.778 INFO:teuthology.orchestra.run.smithi157.stdout:(104/152): ceph-test-19.0.0-2445.gd2c45d97.el9. 6.9 MB/s | 48 MB 00:07 2024-04-24T23:32:46.798 INFO:teuthology.orchestra.run.smithi169.stdout:(20/152): python3-ceph-common-19.0.0-2445.gd2c4 1.2 MB/s | 133 kB 00:00 2024-04-24T23:32:46.803 INFO:teuthology.orchestra.run.smithi157.stdout:(105/152): abseil-cpp-20211102.0-3.el9.x86_64.r 548 kB/s | 548 kB 00:01 2024-04-24T23:32:46.829 INFO:teuthology.orchestra.run.smithi157.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 374 kB/s | 309 kB 00:00 2024-04-24T23:32:46.924 INFO:teuthology.orchestra.run.smithi169.stdout:(21/152): python3-cephfs-19.0.0-2445.gd2c45d97. 1.3 MB/s | 162 kB 00:00 2024-04-24T23:32:46.970 INFO:teuthology.orchestra.run.smithi157.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 183 kB/s | 26 kB 00:00 2024-04-24T23:32:47.096 INFO:teuthology.orchestra.run.smithi157.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 389 kB/s | 49 kB 00:00 2024-04-24T23:32:47.112 INFO:teuthology.orchestra.run.smithi157.stdout:(109/152): grpc-data-1.46.7-10.el9.noarch.rpm 58 kB/s | 19 kB 00:00 2024-04-24T23:32:47.196 INFO:teuthology.orchestra.run.smithi157.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 671 kB/s | 67 kB 00:00 2024-04-24T23:32:47.207 INFO:teuthology.orchestra.run.smithi169.stdout:(22/152): librgw2-19.0.0-2445.gd2c45d97.el9.x86 7.2 MB/s | 5.3 MB 00:00 2024-04-24T23:32:47.274 INFO:teuthology.orchestra.run.smithi169.stdout:(23/152): python3-rados-19.0.0-2445.gd2c45d97.e 917 kB/s | 321 kB 00:00 2024-04-24T23:32:47.299 INFO:teuthology.orchestra.run.smithi169.stdout:(24/152): python3-rbd-19.0.0-2445.gd2c45d97.el9 3.2 MB/s | 299 kB 00:00 2024-04-24T23:32:47.391 INFO:teuthology.orchestra.run.smithi169.stdout:(25/152): python3-rgw-19.0.0-2445.gd2c45d97.el9 851 kB/s | 99 kB 00:00 2024-04-24T23:32:47.413 INFO:teuthology.orchestra.run.smithi157.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 503 kB/s | 151 kB 00:00 2024-04-24T23:32:47.416 INFO:teuthology.orchestra.run.smithi169.stdout:(26/152): rbd-fuse-19.0.0-2445.gd2c45d97.el9.x8 730 kB/s | 85 kB 00:00 2024-04-24T23:32:47.542 INFO:teuthology.orchestra.run.smithi169.stdout:(27/152): rbd-nbd-19.0.0-2445.gd2c45d97.el9.x86 1.3 MB/s | 171 kB 00:00 2024-04-24T23:32:47.650 INFO:teuthology.orchestra.run.smithi169.stdout:(28/152): ceph-grafana-dashboards-19.0.0-2445.g 293 kB/s | 32 kB 00:00 2024-04-24T23:32:47.680 INFO:teuthology.orchestra.run.smithi157.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.7 MB/s | 838 kB 00:00 2024-04-24T23:32:47.726 INFO:teuthology.orchestra.run.smithi169.stdout:(29/152): ceph-mgr-cephadm-19.0.0-2445.gd2c45d9 1.8 MB/s | 141 kB 00:00 2024-04-24T23:32:47.764 INFO:teuthology.orchestra.run.smithi157.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 522 kB/s | 43 kB 00:00 2024-04-24T23:32:47.789 INFO:teuthology.orchestra.run.smithi157.stdout:(114/152): python3-asyncssh-2.13.2-5.el9.noarch 1.4 MB/s | 548 kB 00:00 2024-04-24T23:32:47.864 INFO:teuthology.orchestra.run.smithi157.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 321 kB/s | 32 kB 00:00 2024-04-24T23:32:47.923 INFO:teuthology.orchestra.run.smithi157.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 112 kB/s | 15 kB 00:00 2024-04-24T23:32:47.956 INFO:teuthology.orchestra.run.smithi157.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.8 MB/s | 172 kB 00:00 2024-04-24T23:32:48.140 INFO:teuthology.orchestra.run.smithi157.stdout:(118/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.6 MB/s | 303 kB 00:00 2024-04-24T23:32:48.176 INFO:teuthology.orchestra.run.smithi169.stdout:(30/152): rbd-mirror-19.0.0-2445.gd2c45d97.el9. 3.9 MB/s | 3.1 MB 00:00 2024-04-24T23:32:48.182 INFO:teuthology.orchestra.run.smithi157.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 1.4 MB/s | 358 kB 00:00 2024-04-24T23:32:48.232 INFO:teuthology.orchestra.run.smithi157.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 385 kB/s | 35 kB 00:00 2024-04-24T23:32:48.293 INFO:teuthology.orchestra.run.smithi169.stdout:(31/152): ceph-mgr-dashboard-19.0.0-2445.gd2c45 6.2 MB/s | 3.5 MB 00:00 2024-04-24T23:32:48.349 INFO:teuthology.orchestra.run.smithi157.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 1.2 MB/s | 206 kB 00:00 2024-04-24T23:32:48.368 INFO:teuthology.orchestra.run.smithi169.stdout:(32/152): ceph-mgr-modules-core-19.0.0-2445.gd2 3.3 MB/s | 249 kB 00:00 2024-04-24T23:32:48.450 INFO:teuthology.orchestra.run.smithi157.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 1.4 MB/s | 144 kB 00:00 2024-04-24T23:32:48.516 INFO:teuthology.orchestra.run.smithi157.stdout:(123/152): libarrow-9.0.0-10.el9.x86_64.rpm 2.6 MB/s | 4.4 MB 00:01 2024-04-24T23:32:48.558 INFO:teuthology.orchestra.run.smithi157.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 98 kB/s | 11 kB 00:00 2024-04-24T23:32:48.609 INFO:teuthology.orchestra.run.smithi157.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 192 kB/s | 18 kB 00:00 2024-04-24T23:32:48.651 INFO:teuthology.orchestra.run.smithi157.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 251 kB/s | 23 kB 00:00 2024-04-24T23:32:48.701 INFO:teuthology.orchestra.run.smithi157.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 211 kB/s | 19 kB 00:00 2024-04-24T23:32:48.743 INFO:teuthology.orchestra.run.smithi157.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 214 kB/s | 20 kB 00:00 2024-04-24T23:32:48.835 INFO:teuthology.orchestra.run.smithi157.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 505 kB/s | 46 kB 00:00 2024-04-24T23:32:48.852 INFO:teuthology.orchestra.run.smithi169.stdout:(33/152): ceph-test-19.0.0-2445.gd2c45d97.el9.x 14 MB/s | 48 MB 00:03 2024-04-24T23:32:48.868 INFO:teuthology.orchestra.run.smithi169.stdout:(34/152): ceph-mgr-rook-19.0.0-2445.gd2c45d97.e 98 kB/s | 49 kB 00:00 2024-04-24T23:32:48.886 INFO:teuthology.orchestra.run.smithi169.stdout:(35/152): ceph-prometheus-alerts-19.0.0-2445.gd 462 kB/s | 15 kB 00:00 2024-04-24T23:32:48.927 INFO:teuthology.orchestra.run.smithi157.stdout:(130/152): python3-grpcio-1.46.7-10.el9.x86_64. 2.9 MB/s | 2.0 MB 00:00 2024-04-24T23:32:48.952 INFO:teuthology.orchestra.run.smithi157.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 673 kB/s | 79 kB 00:00 2024-04-24T23:32:48.969 INFO:teuthology.orchestra.run.smithi169.stdout:(36/152): cephadm-19.0.0-2445.gd2c45d97.el9.noa 8.9 MB/s | 762 kB 00:00 2024-04-24T23:32:48.997 INFO:teuthology.orchestra.run.smithi169.stdout:(37/152): ceph-volume-19.0.0-2445.gd2c45d97.el9 2.0 MB/s | 264 kB 00:00 2024-04-24T23:32:49.019 INFO:teuthology.orchestra.run.smithi157.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 628 kB/s | 58 kB 00:00 2024-04-24T23:32:49.022 INFO:teuthology.orchestra.run.smithi169.stdout:(38/152): cephfs-top-19.0.0-2445.gd2c45d97.el9. 349 kB/s | 18 kB 00:00 2024-04-24T23:32:49.053 INFO:teuthology.orchestra.run.smithi157.stdout:(133/152): python3-kubernetes-26.1.0-2.el9.noar 2.9 MB/s | 1.0 MB 00:00 2024-04-24T23:32:49.111 INFO:teuthology.orchestra.run.smithi157.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 179 kB/s | 16 kB 00:00 2024-04-24T23:32:49.139 INFO:teuthology.orchestra.run.smithi169.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 565 kB/s | 80 kB 00:00 2024-04-24T23:32:49.145 INFO:teuthology.orchestra.run.smithi157.stdout:(135/152): python3-pecan-1.4.2-3.el9.noarch.rpm 1.4 MB/s | 272 kB 00:00 2024-04-24T23:32:49.170 INFO:teuthology.orchestra.run.smithi157.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 766 kB/s | 90 kB 00:00 2024-04-24T23:32:49.204 INFO:teuthology.orchestra.run.smithi157.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 305 kB/s | 28 kB 00:00 2024-04-24T23:32:49.206 INFO:teuthology.orchestra.run.smithi169.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 195 kB 00:00 2024-04-24T23:32:49.238 INFO:teuthology.orchestra.run.smithi157.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 334 kB/s | 31 kB 00:00 2024-04-24T23:32:49.243 INFO:teuthology.orchestra.run.smithi169.stdout:(41/152): libgfortran-11.4.1-3.el9.x86_64.rpm 3.6 MB/s | 803 kB 00:00 2024-04-24T23:32:49.263 INFO:teuthology.orchestra.run.smithi157.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-04-24T23:32:49.265 INFO:teuthology.orchestra.run.smithi169.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.2 MB/s | 253 kB 00:00 2024-04-24T23:32:49.297 INFO:teuthology.orchestra.run.smithi157.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 642 kB/s | 59 kB 00:00 2024-04-24T23:32:49.324 INFO:teuthology.orchestra.run.smithi169.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.2 MB 00:00 2024-04-24T23:32:49.330 INFO:teuthology.orchestra.run.smithi157.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 388 kB/s | 36 kB 00:00 2024-04-24T23:32:49.349 INFO:teuthology.orchestra.run.smithi169.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 106 kB 00:00 2024-04-24T23:32:49.355 INFO:teuthology.orchestra.run.smithi157.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 553 kB/s | 51 kB 00:00 2024-04-24T23:32:49.374 INFO:teuthology.orchestra.run.smithi169.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-04-24T23:32:49.389 INFO:teuthology.orchestra.run.smithi157.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-04-24T23:32:49.391 INFO:teuthology.orchestra.run.smithi169.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.9 MB/s | 125 kB 00:00 2024-04-24T23:32:49.422 INFO:teuthology.orchestra.run.smithi157.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 973 kB/s | 90 kB 00:00 2024-04-24T23:32:49.424 INFO:teuthology.orchestra.run.smithi169.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-04-24T23:32:49.481 INFO:teuthology.orchestra.run.smithi157.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 217 kB/s | 20 kB 00:00 2024-04-24T23:32:49.523 INFO:teuthology.orchestra.run.smithi157.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-04-24T23:32:49.567 INFO:teuthology.orchestra.run.smithi157.stdout:(147/152): python3-werkzeug-2.0.3-3.el9.1.noarc 2.0 MB/s | 427 kB 00:00 2024-04-24T23:32:49.583 INFO:teuthology.orchestra.run.smithi169.stdout:(48/152): ceph-mgr-diskprediction-local-19.0.0- 5.3 MB/s | 7.4 MB 00:01 2024-04-24T23:32:49.882 INFO:teuthology.orchestra.run.smithi157.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 3.9 MB/s | 1.6 MB 00:00 2024-04-24T23:32:49.900 INFO:teuthology.orchestra.run.smithi169.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 338 kB/s | 107 kB 00:00 2024-04-24T23:32:50.017 INFO:teuthology.orchestra.run.smithi169.stdout:(50/152): automake-1.16.2-8.el9.noarch.rpm 1.1 MB/s | 689 kB 00:00 2024-04-24T23:32:50.042 INFO:teuthology.orchestra.run.smithi169.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 152 kB/s | 21 kB 00:00 2024-04-24T23:32:50.057 INFO:teuthology.orchestra.run.smithi157.stdout:(149/152): librbd1-19.0.0-2445.gd2c45d97.el9.x8 6.4 MB/s | 3.1 MB 00:00 2024-04-24T23:32:50.084 INFO:teuthology.orchestra.run.smithi169.stdout:(52/152): bison-3.7.4-5.el9.x86_64.rpm 1.4 MB/s | 943 kB 00:00 2024-04-24T23:32:50.116 INFO:teuthology.orchestra.run.smithi157.stdout:(150/152): librados2-19.0.0-2445.gd2c45d97.el9. 5.7 MB/s | 3.4 MB 00:00 2024-04-24T23:32:50.158 INFO:teuthology.orchestra.run.smithi157.stdout:(151/152): openssl-3.2.1-1.el9.x86_64.rpm 4.9 MB/s | 1.3 MB 00:00 2024-04-24T23:32:50.184 INFO:teuthology.orchestra.run.smithi169.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 230 kB/s | 33 kB 00:00 2024-04-24T23:32:50.302 INFO:teuthology.orchestra.run.smithi169.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 1.1 MB/s | 313 kB 00:00 2024-04-24T23:32:50.319 INFO:teuthology.orchestra.run.smithi169.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 136 kB/s | 18 kB 00:00 2024-04-24T23:32:50.333 INFO:teuthology.orchestra.run.smithi157.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 8.7 MB/s | 2.4 MB 00:00 2024-04-24T23:32:50.336 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:32:50.336 INFO:teuthology.orchestra.run.smithi157.stdout:Total 16 MB/s | 220 MB 00:14 2024-04-24T23:32:50.410 INFO:teuthology.orchestra.run.smithi169.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 103 kB/s | 11 kB 00:00 2024-04-24T23:32:50.577 INFO:teuthology.orchestra.run.smithi169.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 620 kB/s | 160 kB 00:00 2024-04-24T23:32:50.628 INFO:teuthology.orchestra.run.smithi169.stdout:(58/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.5 MB/s | 3.0 MB 00:00 2024-04-24T23:32:50.645 INFO:teuthology.orchestra.run.smithi169.stdout:(59/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 193 kB/s | 45 kB 00:00 2024-04-24T23:32:50.778 INFO:teuthology.orchestra.run.smithi169.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 191 kB/s | 25 kB 00:00 2024-04-24T23:32:50.922 INFO:teuthology.orchestra.run.smithi169.stdout:(61/152): libtool-2.4.6-45.el9.x86_64.rpm 1.9 MB/s | 582 kB 00:00 2024-04-24T23:32:50.945 INFO:teuthology.orchestra.run.smithi169.stdout:(62/152): librdkafka-1.6.1-102.el9.x86_64.rpm 1.8 MB/s | 662 kB 00:00 2024-04-24T23:32:50.979 INFO:teuthology.orchestra.run.smithi169.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 237 kB/s | 47 kB 00:00 2024-04-24T23:32:51.113 INFO:teuthology.orchestra.run.smithi169.stdout:(64/152): openblas-0.3.21-2.el9.x86_64.rpm 264 kB/s | 35 kB 00:00 2024-04-24T23:32:51.171 INFO:teuthology.orchestra.run.smithi169.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 834 kB/s | 188 kB 00:00 2024-04-24T23:32:51.197 INFO:teuthology.orchestra.run.smithi169.stdout:(66/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.0 MB/s | 292 kB 00:00 2024-04-24T23:32:51.423 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-04-24T23:32:51.568 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-04-24T23:32:51.568 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-04-24T23:32:51.647 INFO:teuthology.orchestra.run.smithi169.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 3.5 MB/s | 1.6 MB 00:00 2024-04-24T23:32:51.772 INFO:teuthology.orchestra.run.smithi169.stdout:(68/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 7.4 MB/s | 4.4 MB 00:00 2024-04-24T23:32:51.889 INFO:teuthology.orchestra.run.smithi169.stdout:(69/152): openblas-openmp-0.3.21-2.el9.x86_64.r 6.1 MB/s | 4.7 MB 00:00 2024-04-24T23:32:52.022 INFO:teuthology.orchestra.run.smithi169.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 533 kB/s | 133 kB 00:00 2024-04-24T23:32:52.098 INFO:teuthology.orchestra.run.smithi169.stdout:(71/152): postgresql-contrib-13.14-1.el9.x86_64 2.0 MB/s | 911 kB 00:00 2024-04-24T23:32:52.273 INFO:teuthology.orchestra.run.smithi169.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 500 kB/s | 87 kB 00:00 2024-04-24T23:32:52.523 INFO:teuthology.orchestra.run.smithi169.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 2.0 MB/s | 1.0 MB 00:00 2024-04-24T23:32:52.649 INFO:teuthology.orchestra.run.smithi169.stdout:(74/152): postgresql-server-13.14-1.el9.x86_64. 7.6 MB/s | 5.8 MB 00:00 2024-04-24T23:32:52.865 INFO:teuthology.orchestra.run.smithi169.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 714 kB/s | 244 kB 00:00 2024-04-24T23:32:52.932 INFO:teuthology.orchestra.run.smithi169.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 875 kB/s | 248 kB 00:00 2024-04-24T23:32:53.000 INFO:teuthology.orchestra.run.smithi169.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 192 kB/s | 26 kB 00:00 2024-04-24T23:32:53.066 INFO:teuthology.orchestra.run.smithi169.stdout:(78/152): python3-babel-2.9.1-2.el9.noarch.rpm 7.5 MB/s | 6.0 MB 00:00 2024-04-24T23:32:53.092 INFO:teuthology.orchestra.run.smithi169.stdout:(79/152): python3-jsonpointer-2.0-4.el9.noarch. 120 kB/s | 19 kB 00:00 2024-04-24T23:32:53.208 INFO:teuthology.orchestra.run.smithi169.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 245 kB/s | 35 kB 00:00 2024-04-24T23:32:53.234 INFO:teuthology.orchestra.run.smithi169.stdout:(81/152): python3-mako-1.1.4-6.el9.noarch.rpm 735 kB/s | 172 kB 00:00 2024-04-24T23:32:53.317 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-04-24T23:32:53.318 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-04-24T23:32:53.459 INFO:teuthology.orchestra.run.smithi169.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.0 MB/s | 250 kB 00:00 2024-04-24T23:32:53.484 INFO:teuthology.orchestra.run.smithi169.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 888 kB/s | 222 kB 00:00 2024-04-24T23:32:53.660 INFO:teuthology.orchestra.run.smithi169.stdout:(84/152): python3-prettytable-0.7.2-27.el9.noar 241 kB/s | 42 kB 00:00 2024-04-24T23:32:53.676 INFO:teuthology.orchestra.run.smithi169.stdout:(85/152): python3-packaging-20.9-5.el9.noarch.r 355 kB/s | 77 kB 00:00 2024-04-24T23:32:53.885 INFO:teuthology.orchestra.run.smithi169.stdout:(86/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 762 kB/s | 159 kB 00:00 2024-04-24T23:32:53.910 INFO:teuthology.orchestra.run.smithi169.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 1.1 MB/s | 269 kB 00:00 2024-04-24T23:32:54.144 INFO:teuthology.orchestra.run.smithi169.stdout:(88/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.8 MB/s | 5.1 MB 00:01 2024-04-24T23:32:54.169 INFO:teuthology.orchestra.run.smithi169.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 197 kB/s | 51 kB 00:00 2024-04-24T23:32:54.211 INFO:teuthology.orchestra.run.smithi169.stdout:(90/152): python3-pyasn1-modules-0.4.8-6.el9.no 856 kB/s | 279 kB 00:00 2024-04-24T23:32:54.411 INFO:teuthology.orchestra.run.smithi169.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 201 kB/s | 54 kB 00:00 2024-04-24T23:32:54.436 INFO:teuthology.orchestra.run.smithi169.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 185 kB/s | 42 kB 00:00 2024-04-24T23:32:54.620 INFO:teuthology.orchestra.run.smithi169.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 212 kB/s | 44 kB 00:00 2024-04-24T23:32:54.762 INFO:teuthology.orchestra.run.smithi169.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 253 kB/s | 36 kB 00:00 2024-04-24T23:32:54.788 INFO:teuthology.orchestra.run.smithi169.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 629 kB/s | 221 kB 00:00 2024-04-24T23:32:54.963 INFO:teuthology.orchestra.run.smithi169.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 236 kB/s | 47 kB 00:00 2024-04-24T23:32:55.121 INFO:teuthology.orchestra.run.smithi169.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 914 kB/s | 305 kB 00:00 2024-04-24T23:32:55.146 INFO:teuthology.orchestra.run.smithi169.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 315 kB/s | 58 kB 00:00 2024-04-24T23:32:55.330 INFO:teuthology.orchestra.run.smithi169.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 449 kB/s | 94 kB 00:00 2024-04-24T23:32:55.356 INFO:teuthology.orchestra.run.smithi169.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 305 kB/s | 64 kB 00:00 2024-04-24T23:32:55.514 INFO:teuthology.orchestra.run.smithi169.stdout:(101/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 12 MB/s | 16 MB 00:01 2024-04-24T23:32:55.573 INFO:teuthology.orchestra.run.smithi169.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 103 kB/s | 22 kB 00:00 2024-04-24T23:32:55.656 INFO:teuthology.orchestra.run.smithi169.stdout:(103/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 137 kB/s | 45 kB 00:00 2024-04-24T23:32:55.690 INFO:teuthology.orchestra.run.smithi169.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 4.8 MB/s | 863 kB 00:00 2024-04-24T23:32:55.990 INFO:teuthology.orchestra.run.smithi169.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 65 kB/s | 19 kB 00:00 2024-04-24T23:32:56.265 INFO:teuthology.orchestra.run.smithi169.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 507 kB/s | 309 kB 00:00 2024-04-24T23:32:56.391 INFO:teuthology.orchestra.run.smithi169.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 208 kB/s | 26 kB 00:00 2024-04-24T23:32:56.449 INFO:teuthology.orchestra.run.smithi169.stdout:(108/152): abseil-cpp-20211102.0-3.el9.x86_64.r 625 kB/s | 548 kB 00:00 2024-04-24T23:32:56.508 INFO:teuthology.orchestra.run.smithi169.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 415 kB/s | 49 kB 00:00 2024-04-24T23:32:56.575 INFO:teuthology.orchestra.run.smithi169.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 538 kB/s | 67 kB 00:00 2024-04-24T23:32:56.625 INFO:teuthology.orchestra.run.smithi169.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.3 MB/s | 151 kB 00:00 2024-04-24T23:32:56.834 INFO:teuthology.orchestra.run.smithi169.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 2.6 MB/s | 548 kB 00:00 2024-04-24T23:32:56.876 INFO:teuthology.orchestra.run.smithi169.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.7 MB/s | 838 kB 00:00 2024-04-24T23:32:56.984 INFO:teuthology.orchestra.run.smithi169.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 289 kB/s | 43 kB 00:00 2024-04-24T23:32:57.009 INFO:teuthology.orchestra.run.smithi169.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 241 kB/s | 32 kB 00:00 2024-04-24T23:32:57.102 INFO:teuthology.orchestra.run.smithi169.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 128 kB/s | 15 kB 00:00 2024-04-24T23:32:57.127 INFO:teuthology.orchestra.run.smithi169.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.4 MB/s | 172 kB 00:00 2024-04-24T23:32:57.244 INFO:teuthology.orchestra.run.smithi169.stdout:(118/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.5 MB/s | 303 kB 00:00 2024-04-24T23:32:57.303 INFO:teuthology.orchestra.run.smithi169.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 1.7 MB/s | 358 kB 00:00 2024-04-24T23:32:57.361 INFO:teuthology.orchestra.run.smithi169.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 301 kB/s | 35 kB 00:00 2024-04-24T23:32:57.428 INFO:teuthology.orchestra.run.smithi169.stdout:(121/152): libarrow-9.0.0-10.el9.x86_64.rpm 3.1 MB/s | 4.4 MB 00:01 2024-04-24T23:32:57.453 INFO:teuthology.orchestra.run.smithi169.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 1.3 MB/s | 206 kB 00:00 2024-04-24T23:32:57.553 INFO:teuthology.orchestra.run.smithi169.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-04-24T23:32:57.571 INFO:teuthology.orchestra.run.smithi169.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 91 kB/s | 11 kB 00:00 2024-04-24T23:32:57.671 INFO:teuthology.orchestra.run.smithi169.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 151 kB/s | 18 kB 00:00 2024-04-24T23:32:57.688 INFO:teuthology.orchestra.run.smithi169.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 198 kB/s | 23 kB 00:00 2024-04-24T23:32:57.780 INFO:teuthology.orchestra.run.smithi169.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 179 kB/s | 19 kB 00:00 2024-04-24T23:32:57.805 INFO:teuthology.orchestra.run.smithi169.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 168 kB/s | 20 kB 00:00 2024-04-24T23:32:57.930 INFO:teuthology.orchestra.run.smithi169.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 370 kB/s | 46 kB 00:00 2024-04-24T23:32:58.006 INFO:teuthology.orchestra.run.smithi169.stdout:(130/152): python3-grpcio-1.46.7-10.el9.x86_64. 3.2 MB/s | 2.0 MB 00:00 2024-04-24T23:32:58.056 INFO:teuthology.orchestra.run.smithi169.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 630 kB/s | 79 kB 00:00 2024-04-24T23:32:58.123 INFO:teuthology.orchestra.run.smithi169.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 494 kB/s | 58 kB 00:00 2024-04-24T23:32:58.190 INFO:teuthology.orchestra.run.smithi169.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2024-04-24T23:32:58.223 INFO:teuthology.orchestra.run.smithi169.stdout:(134/152): python3-kubernetes-26.1.0-2.el9.noar 2.3 MB/s | 1.0 MB 00:00 2024-04-24T23:32:58.249 INFO:teuthology.orchestra.run.smithi169.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 131 kB/s | 16 kB 00:00 2024-04-24T23:32:58.307 INFO:teuthology.orchestra.run.smithi169.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 768 kB/s | 90 kB 00:00 2024-04-24T23:32:58.341 INFO:teuthology.orchestra.run.smithi169.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 241 kB/s | 28 kB 00:00 2024-04-24T23:32:58.366 INFO:teuthology.orchestra.run.smithi169.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 262 kB/s | 31 kB 00:00 2024-04-24T23:32:58.425 INFO:teuthology.orchestra.run.smithi169.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.6 MB/s | 188 kB 00:00 2024-04-24T23:32:58.458 INFO:teuthology.orchestra.run.smithi169.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 505 kB/s | 59 kB 00:00 2024-04-24T23:32:58.484 INFO:teuthology.orchestra.run.smithi169.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 305 kB/s | 36 kB 00:00 2024-04-24T23:32:58.543 INFO:teuthology.orchestra.run.smithi169.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 435 kB/s | 51 kB 00:00 2024-04-24T23:32:58.576 INFO:teuthology.orchestra.run.smithi169.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-04-24T23:32:58.601 INFO:teuthology.orchestra.run.smithi169.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 766 kB/s | 90 kB 00:00 2024-04-24T23:32:58.660 INFO:teuthology.orchestra.run.smithi169.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2024-04-24T23:32:58.685 INFO:teuthology.orchestra.run.smithi169.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 183 kB/s | 20 kB 00:00 2024-04-24T23:32:58.718 INFO:teuthology.orchestra.run.smithi169.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.6 MB/s | 192 kB 00:00 2024-04-24T23:32:58.969 INFO:teuthology.orchestra.run.smithi169.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 5.1 MB/s | 1.6 MB 00:00 2024-04-24T23:32:59.194 INFO:teuthology.orchestra.run.smithi169.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 6.0 MB/s | 1.3 MB 00:00 2024-04-24T23:32:59.295 INFO:teuthology.orchestra.run.smithi169.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 24 MB/s | 2.4 MB 00:00 2024-04-24T23:32:59.353 INFO:teuthology.orchestra.run.smithi169.stdout:(151/152): librados2-19.0.0-2445.gd2c45d97.el9. 5.1 MB/s | 3.4 MB 00:00 2024-04-24T23:32:59.435 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-04-24T23:32:59.520 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-24T23:32:59.557 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-24T23:32:59.611 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-24T23:32:59.654 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-24T23:32:59.679 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-24T23:32:59.722 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-24T23:32:59.745 INFO:teuthology.orchestra.run.smithi169.stdout:(152/152): librbd1-19.0.0-2445.gd2c45d97.el9.x8 3.1 MB/s | 3.1 MB 00:01 2024-04-24T23:32:59.748 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:32:59.748 INFO:teuthology.orchestra.run.smithi169.stdout:Total 12 MB/s | 220 MB 00:18 2024-04-24T23:33:00.052 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-24T23:33:00.164 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:33:00.285 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:33:00.323 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:33:00.415 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:33:00.452 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-24T23:33:00.600 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-24T23:33:00.650 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-24T23:33:00.677 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:33:00.781 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:33:00.803 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-24T23:33:00.822 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-04-24T23:33:00.866 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-24T23:33:00.888 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T23:33:00.911 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T23:33:00.968 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-04-24T23:33:00.968 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-04-24T23:33:01.095 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T23:33:01.128 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T23:33:01.235 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-24T23:33:01.333 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-24T23:33:01.370 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-24T23:33:01.405 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-24T23:33:01.479 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-24T23:33:01.521 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-24T23:33:01.565 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-24T23:33:01.605 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-24T23:33:01.633 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-24T23:33:01.704 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-24T23:33:01.764 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-24T23:33:01.795 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-24T23:33:01.847 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-24T23:33:01.923 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-24T23:33:01.957 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 31/156 2024-04-24T23:33:01.992 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 32/156 2024-04-24T23:33:02.021 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 33/156 2024-04-24T23:33:02.090 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-24T23:33:02.135 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-24T23:33:02.209 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-24T23:33:02.342 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-24T23:33:02.396 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-24T23:33:02.827 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-04-24T23:33:02.827 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-04-24T23:33:02.916 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-24T23:33:03.000 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-24T23:33:03.046 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-24T23:33:03.088 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-24T23:33:03.127 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 43/156 2024-04-24T23:33:03.168 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-24T23:33:03.321 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-24T23:33:03.380 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-24T23:33:03.434 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-24T23:33:03.507 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-24T23:33:03.538 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-24T23:33:03.575 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-24T23:33:03.622 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-24T23:33:03.658 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-24T23:33:03.701 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-24T23:33:03.743 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-24T23:33:03.881 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-24T23:33:03.924 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-24T23:33:03.954 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-24T23:33:03.995 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-24T23:33:04.091 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-24T23:33:04.120 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-24T23:33:04.160 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-24T23:33:04.186 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-24T23:33:04.211 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-24T23:33:04.285 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-24T23:33:04.414 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-24T23:33:04.443 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-24T23:33:04.471 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-24T23:33:04.505 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-24T23:33:04.544 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-24T23:33:04.593 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-24T23:33:04.623 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-24T23:33:04.657 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-24T23:33:04.691 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-24T23:33:04.732 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-24T23:33:04.767 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-24T23:33:04.845 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-24T23:33:04.905 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-24T23:33:05.155 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-24T23:33:05.278 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-24T23:33:05.388 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-24T23:33:05.439 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-24T23:33:06.160 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-24T23:33:06.211 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-24T23:33:06.244 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-24T23:33:06.342 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-24T23:33:06.393 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-24T23:33:06.480 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-24T23:33:06.521 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-24T23:33:06.699 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-24T23:33:06.736 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-24T23:33:06.813 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-24T23:33:07.451 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-24T23:33:07.581 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-24T23:33:08.773 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-04-24T23:33:08.785 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-24T23:33:08.848 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-24T23:33:08.865 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-24T23:33:08.886 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-24T23:33:08.893 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-24T23:33:08.926 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-24T23:33:08.948 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-24T23:33:08.982 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-24T23:33:09.008 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-24T23:33:09.050 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-24T23:33:09.246 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-24T23:33:09.280 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-24T23:33:09.371 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-24T23:33:09.402 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 100/156 2024-04-24T23:33:09.432 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-24T23:33:09.438 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-24T23:33:09.492 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:33:09.597 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:33:09.626 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:33:09.718 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:33:09.748 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-24T23:33:09.897 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-24T23:33:09.907 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-24T23:33:09.933 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-24T23:33:09.952 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-24T23:33:09.981 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:33:10.005 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-24T23:33:10.035 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 104/156 2024-04-24T23:33:10.084 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:33:10.107 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-24T23:33:10.169 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-24T23:33:10.191 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T23:33:10.211 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T23:33:10.366 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T23:33:10.399 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T23:33:10.506 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-24T23:33:10.594 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-24T23:33:10.662 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-24T23:33:10.699 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-24T23:33:10.765 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-24T23:33:10.808 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-24T23:33:10.851 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-24T23:33:10.883 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-24T23:33:10.911 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-24T23:33:10.982 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-24T23:33:11.034 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-24T23:33:11.064 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-24T23:33:11.108 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-24T23:33:11.184 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-24T23:33:11.219 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 31/156 2024-04-24T23:33:11.262 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 32/156 2024-04-24T23:33:11.291 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 33/156 2024-04-24T23:33:11.360 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-24T23:33:11.405 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-24T23:33:11.479 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-24T23:33:11.623 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-24T23:33:11.675 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-24T23:33:12.200 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-24T23:33:12.286 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-24T23:33:12.296 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:33:12.315 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-24T23:33:12.350 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:33:12.358 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-24T23:33:12.396 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 43/156 2024-04-24T23:33:12.438 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-24T23:33:12.593 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-24T23:33:12.649 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-24T23:33:12.704 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-24T23:33:12.777 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-24T23:33:12.816 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-24T23:33:12.853 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-24T23:33:12.890 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:33:12.900 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-24T23:33:12.935 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-24T23:33:12.945 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-24T23:33:12.979 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-24T23:33:13.020 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-24T23:33:13.038 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-24T23:33:13.038 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-24T23:33:13.038 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-24T23:33:13.038 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:33:13.071 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-24T23:33:13.158 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-24T23:33:13.209 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-24T23:33:13.240 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-24T23:33:13.281 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-24T23:33:13.375 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-24T23:33:13.414 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-24T23:33:13.454 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-24T23:33:13.480 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-24T23:33:13.504 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-24T23:33:13.579 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-24T23:33:13.709 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-24T23:33:13.746 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-24T23:33:13.773 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-24T23:33:13.807 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-24T23:33:13.847 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-24T23:33:13.878 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-24T23:33:13.908 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-24T23:33:13.943 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-24T23:33:13.976 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-24T23:33:14.018 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-24T23:33:14.061 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-24T23:33:14.156 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-24T23:33:14.199 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-24T23:33:14.239 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-24T23:33:14.286 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-24T23:33:14.385 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-24T23:33:14.642 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-24T23:33:15.412 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-24T23:33:15.471 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-24T23:33:15.513 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-24T23:33:15.615 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-24T23:33:15.678 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-24T23:33:15.767 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-24T23:33:15.798 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-24T23:33:15.976 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-24T23:33:16.004 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-24T23:33:16.081 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-24T23:33:16.729 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-24T23:33:16.861 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-24T23:33:18.093 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-24T23:33:18.184 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-24T23:33:18.211 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-24T23:33:18.244 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-24T23:33:18.556 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-24T23:33:18.607 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-24T23:33:18.698 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-24T23:33:18.720 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 100/156 2024-04-24T23:33:18.765 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-24T23:33:19.234 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-24T23:33:19.260 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-24T23:33:19.331 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-24T23:33:19.362 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 104/156 2024-04-24T23:33:21.611 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:33:21.658 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:33:22.200 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:33:22.245 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-24T23:33:22.354 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-24T23:33:22.354 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-24T23:33:22.354 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-24T23:33:22.355 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-24T23:33:22.388 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-24T23:33:24.456 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-24T23:33:24.456 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /sys 2024-04-24T23:33:24.456 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /proc 2024-04-24T23:33:24.456 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /mnt 2024-04-24T23:33:24.456 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /var/tmp 2024-04-24T23:33:24.456 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /home 2024-04-24T23:33:24.456 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /root 2024-04-24T23:33:24.456 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /tmp 2024-04-24T23:33:24.456 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:33:24.634 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-24T23:33:24.649 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-24T23:33:24.649 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:33:24.650 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-24T23:33:24.650 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:33:26.146 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-24T23:33:26.171 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-24T23:33:26.171 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:33:26.171 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-24T23:33:26.171 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T23:33:26.171 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T23:33:26.171 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:33:26.404 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-24T23:33:26.426 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-24T23:33:26.426 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:33:26.426 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-24T23:33:26.426 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T23:33:26.427 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T23:33:26.427 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:33:26.871 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-24T23:33:26.893 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-24T23:33:26.894 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:33:26.894 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-24T23:33:26.894 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T23:33:26.894 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T23:33:26.894 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:33:27.002 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-24T23:33:27.058 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-24T23:33:27.113 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-24T23:33:27.302 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-24T23:33:27.347 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-24T23:33:27.425 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-24T23:33:27.514 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T23:33:27.654 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T23:33:27.697 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-24T23:33:27.729 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-24T23:33:27.757 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-24T23:33:28.197 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-24T23:33:28.231 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-24T23:33:28.262 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 124/156 2024-04-24T23:33:28.324 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 125/156 2024-04-24T23:33:28.353 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-24T23:33:29.381 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-24T23:33:29.422 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-24T23:33:30.387 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-24T23:33:30.414 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-24T23:33:30.516 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-24T23:33:30.685 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 129/156 2024-04-24T23:33:30.716 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-24T23:33:30.739 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-24T23:33:30.739 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:33:30.739 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-24T23:33:30.739 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T23:33:30.739 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T23:33:30.740 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:33:30.783 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-24T23:33:30.794 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-24T23:33:30.893 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 132/156 2024-04-24T23:33:30.939 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 133/156 2024-04-24T23:33:31.835 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-24T23:33:31.878 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-24T23:33:31.900 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-24T23:33:31.901 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:33:31.901 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-24T23:33:31.901 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T23:33:31.901 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T23:33:31.901 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:33:31.945 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-24T23:33:31.967 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-24T23:33:31.967 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:33:31.967 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-24T23:33:31.967 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:33:32.021 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-24T23:33:32.043 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-24T23:33:32.043 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:33:32.043 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-24T23:33:32.043 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T23:33:32.044 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T23:33:32.044 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:33:32.346 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-24T23:33:32.369 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-24T23:33:32.369 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:33:32.369 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-24T23:33:32.369 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T23:33:32.369 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T23:33:32.369 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:33:33.970 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-24T23:33:33.971 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /sys 2024-04-24T23:33:33.971 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /proc 2024-04-24T23:33:33.971 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /mnt 2024-04-24T23:33:33.971 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /var/tmp 2024-04-24T23:33:33.971 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /home 2024-04-24T23:33:33.971 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /root 2024-04-24T23:33:33.971 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /tmp 2024-04-24T23:33:33.971 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-24T23:33:34.133 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-24T23:33:34.148 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-24T23:33:34.148 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:33:34.148 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-24T23:33:34.148 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-24T23:33:35.646 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-24T23:33:35.671 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-24T23:33:35.671 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:33:35.671 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-24T23:33:35.671 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T23:33:35.671 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T23:33:35.671 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-24T23:33:35.888 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-24T23:33:35.910 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-24T23:33:35.910 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:33:35.910 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-24T23:33:35.910 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T23:33:35.910 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T23:33:35.910 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-24T23:33:36.347 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-24T23:33:36.369 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-24T23:33:36.369 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:33:36.369 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-24T23:33:36.369 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T23:33:36.369 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T23:33:36.369 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-24T23:33:36.469 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-24T23:33:36.517 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-24T23:33:36.573 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-24T23:33:36.759 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-24T23:33:36.814 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-24T23:33:36.893 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-24T23:33:36.908 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 139/156 2024-04-24T23:33:36.950 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 140/156 2024-04-24T23:33:36.980 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T23:33:36.990 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 141/156 2024-04-24T23:33:37.023 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-24T23:33:37.050 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-24T23:33:37.115 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T23:33:37.156 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-24T23:33:37.188 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-24T23:33:37.216 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-24T23:33:37.233 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 144/156 2024-04-24T23:33:37.271 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-24T23:33:37.301 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 146/156 2024-04-24T23:33:37.640 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-24T23:33:37.673 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-24T23:33:37.705 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 124/156 2024-04-24T23:33:37.767 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 125/156 2024-04-24T23:33:37.804 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-24T23:33:37.890 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:33:37.932 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:33:37.952 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:33:38.793 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 148/156 2024-04-24T23:33:38.833 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-24T23:33:38.864 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-24T23:33:38.870 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-24T23:33:38.911 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-24T23:33:38.993 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-24T23:33:39.033 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-24T23:33:39.033 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T23:33:39.109 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T23:33:39.142 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T23:33:39.142 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T23:33:39.216 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T23:33:39.282 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:33:39.827 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-24T23:33:39.857 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-24T23:33:39.957 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-24T23:33:40.120 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 129/156 2024-04-24T23:33:40.150 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-24T23:33:40.172 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-24T23:33:40.172 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:33:40.173 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-24T23:33:40.173 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T23:33:40.173 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T23:33:40.173 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-24T23:33:40.217 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-24T23:33:40.228 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-24T23:33:40.335 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 132/156 2024-04-24T23:33:40.390 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 133/156 2024-04-24T23:33:41.286 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-24T23:33:41.320 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-24T23:33:41.343 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-24T23:33:41.343 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:33:41.343 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-24T23:33:41.343 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T23:33:41.343 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T23:33:41.343 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-24T23:33:41.388 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-24T23:33:41.409 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-24T23:33:41.409 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:33:41.409 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-24T23:33:41.409 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-24T23:33:41.486 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-24T23:33:41.508 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-24T23:33:41.508 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:33:41.508 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-24T23:33:41.508 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T23:33:41.508 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T23:33:41.509 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-24T23:33:41.814 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-24T23:33:41.836 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-24T23:33:41.836 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:33:41.836 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-24T23:33:41.836 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T23:33:41.836 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T23:33:41.836 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-24T23:33:44.993 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:33:44.993 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/156 2024-04-24T23:33:44.993 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2/156 2024-04-24T23:33:44.993 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 3/156 2024-04-24T23:33:44.993 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 4/156 2024-04-24T23:33:44.993 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 5/156 2024-04-24T23:33:44.993 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 6/156 2024-04-24T23:33:44.993 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 7/156 2024-04-24T23:33:44.994 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:33:44.994 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:33:44.994 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-24T23:33:44.994 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 11/156 2024-04-24T23:33:44.994 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 12/156 2024-04-24T23:33:44.994 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:33:44.994 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 14/156 2024-04-24T23:33:44.994 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 15/156 2024-04-24T23:33:44.994 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 16/156 2024-04-24T23:33:44.994 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 17/156 2024-04-24T23:33:44.994 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 18/156 2024-04-24T23:33:44.995 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 19/156 2024-04-24T23:33:44.995 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 20/156 2024-04-24T23:33:44.995 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 21/156 2024-04-24T23:33:44.995 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 22/156 2024-04-24T23:33:44.995 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 23/156 2024-04-24T23:33:44.995 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 24/156 2024-04-24T23:33:44.995 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 25/156 2024-04-24T23:33:44.995 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 26/156 2024-04-24T23:33:44.995 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 27/156 2024-04-24T23:33:44.995 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 28/156 2024-04-24T23:33:44.995 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 29/156 2024-04-24T23:33:45.008 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 30/156 2024-04-24T23:33:45.008 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 31/156 2024-04-24T23:33:45.008 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 32/156 2024-04-24T23:33:45.008 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 33/156 2024-04-24T23:33:45.008 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 34/156 2024-04-24T23:33:45.009 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 35/156 2024-04-24T23:33:45.009 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 36/156 2024-04-24T23:33:45.009 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 37/156 2024-04-24T23:33:45.009 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 38/156 2024-04-24T23:33:45.009 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 39/156 2024-04-24T23:33:45.009 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-24T23:33:45.009 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-24T23:33:45.009 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-24T23:33:45.009 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-24T23:33:45.009 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-24T23:33:45.009 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-24T23:33:45.009 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-24T23:33:45.009 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-24T23:33:45.010 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-24T23:33:45.010 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-24T23:33:45.010 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-24T23:33:45.010 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-24T23:33:45.010 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-24T23:33:45.010 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-24T23:33:45.010 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-24T23:33:45.010 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-24T23:33:45.010 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-24T23:33:45.010 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-24T23:33:45.010 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-24T23:33:45.010 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-24T23:33:45.010 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-24T23:33:45.011 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-24T23:33:45.011 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-24T23:33:45.011 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-24T23:33:45.011 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-24T23:33:45.011 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-24T23:33:45.011 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-24T23:33:45.011 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-24T23:33:45.011 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-24T23:33:45.011 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-24T23:33:45.011 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-24T23:33:45.011 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-24T23:33:45.012 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-24T23:33:45.012 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-24T23:33:45.012 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-24T23:33:45.012 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-24T23:33:45.012 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-24T23:33:45.012 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-24T23:33:45.012 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-24T23:33:45.012 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-24T23:33:45.012 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-24T23:33:45.012 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-24T23:33:45.012 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-24T23:33:45.012 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-24T23:33:45.012 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-24T23:33:45.013 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-24T23:33:45.013 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-24T23:33:45.013 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-24T23:33:45.013 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-24T23:33:45.013 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-24T23:33:45.013 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-24T23:33:45.013 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-24T23:33:45.013 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-24T23:33:45.013 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-24T23:33:45.013 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-24T23:33:45.013 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-24T23:33:45.014 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-24T23:33:45.014 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-24T23:33:45.014 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-24T23:33:45.014 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-24T23:33:45.014 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-24T23:33:45.014 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-24T23:33:45.014 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-24T23:33:45.014 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-24T23:33:45.014 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-24T23:33:45.014 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-24T23:33:45.014 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-24T23:33:45.014 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-24T23:33:45.014 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-24T23:33:45.014 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-24T23:33:45.014 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-24T23:33:45.015 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-24T23:33:45.015 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-24T23:33:45.015 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-24T23:33:45.015 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-24T23:33:45.015 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-24T23:33:45.015 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-24T23:33:45.015 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-24T23:33:45.015 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T23:33:45.015 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T23:33:45.015 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-24T23:33:45.015 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-24T23:33:45.015 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-24T23:33:45.016 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-24T23:33:45.016 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-24T23:33:45.016 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-24T23:33:45.016 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-24T23:33:45.016 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-24T23:33:45.016 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-24T23:33:45.016 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-24T23:33:45.016 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-24T23:33:45.016 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-24T23:33:45.016 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-24T23:33:45.016 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-24T23:33:45.016 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-24T23:33:45.016 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-24T23:33:45.017 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-24T23:33:45.017 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-24T23:33:45.017 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-24T23:33:45.017 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-24T23:33:45.017 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-24T23:33:45.017 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-24T23:33:45.017 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-24T23:33:45.017 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-24T23:33:45.017 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-24T23:33:45.017 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-24T23:33:45.017 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-24T23:33:45.017 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-24T23:33:45.017 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-24T23:33:45.018 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 149/156 2024-04-24T23:33:45.018 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-24T23:33:45.018 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 151/156 2024-04-24T23:33:45.018 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-24T23:33:45.018 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-24T23:33:45.018 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T23:33:45.018 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-24T23:33:46.342 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 139/156 2024-04-24T23:33:46.384 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 140/156 2024-04-24T23:33:46.425 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 141/156 2024-04-24T23:33:46.457 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-24T23:33:46.484 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-24T23:33:46.665 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 144/156 2024-04-24T23:33:46.722 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-24T23:33:46.752 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 146/156 2024-04-24T23:33:47.140 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:33:47.140 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:33:47.140 INFO:teuthology.orchestra.run.smithi157.stdout:Upgraded: 2024-04-24T23:33:47.140 INFO:teuthology.orchestra.run.smithi157.stdout: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:47.140 INFO:teuthology.orchestra.run.smithi157.stdout: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:47.140 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-24T23:33:47.141 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-24T23:33:47.141 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-04-24T23:33:47.141 INFO:teuthology.orchestra.run.smithi157.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-24T23:33:47.141 INFO:teuthology.orchestra.run.smithi157.stdout: automake-1.16.2-8.el9.noarch 2024-04-24T23:33:47.141 INFO:teuthology.orchestra.run.smithi157.stdout: bison-3.7.4-5.el9.x86_64 2024-04-24T23:33:47.141 INFO:teuthology.orchestra.run.smithi157.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-24T23:33:47.141 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:47.141 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:47.141 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:47.141 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:47.141 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:33:47.141 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:47.141 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:47.142 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:47.142 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:33:47.142 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:33:47.142 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:33:47.142 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:33:47.142 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:33:47.142 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:47.142 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:47.142 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:33:47.142 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:47.142 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:47.143 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:47.143 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:33:47.143 INFO:teuthology.orchestra.run.smithi157.stdout: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:33:47.143 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:47.143 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:47.143 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:33:47.143 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-24T23:33:47.143 INFO:teuthology.orchestra.run.smithi157.stdout: flex-2.6.4-9.el9.x86_64 2024-04-24T23:33:47.143 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-24T23:33:47.143 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-24T23:33:47.143 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-24T23:33:47.143 INFO:teuthology.orchestra.run.smithi157.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-24T23:33:47.143 INFO:teuthology.orchestra.run.smithi157.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-24T23:33:47.144 INFO:teuthology.orchestra.run.smithi157.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-24T23:33:47.144 INFO:teuthology.orchestra.run.smithi157.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-24T23:33:47.144 INFO:teuthology.orchestra.run.smithi157.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-24T23:33:47.144 INFO:teuthology.orchestra.run.smithi157.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-24T23:33:47.144 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:47.144 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:47.144 INFO:teuthology.orchestra.run.smithi157.stdout: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:47.144 INFO:teuthology.orchestra.run.smithi157.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-24T23:33:47.144 INFO:teuthology.orchestra.run.smithi157.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-24T23:33:47.144 INFO:teuthology.orchestra.run.smithi157.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-24T23:33:47.144 INFO:teuthology.orchestra.run.smithi157.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-24T23:33:47.144 INFO:teuthology.orchestra.run.smithi157.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-24T23:33:47.144 INFO:teuthology.orchestra.run.smithi157.stdout: librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:47.144 INFO:teuthology.orchestra.run.smithi157.stdout: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:47.145 INFO:teuthology.orchestra.run.smithi157.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-24T23:33:47.145 INFO:teuthology.orchestra.run.smithi157.stdout: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:47.145 INFO:teuthology.orchestra.run.smithi157.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-24T23:33:47.145 INFO:teuthology.orchestra.run.smithi157.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-24T23:33:47.145 INFO:teuthology.orchestra.run.smithi157.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-24T23:33:47.145 INFO:teuthology.orchestra.run.smithi157.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-24T23:33:47.145 INFO:teuthology.orchestra.run.smithi157.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-24T23:33:47.145 INFO:teuthology.orchestra.run.smithi157.stdout: lua-5.4.4-4.el9.x86_64 2024-04-24T23:33:47.145 INFO:teuthology.orchestra.run.smithi157.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-24T23:33:47.145 INFO:teuthology.orchestra.run.smithi157.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-24T23:33:47.145 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-24T23:33:47.146 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-24T23:33:47.146 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-24T23:33:47.146 INFO:teuthology.orchestra.run.smithi157.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-24T23:33:47.146 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-13.14-1.el9.x86_64 2024-04-24T23:33:47.146 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-24T23:33:47.146 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-24T23:33:47.146 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-24T23:33:47.146 INFO:teuthology.orchestra.run.smithi157.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-24T23:33:47.146 INFO:teuthology.orchestra.run.smithi157.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-24T23:33:47.146 INFO:teuthology.orchestra.run.smithi157.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-24T23:33:47.146 INFO:teuthology.orchestra.run.smithi157.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-24T23:33:47.146 INFO:teuthology.orchestra.run.smithi157.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-24T23:33:47.146 INFO:teuthology.orchestra.run.smithi157.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-24T23:33:47.147 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-24T23:33:47.147 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:47.147 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:47.147 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:47.147 INFO:teuthology.orchestra.run.smithi157.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-24T23:33:47.147 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-24T23:33:47.147 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-24T23:33:47.147 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-24T23:33:47.147 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-24T23:33:47.147 INFO:teuthology.orchestra.run.smithi157.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-24T23:33:47.147 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-24T23:33:47.147 INFO:teuthology.orchestra.run.smithi157.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-24T23:33:47.147 INFO:teuthology.orchestra.run.smithi157.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-24T23:33:47.148 INFO:teuthology.orchestra.run.smithi157.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-24T23:33:47.148 INFO:teuthology.orchestra.run.smithi157.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-24T23:33:47.148 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-24T23:33:47.148 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-24T23:33:47.148 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-24T23:33:47.148 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-24T23:33:47.148 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-24T23:33:47.148 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-24T23:33:47.148 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-24T23:33:47.148 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-24T23:33:47.148 INFO:teuthology.orchestra.run.smithi157.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-24T23:33:47.148 INFO:teuthology.orchestra.run.smithi157.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-24T23:33:47.148 INFO:teuthology.orchestra.run.smithi157.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-24T23:33:47.149 INFO:teuthology.orchestra.run.smithi157.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-24T23:33:47.149 INFO:teuthology.orchestra.run.smithi157.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-24T23:33:47.149 INFO:teuthology.orchestra.run.smithi157.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-24T23:33:47.149 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-24T23:33:47.149 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-24T23:33:47.149 INFO:teuthology.orchestra.run.smithi157.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-24T23:33:47.149 INFO:teuthology.orchestra.run.smithi157.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-24T23:33:47.149 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-24T23:33:47.149 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ply-3.11-14.el9.noarch 2024-04-24T23:33:47.149 INFO:teuthology.orchestra.run.smithi157.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-24T23:33:47.149 INFO:teuthology.orchestra.run.smithi157.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-24T23:33:47.149 INFO:teuthology.orchestra.run.smithi157.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-24T23:33:47.149 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-24T23:33:47.150 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-24T23:33:47.150 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-24T23:33:47.150 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-24T23:33:47.150 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-24T23:33:47.150 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-24T23:33:47.150 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:47.150 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:47.150 INFO:teuthology.orchestra.run.smithi157.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-24T23:33:47.150 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-24T23:33:47.150 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-24T23:33:47.150 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:47.151 INFO:teuthology.orchestra.run.smithi157.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-24T23:33:47.151 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-24T23:33:47.151 INFO:teuthology.orchestra.run.smithi157.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-24T23:33:47.151 INFO:teuthology.orchestra.run.smithi157.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-24T23:33:47.151 INFO:teuthology.orchestra.run.smithi157.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-24T23:33:47.151 INFO:teuthology.orchestra.run.smithi157.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-24T23:33:47.151 INFO:teuthology.orchestra.run.smithi157.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-24T23:33:47.151 INFO:teuthology.orchestra.run.smithi157.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-24T23:33:47.151 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-24T23:33:47.151 INFO:teuthology.orchestra.run.smithi157.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-24T23:33:47.151 INFO:teuthology.orchestra.run.smithi157.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-24T23:33:47.151 INFO:teuthology.orchestra.run.smithi157.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-24T23:33:47.151 INFO:teuthology.orchestra.run.smithi157.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-24T23:33:47.151 INFO:teuthology.orchestra.run.smithi157.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-24T23:33:47.152 INFO:teuthology.orchestra.run.smithi157.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-24T23:33:47.152 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:47.152 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:47.152 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:47.152 INFO:teuthology.orchestra.run.smithi157.stdout: re2-1:20211101-3.el9.x86_64 2024-04-24T23:33:47.152 INFO:teuthology.orchestra.run.smithi157.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-24T23:33:47.152 INFO:teuthology.orchestra.run.smithi157.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-24T23:33:47.152 INFO:teuthology.orchestra.run.smithi157.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-24T23:33:47.152 INFO:teuthology.orchestra.run.smithi157.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-24T23:33:47.152 INFO:teuthology.orchestra.run.smithi157.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-24T23:33:47.153 INFO:teuthology.orchestra.run.smithi157.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-24T23:33:47.153 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:33:47.153 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-04-24T23:33:47.368 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:33:47.408 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:33:47.427 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:33:47.528 DEBUG:teuthology.parallel:result is None 2024-04-24T23:33:48.272 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 148/156 2024-04-24T23:33:48.363 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-24T23:33:48.404 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-24T23:33:48.484 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-24T23:33:48.517 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-24T23:33:48.517 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T23:33:48.601 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T23:33:48.626 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T23:33:48.626 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T23:33:48.716 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T23:33:48.781 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:33:54.762 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:33:54.762 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/156 2024-04-24T23:33:54.762 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2/156 2024-04-24T23:33:54.762 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 3/156 2024-04-24T23:33:54.763 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 4/156 2024-04-24T23:33:54.763 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 5/156 2024-04-24T23:33:54.763 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 6/156 2024-04-24T23:33:54.763 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 7/156 2024-04-24T23:33:54.763 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:33:54.763 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:33:54.763 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-24T23:33:54.763 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 11/156 2024-04-24T23:33:54.763 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 12/156 2024-04-24T23:33:54.763 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:33:54.763 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 14/156 2024-04-24T23:33:54.763 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 15/156 2024-04-24T23:33:54.763 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 16/156 2024-04-24T23:33:54.764 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 17/156 2024-04-24T23:33:54.764 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 18/156 2024-04-24T23:33:54.764 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 19/156 2024-04-24T23:33:54.764 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 20/156 2024-04-24T23:33:54.764 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 21/156 2024-04-24T23:33:54.764 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 22/156 2024-04-24T23:33:54.764 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 23/156 2024-04-24T23:33:54.764 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 24/156 2024-04-24T23:33:54.764 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 25/156 2024-04-24T23:33:54.764 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 26/156 2024-04-24T23:33:54.764 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 27/156 2024-04-24T23:33:54.764 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 28/156 2024-04-24T23:33:54.764 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 29/156 2024-04-24T23:33:54.765 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 30/156 2024-04-24T23:33:54.765 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 31/156 2024-04-24T23:33:54.765 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 32/156 2024-04-24T23:33:54.765 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 33/156 2024-04-24T23:33:54.765 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 34/156 2024-04-24T23:33:54.765 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 35/156 2024-04-24T23:33:54.765 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 36/156 2024-04-24T23:33:54.765 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 37/156 2024-04-24T23:33:54.765 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 38/156 2024-04-24T23:33:54.765 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 39/156 2024-04-24T23:33:54.765 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-24T23:33:54.765 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-24T23:33:54.765 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-24T23:33:54.766 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-24T23:33:54.766 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-24T23:33:54.766 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-24T23:33:54.766 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-24T23:33:54.766 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-24T23:33:54.766 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-24T23:33:54.766 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-24T23:33:54.766 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-24T23:33:54.766 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-24T23:33:54.766 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-24T23:33:54.766 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-24T23:33:54.766 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-24T23:33:54.767 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-24T23:33:54.767 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-24T23:33:54.767 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-24T23:33:54.767 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-24T23:33:54.767 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-24T23:33:54.767 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-24T23:33:54.767 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-24T23:33:54.767 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-24T23:33:54.767 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-24T23:33:54.767 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-24T23:33:54.767 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-24T23:33:54.767 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-24T23:33:54.767 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-24T23:33:54.767 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-24T23:33:54.767 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-24T23:33:54.768 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-24T23:33:54.768 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-24T23:33:54.768 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-24T23:33:54.768 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-24T23:33:54.768 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-24T23:33:54.768 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-24T23:33:54.768 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-24T23:33:54.768 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-24T23:33:54.768 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-24T23:33:54.768 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-24T23:33:54.768 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-24T23:33:54.768 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-24T23:33:54.768 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-24T23:33:54.769 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-24T23:33:54.769 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-24T23:33:54.769 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-24T23:33:54.769 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-24T23:33:54.769 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-24T23:33:54.769 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-24T23:33:54.769 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-24T23:33:54.769 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-24T23:33:54.769 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-24T23:33:54.769 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-24T23:33:54.769 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-24T23:33:54.769 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-24T23:33:54.770 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-24T23:33:54.770 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-24T23:33:54.770 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-24T23:33:54.770 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-24T23:33:54.770 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-24T23:33:54.770 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-24T23:33:54.770 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-24T23:33:54.770 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-24T23:33:54.770 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-24T23:33:54.770 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-24T23:33:54.770 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-24T23:33:54.770 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-24T23:33:54.771 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-24T23:33:54.771 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-24T23:33:54.771 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-24T23:33:54.771 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-24T23:33:54.771 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-24T23:33:54.771 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-24T23:33:54.771 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-24T23:33:54.771 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-24T23:33:54.771 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-24T23:33:54.771 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-24T23:33:54.771 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-24T23:33:54.771 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T23:33:54.771 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T23:33:54.771 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-24T23:33:54.771 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-24T23:33:54.772 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-24T23:33:54.772 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-24T23:33:54.772 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-24T23:33:54.772 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-24T23:33:54.772 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-24T23:33:54.772 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-24T23:33:54.772 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-24T23:33:54.772 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-24T23:33:54.772 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-24T23:33:54.772 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-24T23:33:54.772 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-24T23:33:54.772 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-24T23:33:54.772 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-24T23:33:54.773 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-24T23:33:54.773 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-24T23:33:54.773 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-24T23:33:54.773 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-24T23:33:54.773 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-24T23:33:54.773 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-24T23:33:54.773 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-24T23:33:54.773 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-24T23:33:54.773 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-24T23:33:54.773 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-24T23:33:54.773 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-24T23:33:54.773 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-24T23:33:54.773 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-24T23:33:54.773 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-24T23:33:54.773 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 149/156 2024-04-24T23:33:54.774 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-24T23:33:54.774 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 151/156 2024-04-24T23:33:54.774 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-24T23:33:54.774 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-24T23:33:54.774 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T23:33:54.774 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-24T23:33:56.825 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:33:56.825 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-24T23:33:56.825 INFO:teuthology.orchestra.run.smithi169.stdout:Upgraded: 2024-04-24T23:33:56.825 INFO:teuthology.orchestra.run.smithi169.stdout: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:56.825 INFO:teuthology.orchestra.run.smithi169.stdout: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:56.825 INFO:teuthology.orchestra.run.smithi169.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-24T23:33:56.825 INFO:teuthology.orchestra.run.smithi169.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-24T23:33:56.826 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-04-24T23:33:56.826 INFO:teuthology.orchestra.run.smithi169.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-24T23:33:56.826 INFO:teuthology.orchestra.run.smithi169.stdout: automake-1.16.2-8.el9.noarch 2024-04-24T23:33:56.826 INFO:teuthology.orchestra.run.smithi169.stdout: bison-3.7.4-5.el9.x86_64 2024-04-24T23:33:56.826 INFO:teuthology.orchestra.run.smithi169.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-24T23:33:56.826 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:56.826 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:56.826 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:56.826 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:56.826 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:33:56.826 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:56.826 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:56.826 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:56.826 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:33:56.827 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:33:56.827 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:33:56.827 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:33:56.827 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:33:56.827 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:56.827 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:56.827 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:33:56.827 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:56.827 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:56.827 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:56.827 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:33:56.827 INFO:teuthology.orchestra.run.smithi169.stdout: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:33:56.827 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:56.827 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:56.828 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:33:56.828 INFO:teuthology.orchestra.run.smithi169.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-24T23:33:56.828 INFO:teuthology.orchestra.run.smithi169.stdout: flex-2.6.4-9.el9.x86_64 2024-04-24T23:33:56.828 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-24T23:33:56.828 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-24T23:33:56.828 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-24T23:33:56.828 INFO:teuthology.orchestra.run.smithi169.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-24T23:33:56.828 INFO:teuthology.orchestra.run.smithi169.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-24T23:33:56.828 INFO:teuthology.orchestra.run.smithi169.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-24T23:33:56.828 INFO:teuthology.orchestra.run.smithi169.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-24T23:33:56.828 INFO:teuthology.orchestra.run.smithi169.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-24T23:33:56.828 INFO:teuthology.orchestra.run.smithi169.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-24T23:33:56.828 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:56.828 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:56.829 INFO:teuthology.orchestra.run.smithi169.stdout: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:56.829 INFO:teuthology.orchestra.run.smithi169.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-24T23:33:56.829 INFO:teuthology.orchestra.run.smithi169.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-24T23:33:56.829 INFO:teuthology.orchestra.run.smithi169.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-24T23:33:56.829 INFO:teuthology.orchestra.run.smithi169.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-24T23:33:56.829 INFO:teuthology.orchestra.run.smithi169.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-24T23:33:56.829 INFO:teuthology.orchestra.run.smithi169.stdout: librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:56.829 INFO:teuthology.orchestra.run.smithi169.stdout: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:56.829 INFO:teuthology.orchestra.run.smithi169.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-24T23:33:56.829 INFO:teuthology.orchestra.run.smithi169.stdout: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:56.829 INFO:teuthology.orchestra.run.smithi169.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-24T23:33:56.829 INFO:teuthology.orchestra.run.smithi169.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-24T23:33:56.829 INFO:teuthology.orchestra.run.smithi169.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-24T23:33:56.829 INFO:teuthology.orchestra.run.smithi169.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-24T23:33:56.829 INFO:teuthology.orchestra.run.smithi169.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-24T23:33:56.830 INFO:teuthology.orchestra.run.smithi169.stdout: lua-5.4.4-4.el9.x86_64 2024-04-24T23:33:56.830 INFO:teuthology.orchestra.run.smithi169.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-24T23:33:56.830 INFO:teuthology.orchestra.run.smithi169.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-24T23:33:56.830 INFO:teuthology.orchestra.run.smithi169.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-24T23:33:56.830 INFO:teuthology.orchestra.run.smithi169.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-24T23:33:56.830 INFO:teuthology.orchestra.run.smithi169.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-24T23:33:56.830 INFO:teuthology.orchestra.run.smithi169.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-24T23:33:56.830 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-13.14-1.el9.x86_64 2024-04-24T23:33:56.830 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-24T23:33:56.830 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-24T23:33:56.830 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-24T23:33:56.830 INFO:teuthology.orchestra.run.smithi169.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-24T23:33:56.830 INFO:teuthology.orchestra.run.smithi169.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-24T23:33:56.830 INFO:teuthology.orchestra.run.smithi169.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-24T23:33:56.831 INFO:teuthology.orchestra.run.smithi169.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-24T23:33:56.831 INFO:teuthology.orchestra.run.smithi169.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-24T23:33:56.831 INFO:teuthology.orchestra.run.smithi169.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-24T23:33:56.831 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-24T23:33:56.831 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:56.831 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:56.831 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:56.831 INFO:teuthology.orchestra.run.smithi169.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-24T23:33:56.831 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-24T23:33:56.831 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-24T23:33:56.831 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-24T23:33:56.831 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-24T23:33:56.831 INFO:teuthology.orchestra.run.smithi169.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-24T23:33:56.831 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-24T23:33:56.831 INFO:teuthology.orchestra.run.smithi169.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-24T23:33:56.832 INFO:teuthology.orchestra.run.smithi169.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-24T23:33:56.832 INFO:teuthology.orchestra.run.smithi169.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-24T23:33:56.832 INFO:teuthology.orchestra.run.smithi169.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-24T23:33:56.832 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-24T23:33:56.832 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-24T23:33:56.832 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-24T23:33:56.832 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-24T23:33:56.832 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-24T23:33:56.832 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-24T23:33:56.832 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-24T23:33:56.832 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-24T23:33:56.832 INFO:teuthology.orchestra.run.smithi169.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-24T23:33:56.832 INFO:teuthology.orchestra.run.smithi169.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-24T23:33:56.832 INFO:teuthology.orchestra.run.smithi169.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-24T23:33:56.833 INFO:teuthology.orchestra.run.smithi169.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-24T23:33:56.833 INFO:teuthology.orchestra.run.smithi169.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-24T23:33:56.833 INFO:teuthology.orchestra.run.smithi169.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-24T23:33:56.833 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-24T23:33:56.833 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-24T23:33:56.833 INFO:teuthology.orchestra.run.smithi169.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-24T23:33:56.833 INFO:teuthology.orchestra.run.smithi169.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-24T23:33:56.833 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-24T23:33:56.833 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ply-3.11-14.el9.noarch 2024-04-24T23:33:56.833 INFO:teuthology.orchestra.run.smithi169.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-24T23:33:56.833 INFO:teuthology.orchestra.run.smithi169.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-24T23:33:56.833 INFO:teuthology.orchestra.run.smithi169.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-24T23:33:56.833 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-24T23:33:56.833 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-24T23:33:56.834 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-24T23:33:56.834 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-24T23:33:56.834 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-24T23:33:56.834 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-24T23:33:56.834 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:56.834 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:56.834 INFO:teuthology.orchestra.run.smithi169.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-24T23:33:56.834 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-24T23:33:56.834 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-24T23:33:56.834 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:56.834 INFO:teuthology.orchestra.run.smithi169.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-24T23:33:56.834 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-24T23:33:56.834 INFO:teuthology.orchestra.run.smithi169.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-24T23:33:56.834 INFO:teuthology.orchestra.run.smithi169.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-24T23:33:56.835 INFO:teuthology.orchestra.run.smithi169.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-24T23:33:56.835 INFO:teuthology.orchestra.run.smithi169.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-24T23:33:56.835 INFO:teuthology.orchestra.run.smithi169.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-24T23:33:56.835 INFO:teuthology.orchestra.run.smithi169.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-24T23:33:56.835 INFO:teuthology.orchestra.run.smithi169.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-24T23:33:56.835 INFO:teuthology.orchestra.run.smithi169.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-24T23:33:56.835 INFO:teuthology.orchestra.run.smithi169.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-24T23:33:56.835 INFO:teuthology.orchestra.run.smithi169.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-24T23:33:56.835 INFO:teuthology.orchestra.run.smithi169.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-24T23:33:56.835 INFO:teuthology.orchestra.run.smithi169.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-24T23:33:56.835 INFO:teuthology.orchestra.run.smithi169.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-24T23:33:56.835 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:56.835 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:56.835 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:33:56.836 INFO:teuthology.orchestra.run.smithi169.stdout: re2-1:20211101-3.el9.x86_64 2024-04-24T23:33:56.836 INFO:teuthology.orchestra.run.smithi169.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-24T23:33:56.836 INFO:teuthology.orchestra.run.smithi169.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-24T23:33:56.836 INFO:teuthology.orchestra.run.smithi169.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-24T23:33:56.836 INFO:teuthology.orchestra.run.smithi169.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-24T23:33:56.836 INFO:teuthology.orchestra.run.smithi169.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-24T23:33:56.836 INFO:teuthology.orchestra.run.smithi169.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-24T23:33:56.836 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-24T23:33:56.836 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-04-24T23:33:57.253 DEBUG:teuthology.parallel:result is None 2024-04-24T23:33:57.253 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:33:57.477 DEBUG:teuthology.orchestra.run.smithi157:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-24T23:33:57.503 INFO:teuthology.orchestra.run.smithi157.stdout:19.0.0-2445.gd2c45d97.el9 2024-04-24T23:33:57.504 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445.gd2c45d97.el9 2024-04-24T23:33:57.504 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445.gd2c45d97 is installed. 2024-04-24T23:33:57.506 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:33:57.649 DEBUG:teuthology.orchestra.run.smithi169:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-24T23:33:57.671 INFO:teuthology.orchestra.run.smithi169.stdout:19.0.0-2445.gd2c45d97.el9 2024-04-24T23:33:57.671 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445.gd2c45d97.el9 2024-04-24T23:33:57.671 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445.gd2c45d97 is installed. 2024-04-24T23:33:57.673 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-24T23:33:57.673 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-24T23:33:57.673 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-24T23:33:57.705 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-24T23:33:57.705 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-24T23:33:57.736 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-24T23:33:57.736 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-24T23:33:57.737 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/daemon-helper 2024-04-24T23:33:57.773 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-24T23:33:57.840 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-24T23:33:57.840 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/usr/bin/daemon-helper 2024-04-24T23:33:57.864 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-24T23:33:57.928 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-24T23:33:57.928 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-24T23:33:57.928 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-24T23:33:57.956 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-24T23:33:58.024 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-24T23:33:58.024 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-24T23:33:58.048 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-24T23:33:58.111 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-24T23:33:58.111 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-24T23:33:58.111 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/stdin-killer 2024-04-24T23:33:58.138 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-24T23:33:58.204 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-24T23:33:58.204 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/usr/bin/stdin-killer 2024-04-24T23:33:58.228 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-24T23:33:58.291 INFO:teuthology.run_tasks:Running task ceph... 2024-04-24T23:33:58.374 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-24T23:33:58.374 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 777 /var/log/ceph 2024-04-24T23:33:58.377 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 777 /var/log/ceph 2024-04-24T23:33:58.401 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-24T23:33:58.401 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-24T23:33:58.444 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-24T23:33:58.467 INFO:tasks.ceph:Creating extra log directories... 2024-04-24T23:33:58.467 DEBUG:teuthology.orchestra.run.smithi157:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-24T23:33:58.508 DEBUG:teuthology.orchestra.run.smithi169:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-24T23:33:58.539 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-24T23:33:58.539 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-24T23:33:58.539 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-24T23:33:58.539 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-24T23:33:58.539 DEBUG:teuthology.orchestra.run.smithi157:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-24T23:33:58.588 DEBUG:teuthology.orchestra.run.smithi157:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-24T23:33:58.698 DEBUG:teuthology.orchestra.run.smithi157:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-24T23:33:58.722 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-24T23:33:58.722 DEBUG:teuthology.orchestra.run.smithi169:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-24T23:33:58.737 DEBUG:teuthology.orchestra.run.smithi169:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-24T23:33:58.848 DEBUG:teuthology.orchestra.run.smithi169:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-24T23:33:58.876 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-24T23:33:58.877 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-24T23:33:58.877 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672205', 'branch': 'squid', 'description': 'fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu/{overrides ubuntu_latest}} kclient-overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health} tasks/kernel_cfuse_workunits_dbench_iozone}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7672205', 'kclient-workload': {'sequential': [{'kclient': ['client.1']}, {'workunit': {'clients': {'client.1': ['suites/dbench.sh']}}}]}, 'kernel': {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'valgrind': {'exit_on_first_error': False}}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1']], 'seed': 8528, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'sleep_before_teardown': 0, 'subset': '122/512', 'suite': 'fs', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi157.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHAfRS1+kKdGxkV0ecDPkMU4fbI8Zk7hbDuWL3iDXy2xV3OMdaboLf/Y1BX9iPjRBJLx5eWgCssb0hZqOoSmMlI=', 'smithi169.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGUNF8X44qTR0U8iy8BEnPwE+8Z0hrRptOpkJu67oUJaC9XZZroz6yT9WgSl99I8qyu5Rov21j+zdmWFZ0gcnvI='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'valgrind': {'exit_on_first_error': False}, 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'parallel': ['user-workload', 'kclient-workload']}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-24_21:24:02', 'tube': 'smithi', 'user': 'teuthology', 'user-workload': {'sequential': [{'ceph-fuse': ['client.0']}, {'workunit': {'clients': {'client.0': ['suites/iozone.sh']}}}]}, 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-24T23:33:58.877 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-24T23:33:58.880 DEBUG:teuthology.orchestra.run.smithi169:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-24T23:33:58.927 DEBUG:teuthology.orchestra.run.smithi157:> sudo install -d -m0777 -- /var/run/ceph 2024-04-24T23:33:58.935 DEBUG:teuthology.orchestra.run.smithi169:> sudo install -d -m0777 -- /var/run/ceph 2024-04-24T23:33:58.993 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-24T23:33:58.993 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/scratch_devs of=/dev/stdout 2024-04-24T23:33:59.013 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-24T23:33:59.013 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_1 2024-04-24T23:33:59.067 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-24T23:33:59.067 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:33:59.067 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-24T23:33:59.067 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:33:59.067 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:33:59.067 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-04-24 23:33:39.640411034 +0000 2024-04-24T23:33:59.067 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-04-24 23:31:53.794699501 +0000 2024-04-24T23:33:59.067 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-04-24 23:31:53.794699501 +0000 2024-04-24T23:33:59.067 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-04-24 23:31:53.794699501 +0000 2024-04-24T23:33:59.068 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-24T23:33:59.129 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-04-24T23:33:59.129 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-04-24T23:33:59.129 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 8.3891e-05 s, 6.1 MB/s 2024-04-24T23:33:59.130 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-24T23:33:59.186 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_2 2024-04-24T23:33:59.241 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-24T23:33:59.241 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:33:59.241 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-24T23:33:59.241 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:33:59.241 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:33:59.241 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-04-24 23:33:39.641411016 +0000 2024-04-24T23:33:59.241 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-04-24 23:31:54.125693241 +0000 2024-04-24T23:33:59.241 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-04-24 23:31:54.125693241 +0000 2024-04-24T23:33:59.241 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-04-24 23:31:54.125693241 +0000 2024-04-24T23:33:59.242 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-24T23:33:59.303 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-04-24T23:33:59.303 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-04-24T23:33:59.303 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 8.0749e-05 s, 6.3 MB/s 2024-04-24T23:33:59.304 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-24T23:33:59.360 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_3 2024-04-24T23:33:59.415 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-24T23:33:59.415 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:33:59.415 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-24T23:33:59.415 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:33:59.415 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:33:59.415 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-04-24 23:33:39.641411016 +0000 2024-04-24T23:33:59.415 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-04-24 23:31:54.468686756 +0000 2024-04-24T23:33:59.415 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-04-24 23:31:54.468686756 +0000 2024-04-24T23:33:59.415 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-04-24 23:31:54.468686756 +0000 2024-04-24T23:33:59.415 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-24T23:33:59.477 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-04-24T23:33:59.477 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-04-24T23:33:59.477 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 8.0927e-05 s, 6.3 MB/s 2024-04-24T23:33:59.478 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-24T23:33:59.533 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_4 2024-04-24T23:33:59.587 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-24T23:33:59.588 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:33:59.588 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-24T23:33:59.588 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:33:59.588 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:33:59.588 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-04-24 23:33:39.641411016 +0000 2024-04-24T23:33:59.588 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-04-24 23:31:54.798680515 +0000 2024-04-24T23:33:59.588 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-04-24 23:31:54.798680515 +0000 2024-04-24T23:33:59.588 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-04-24 23:31:54.798680515 +0000 2024-04-24T23:33:59.588 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-24T23:33:59.649 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-04-24T23:33:59.649 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-04-24T23:33:59.649 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000101787 s, 5.0 MB/s 2024-04-24T23:33:59.650 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-24T23:33:59.706 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-24T23:33:59.706 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-24T23:33:59.706 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/scratch_devs of=/dev/stdout 2024-04-24T23:33:59.722 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-24T23:33:59.722 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_1 2024-04-24T23:33:59.778 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-24T23:33:59.778 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:33:59.778 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-24T23:33:59.778 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:33:59.778 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:33:59.778 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-04-24 23:33:49.138698404 +0000 2024-04-24T23:33:59.778 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-04-24 23:31:53.815847565 +0000 2024-04-24T23:33:59.778 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-04-24 23:31:53.815847565 +0000 2024-04-24T23:33:59.778 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-04-24 23:31:53.815847565 +0000 2024-04-24T23:33:59.779 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-24T23:33:59.840 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-04-24T23:33:59.840 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-04-24T23:33:59.840 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000248943 s, 2.1 MB/s 2024-04-24T23:33:59.841 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-24T23:33:59.898 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_2 2024-04-24T23:33:59.953 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-24T23:33:59.953 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:33:59.953 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-24T23:33:59.953 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:33:59.953 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:33:59.953 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-04-24 23:33:49.139698384 +0000 2024-04-24T23:33:59.953 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-04-24 23:31:54.153840714 +0000 2024-04-24T23:33:59.953 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-04-24 23:31:54.153840714 +0000 2024-04-24T23:33:59.953 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-04-24 23:31:54.153840714 +0000 2024-04-24T23:33:59.953 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-24T23:34:00.015 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-04-24T23:34:00.015 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-04-24T23:34:00.015 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.00029418 s, 1.7 MB/s 2024-04-24T23:34:00.017 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-24T23:34:00.072 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_3 2024-04-24T23:34:00.127 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-24T23:34:00.127 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:34:00.127 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-24T23:34:00.128 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:34:00.128 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:34:00.128 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-04-24 23:33:49.139698384 +0000 2024-04-24T23:34:00.128 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-04-24 23:31:54.491833864 +0000 2024-04-24T23:34:00.128 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-04-24 23:31:54.491833864 +0000 2024-04-24T23:34:00.128 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-04-24 23:31:54.491833864 +0000 2024-04-24T23:34:00.128 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-24T23:34:00.190 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-04-24T23:34:00.190 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-04-24T23:34:00.190 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.0002372 s, 2.2 MB/s 2024-04-24T23:34:00.192 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-24T23:34:00.247 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_4 2024-04-24T23:34:00.303 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-24T23:34:00.303 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:34:00.303 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-24T23:34:00.303 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:34:00.303 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:34:00.303 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-04-24 23:33:49.139698384 +0000 2024-04-24T23:34:00.303 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-04-24 23:31:54.836826872 +0000 2024-04-24T23:34:00.303 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-04-24 23:31:54.836826872 +0000 2024-04-24T23:34:00.303 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-04-24 23:31:54.836826872 +0000 2024-04-24T23:34:00.304 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-24T23:34:00.364 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-04-24T23:34:00.364 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-04-24T23:34:00.365 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000216856 s, 2.4 MB/s 2024-04-24T23:34:00.366 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-24T23:34:00.421 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-24T23:34:00.421 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi157.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi169.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-24T23:34:00.422 INFO:tasks.ceph:Generating config... 2024-04-24T23:34:00.423 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-24T23:34:00.423 INFO:tasks.ceph:[client] debug client = 20 2024-04-24T23:34:00.423 INFO:tasks.ceph:[client] debug ms = 1 2024-04-24T23:34:00.423 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-24T23:34:00.423 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-24T23:34:00.423 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-04-24T23:34:00.423 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-24T23:34:00.423 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-24T23:34:00.423 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-24T23:34:00.423 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-24T23:34:00.423 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-24T23:34:00.424 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-24T23:34:00.424 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-24T23:34:00.424 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-24T23:34:00.424 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-24T23:34:00.424 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-24T23:34:00.424 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-24T23:34:00.424 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-24T23:34:00.424 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-24T23:34:00.424 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-24T23:34:00.424 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-24T23:34:00.424 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-24T23:34:00.424 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-24T23:34:00.424 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-24T23:34:00.424 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-24T23:34:00.424 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-04-24T23:34:00.425 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-24T23:34:00.425 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-04-24T23:34:00.425 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-04-24T23:34:00.425 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-24T23:34:00.425 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-24T23:34:00.425 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-04-24T23:34:00.425 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-24T23:34:00.425 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-24T23:34:00.425 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-24T23:34:00.425 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-24T23:34:00.425 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-24T23:34:00.425 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-24T23:34:00.425 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-04-24T23:34:00.426 INFO:tasks.ceph:Setting up mon.a... 2024-04-24T23:34:00.426 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-24T23:34:00.481 INFO:teuthology.orchestra.run.smithi157.stdout:creating /etc/ceph/ceph.keyring 2024-04-24T23:34:00.484 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-24T23:34:00.525 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-24T23:34:00.589 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.157'), ('mon.b', '172.21.15.169'), ('mon.c', '[v2:172.21.15.169:3301,v1:172.21.15.169:6790]')] 2024-04-24T23:34:00.590 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.157,172.21.15.169,[v2:172.21.15.169:3301,v1:172.21.15.169:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-24T23:34:00.591 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-24T23:34:00.591 DEBUG:teuthology.orchestra.run.smithi157:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-24T23:34:00.644 DEBUG:teuthology.orchestra.run.smithi157:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.157 --add b 172.21.15.169 --addv c '[v2:172.21.15.169:3301,v1:172.21.15.169:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-24T23:34:00.730 INFO:teuthology.orchestra.run.smithi157.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-24T23:34:00.731 INFO:teuthology.orchestra.run.smithi157.stdout:monmaptool: generated fsid 223685fb-5b64-46a5-9857-3791c9c07df2 2024-04-24T23:34:00.731 INFO:teuthology.orchestra.run.smithi157.stdout:setting min_mon_release = quincy 2024-04-24T23:34:00.731 INFO:teuthology.orchestra.run.smithi157.stdout:epoch 0 2024-04-24T23:34:00.731 INFO:teuthology.orchestra.run.smithi157.stdout:fsid 223685fb-5b64-46a5-9857-3791c9c07df2 2024-04-24T23:34:00.731 INFO:teuthology.orchestra.run.smithi157.stdout:last_changed 2024-04-24T23:34:00.731188+0000 2024-04-24T23:34:00.731 INFO:teuthology.orchestra.run.smithi157.stdout:created 2024-04-24T23:34:00.731188+0000 2024-04-24T23:34:00.732 INFO:teuthology.orchestra.run.smithi157.stdout:min_mon_release 17 (quincy) 2024-04-24T23:34:00.732 INFO:teuthology.orchestra.run.smithi157.stdout:election_strategy: 1 2024-04-24T23:34:00.732 INFO:teuthology.orchestra.run.smithi157.stdout:0: [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon.a 2024-04-24T23:34:00.732 INFO:teuthology.orchestra.run.smithi157.stdout:1: [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon.b 2024-04-24T23:34:00.732 INFO:teuthology.orchestra.run.smithi157.stdout:2: [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] mon.c 2024-04-24T23:34:00.732 INFO:teuthology.orchestra.run.smithi157.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-24T23:34:00.734 DEBUG:teuthology.orchestra.run.smithi157:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-24T23:34:00.750 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 223685fb-5b64-46a5-9857-3791c9c07df2... 2024-04-24T23:34:00.751 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-24T23:34:00.792 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-24T23:34:00.829 INFO:teuthology.orchestra.run.smithi157.stdout:[global] 2024-04-24T23:34:00.829 INFO:teuthology.orchestra.run.smithi157.stdout: chdir = "" 2024-04-24T23:34:00.829 INFO:teuthology.orchestra.run.smithi157.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-24T23:34:00.830 INFO:teuthology.orchestra.run.smithi157.stdout: auth supported = cephx 2024-04-24T23:34:00.830 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:34:00.830 INFO:teuthology.orchestra.run.smithi157.stdout: filestore xattr use omap = true 2024-04-24T23:34:00.830 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:34:00.830 INFO:teuthology.orchestra.run.smithi157.stdout: mon clock drift allowed = 1.000 2024-04-24T23:34:00.830 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:34:00.830 INFO:teuthology.orchestra.run.smithi157.stdout: osd crush chooseleaf type = 0 2024-04-24T23:34:00.830 INFO:teuthology.orchestra.run.smithi157.stdout: auth debug = true 2024-04-24T23:34:00.830 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:34:00.830 INFO:teuthology.orchestra.run.smithi157.stdout: ms die on old message = true 2024-04-24T23:34:00.830 INFO:teuthology.orchestra.run.smithi157.stdout: ms die on bug = true 2024-04-24T23:34:00.830 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:34:00.831 INFO:teuthology.orchestra.run.smithi157.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-24T23:34:00.831 INFO:teuthology.orchestra.run.smithi157.stdout: mon pg warn max object skew = 0 2024-04-24T23:34:00.831 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:34:00.831 INFO:teuthology.orchestra.run.smithi157.stdout: # disable pg_autoscaler by default for new pools 2024-04-24T23:34:00.831 INFO:teuthology.orchestra.run.smithi157.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-24T23:34:00.831 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:34:00.831 INFO:teuthology.orchestra.run.smithi157.stdout: osd pool default size = 2 2024-04-24T23:34:00.831 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:34:00.831 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd allow primary affinity = true 2024-04-24T23:34:00.831 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd allow pg remap = true 2024-04-24T23:34:00.831 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on legacy crush tunables = false 2024-04-24T23:34:00.831 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on crush straw calc version zero = false 2024-04-24T23:34:00.831 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on no sortbitwise = false 2024-04-24T23:34:00.832 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on osd down out interval zero = false 2024-04-24T23:34:00.832 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on too few osds = false 2024-04-24T23:34:00.832 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-24T23:34:00.832 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-24T23:34:00.832 INFO:teuthology.orchestra.run.smithi157.stdout: mon_allow_pool_size_one = true 2024-04-24T23:34:00.832 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:34:00.832 INFO:teuthology.orchestra.run.smithi157.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-24T23:34:00.832 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:34:00.832 INFO:teuthology.orchestra.run.smithi157.stdout: osd default data pool replay window = 5 2024-04-24T23:34:00.832 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:34:00.832 INFO:teuthology.orchestra.run.smithi157.stdout: mon allow pool delete = true 2024-04-24T23:34:00.832 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:34:00.832 INFO:teuthology.orchestra.run.smithi157.stdout: mon cluster log file level = debug 2024-04-24T23:34:00.833 INFO:teuthology.orchestra.run.smithi157.stdout: debug asserts on shutdown = true 2024-04-24T23:34:00.833 INFO:teuthology.orchestra.run.smithi157.stdout: mon health detail to clog = false 2024-04-24T23:34:00.833 INFO:teuthology.orchestra.run.smithi157.stdout: mon host = "172.21.15.157,172.21.15.169,[v2:172.21.15.169:3301,v1:172.21.15.169:6790]" 2024-04-24T23:34:00.833 INFO:teuthology.orchestra.run.smithi157.stdout: ms die on skipped message = False 2024-04-24T23:34:00.833 INFO:teuthology.orchestra.run.smithi157.stdout: fsid = 223685fb-5b64-46a5-9857-3791c9c07df2 2024-04-24T23:34:00.833 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:34:00.833 INFO:teuthology.orchestra.run.smithi157.stdout:[osd] 2024-04-24T23:34:00.833 INFO:teuthology.orchestra.run.smithi157.stdout: osd journal size = 100 2024-04-24T23:34:00.833 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:34:00.833 INFO:teuthology.orchestra.run.smithi157.stdout: osd scrub load threshold = 5.0 2024-04-24T23:34:00.833 INFO:teuthology.orchestra.run.smithi157.stdout: osd scrub max interval = 600 2024-04-24T23:34:00.833 INFO:teuthology.orchestra.run.smithi157.stdout: osd mclock profile = high_recovery_ops 2024-04-24T23:34:00.834 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:34:00.834 INFO:teuthology.orchestra.run.smithi157.stdout: osd recover clone overlap = true 2024-04-24T23:34:00.834 INFO:teuthology.orchestra.run.smithi157.stdout: osd recovery max chunk = 1048576 2024-04-24T23:34:00.834 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:34:00.834 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug shutdown = true 2024-04-24T23:34:00.834 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug op order = true 2024-04-24T23:34:00.834 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug verify stray on activate = true 2024-04-24T23:34:00.834 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug trim objects = true 2024-04-24T23:34:00.834 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:34:00.834 INFO:teuthology.orchestra.run.smithi157.stdout: osd open classes on start = true 2024-04-24T23:34:00.834 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug pg log writeout = true 2024-04-24T23:34:00.834 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:34:00.834 INFO:teuthology.orchestra.run.smithi157.stdout: osd deep scrub update digest min age = 30 2024-04-24T23:34:00.835 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:34:00.835 INFO:teuthology.orchestra.run.smithi157.stdout: osd map max advance = 10 2024-04-24T23:34:00.835 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:34:00.835 INFO:teuthology.orchestra.run.smithi157.stdout: journal zero on create = true 2024-04-24T23:34:00.835 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:34:00.835 INFO:teuthology.orchestra.run.smithi157.stdout: filestore ondisk finisher threads = 3 2024-04-24T23:34:00.835 INFO:teuthology.orchestra.run.smithi157.stdout: filestore apply finisher threads = 3 2024-04-24T23:34:00.835 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:34:00.835 INFO:teuthology.orchestra.run.smithi157.stdout: bdev debug aio = true 2024-04-24T23:34:00.835 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug misdirected ops = true 2024-04-24T23:34:00.835 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore block size = 96636764160 2024-04-24T23:34:00.835 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore compression mode = aggressive 2024-04-24T23:34:00.836 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore fsck on mount = True 2024-04-24T23:34:00.836 INFO:teuthology.orchestra.run.smithi157.stdout: debug bluefs = 20 2024-04-24T23:34:00.836 INFO:teuthology.orchestra.run.smithi157.stdout: debug bluestore = 20 2024-04-24T23:34:00.836 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2024-04-24T23:34:00.836 INFO:teuthology.orchestra.run.smithi157.stdout: debug osd = 20 2024-04-24T23:34:00.836 INFO:teuthology.orchestra.run.smithi157.stdout: debug rocksdb = 10 2024-04-24T23:34:00.836 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd backfillfull_ratio = 0.85 2024-04-24T23:34:00.836 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd full ratio = 0.9 2024-04-24T23:34:00.836 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd nearfull ratio = 0.8 2024-04-24T23:34:00.836 INFO:teuthology.orchestra.run.smithi157.stdout: osd failsafe full ratio = 0.95 2024-04-24T23:34:00.836 INFO:teuthology.orchestra.run.smithi157.stdout: osd objectstore = bluestore 2024-04-24T23:34:00.836 INFO:teuthology.orchestra.run.smithi157.stdout: osd op complaint time = 180 2024-04-24T23:34:00.836 INFO:teuthology.orchestra.run.smithi157.stdout: osd shutdown pgref assert = True 2024-04-24T23:34:00.837 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:34:00.837 INFO:teuthology.orchestra.run.smithi157.stdout:[mgr] 2024-04-24T23:34:00.837 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2024-04-24T23:34:00.837 INFO:teuthology.orchestra.run.smithi157.stdout: debug mgr = 20 2024-04-24T23:34:00.837 INFO:teuthology.orchestra.run.smithi157.stdout: debug mon = 20 2024-04-24T23:34:00.837 INFO:teuthology.orchestra.run.smithi157.stdout: debug auth = 20 2024-04-24T23:34:00.837 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min pgs per osd = 4 2024-04-24T23:34:00.837 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min bytes per osd = 10 2024-04-24T23:34:00.837 INFO:teuthology.orchestra.run.smithi157.stdout: mgr/telemetry/nag = false 2024-04-24T23:34:00.837 INFO:teuthology.orchestra.run.smithi157.stdout: debug client = 20 2024-04-24T23:34:00.837 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:34:00.837 INFO:teuthology.orchestra.run.smithi157.stdout:[mon] 2024-04-24T23:34:00.837 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2024-04-24T23:34:00.838 INFO:teuthology.orchestra.run.smithi157.stdout: debug mon = 20 2024-04-24T23:34:00.838 INFO:teuthology.orchestra.run.smithi157.stdout: debug paxos = 20 2024-04-24T23:34:00.838 INFO:teuthology.orchestra.run.smithi157.stdout: debug auth = 20 2024-04-24T23:34:00.838 INFO:teuthology.orchestra.run.smithi157.stdout: mon data avail warn = 5 2024-04-24T23:34:00.838 INFO:teuthology.orchestra.run.smithi157.stdout: mon mgr mkfs grace = 240 2024-04-24T23:34:00.838 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min pgs per osd = 4 2024-04-24T23:34:00.838 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd reporter subtree level = osd 2024-04-24T23:34:00.838 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd prime pg temp = true 2024-04-24T23:34:00.838 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min bytes per osd = 10 2024-04-24T23:34:00.838 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:34:00.838 INFO:teuthology.orchestra.run.smithi157.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-24T23:34:00.838 INFO:teuthology.orchestra.run.smithi157.stdout: auth mon ticket ttl = 660 # 11m 2024-04-24T23:34:00.839 INFO:teuthology.orchestra.run.smithi157.stdout: auth service ticket ttl = 240 # 4m 2024-04-24T23:34:00.839 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:34:00.839 INFO:teuthology.orchestra.run.smithi157.stdout: # don't complain about insecure global_id in the test suite 2024-04-24T23:34:00.839 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-24T23:34:00.839 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-24T23:34:00.839 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:34:00.839 INFO:teuthology.orchestra.run.smithi157.stdout: # 1m isn't quite enough 2024-04-24T23:34:00.839 INFO:teuthology.orchestra.run.smithi157.stdout: mon_down_mkfs_grace = 2m 2024-04-24T23:34:00.839 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:34:00.839 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_filestore_osds = false 2024-04-24T23:34:00.839 INFO:teuthology.orchestra.run.smithi157.stdout: mon down mkfs grace = 300 2024-04-24T23:34:00.839 INFO:teuthology.orchestra.run.smithi157.stdout: mon op complaint time = 120 2024-04-24T23:34:00.839 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:34:00.840 INFO:teuthology.orchestra.run.smithi157.stdout:[client] 2024-04-24T23:34:00.840 INFO:teuthology.orchestra.run.smithi157.stdout: rgw cache enabled = true 2024-04-24T23:34:00.840 INFO:teuthology.orchestra.run.smithi157.stdout: rgw enable ops log = true 2024-04-24T23:34:00.840 INFO:teuthology.orchestra.run.smithi157.stdout: rgw enable usage log = true 2024-04-24T23:34:00.840 INFO:teuthology.orchestra.run.smithi157.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-24T23:34:00.840 INFO:teuthology.orchestra.run.smithi157.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-24T23:34:00.840 INFO:teuthology.orchestra.run.smithi157.stdout: client mount timeout = 600 2024-04-24T23:34:00.840 INFO:teuthology.orchestra.run.smithi157.stdout: debug client = 20 2024-04-24T23:34:00.840 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2024-04-24T23:34:00.840 INFO:teuthology.orchestra.run.smithi157.stdout: rados mon op timeout = 900 2024-04-24T23:34:00.840 INFO:teuthology.orchestra.run.smithi157.stdout: rados osd op timeout = 900 2024-04-24T23:34:00.840 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.a] 2024-04-24T23:34:00.841 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.b] 2024-04-24T23:34:00.841 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.c] 2024-04-24T23:34:00.841 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.a] 2024-04-24T23:34:00.841 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.c] 2024-04-24T23:34:00.841 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.b] 2024-04-24T23:34:00.841 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.d] 2024-04-24T23:34:00.841 INFO:teuthology.orchestra.run.smithi157.stdout:[mds] 2024-04-24T23:34:00.841 INFO:teuthology.orchestra.run.smithi157.stdout: debug mds = 20 2024-04-24T23:34:00.841 INFO:teuthology.orchestra.run.smithi157.stdout: debug mds balancer = 20 2024-04-24T23:34:00.841 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2024-04-24T23:34:00.841 INFO:teuthology.orchestra.run.smithi157.stdout: mds debug frag = True 2024-04-24T23:34:00.841 INFO:teuthology.orchestra.run.smithi157.stdout: mds debug scatterstat = True 2024-04-24T23:34:00.842 INFO:teuthology.orchestra.run.smithi157.stdout: mds op complaint time = 180 2024-04-24T23:34:00.842 INFO:teuthology.orchestra.run.smithi157.stdout: mds verify scatter = True 2024-04-24T23:34:00.842 INFO:teuthology.orchestra.run.smithi157.stdout: osd op complaint time = 180 2024-04-24T23:34:00.842 INFO:teuthology.orchestra.run.smithi157.stdout: rados mon op timeout = 900 2024-04-24T23:34:00.842 INFO:teuthology.orchestra.run.smithi157.stdout: rados osd op timeout = 900 2024-04-24T23:34:00.842 INFO:teuthology.orchestra.run.smithi169.stdout:[global] 2024-04-24T23:34:00.843 INFO:teuthology.orchestra.run.smithi169.stdout: chdir = "" 2024-04-24T23:34:00.843 INFO:teuthology.orchestra.run.smithi169.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-24T23:34:00.843 INFO:teuthology.orchestra.run.smithi169.stdout: auth supported = cephx 2024-04-24T23:34:00.843 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-24T23:34:00.843 INFO:teuthology.orchestra.run.smithi169.stdout: filestore xattr use omap = true 2024-04-24T23:34:00.843 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-24T23:34:00.843 INFO:teuthology.orchestra.run.smithi169.stdout: mon clock drift allowed = 1.000 2024-04-24T23:34:00.843 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-24T23:34:00.843 INFO:teuthology.orchestra.run.smithi169.stdout: osd crush chooseleaf type = 0 2024-04-24T23:34:00.843 INFO:teuthology.orchestra.run.smithi169.stdout: auth debug = true 2024-04-24T23:34:00.843 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-24T23:34:00.843 INFO:teuthology.orchestra.run.smithi169.stdout: ms die on old message = true 2024-04-24T23:34:00.844 INFO:teuthology.orchestra.run.smithi169.stdout: ms die on bug = true 2024-04-24T23:34:00.844 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-24T23:34:00.844 INFO:teuthology.orchestra.run.smithi169.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-24T23:34:00.844 INFO:teuthology.orchestra.run.smithi169.stdout: mon pg warn max object skew = 0 2024-04-24T23:34:00.844 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-24T23:34:00.844 INFO:teuthology.orchestra.run.smithi169.stdout: # disable pg_autoscaler by default for new pools 2024-04-24T23:34:00.844 INFO:teuthology.orchestra.run.smithi169.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-24T23:34:00.844 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-24T23:34:00.844 INFO:teuthology.orchestra.run.smithi169.stdout: osd pool default size = 2 2024-04-24T23:34:00.844 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-24T23:34:00.844 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd allow primary affinity = true 2024-04-24T23:34:00.844 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd allow pg remap = true 2024-04-24T23:34:00.844 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on legacy crush tunables = false 2024-04-24T23:34:00.845 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on crush straw calc version zero = false 2024-04-24T23:34:00.845 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on no sortbitwise = false 2024-04-24T23:34:00.845 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on osd down out interval zero = false 2024-04-24T23:34:00.845 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on too few osds = false 2024-04-24T23:34:00.845 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-24T23:34:00.845 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-24T23:34:00.845 INFO:teuthology.orchestra.run.smithi169.stdout: mon_allow_pool_size_one = true 2024-04-24T23:34:00.845 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-24T23:34:00.845 INFO:teuthology.orchestra.run.smithi169.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-24T23:34:00.845 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-24T23:34:00.845 INFO:teuthology.orchestra.run.smithi169.stdout: osd default data pool replay window = 5 2024-04-24T23:34:00.845 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-24T23:34:00.846 INFO:teuthology.orchestra.run.smithi169.stdout: mon allow pool delete = true 2024-04-24T23:34:00.846 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-24T23:34:00.846 INFO:teuthology.orchestra.run.smithi169.stdout: mon cluster log file level = debug 2024-04-24T23:34:00.846 INFO:teuthology.orchestra.run.smithi169.stdout: debug asserts on shutdown = true 2024-04-24T23:34:00.846 INFO:teuthology.orchestra.run.smithi169.stdout: mon health detail to clog = false 2024-04-24T23:34:00.846 INFO:teuthology.orchestra.run.smithi169.stdout: mon host = "172.21.15.157,172.21.15.169,[v2:172.21.15.169:3301,v1:172.21.15.169:6790]" 2024-04-24T23:34:00.846 INFO:teuthology.orchestra.run.smithi169.stdout: ms die on skipped message = False 2024-04-24T23:34:00.846 INFO:teuthology.orchestra.run.smithi169.stdout: fsid = 223685fb-5b64-46a5-9857-3791c9c07df2 2024-04-24T23:34:00.846 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-24T23:34:00.846 INFO:teuthology.orchestra.run.smithi169.stdout:[osd] 2024-04-24T23:34:00.846 INFO:teuthology.orchestra.run.smithi169.stdout: osd journal size = 100 2024-04-24T23:34:00.846 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-24T23:34:00.847 INFO:teuthology.orchestra.run.smithi169.stdout: osd scrub load threshold = 5.0 2024-04-24T23:34:00.847 INFO:teuthology.orchestra.run.smithi169.stdout: osd scrub max interval = 600 2024-04-24T23:34:00.847 INFO:teuthology.orchestra.run.smithi169.stdout: osd mclock profile = high_recovery_ops 2024-04-24T23:34:00.847 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-24T23:34:00.847 INFO:teuthology.orchestra.run.smithi169.stdout: osd recover clone overlap = true 2024-04-24T23:34:00.847 INFO:teuthology.orchestra.run.smithi169.stdout: osd recovery max chunk = 1048576 2024-04-24T23:34:00.847 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-24T23:34:00.847 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug shutdown = true 2024-04-24T23:34:00.847 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug op order = true 2024-04-24T23:34:00.847 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug verify stray on activate = true 2024-04-24T23:34:00.847 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug trim objects = true 2024-04-24T23:34:00.847 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-24T23:34:00.848 INFO:teuthology.orchestra.run.smithi169.stdout: osd open classes on start = true 2024-04-24T23:34:00.848 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug pg log writeout = true 2024-04-24T23:34:00.848 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-24T23:34:00.848 INFO:teuthology.orchestra.run.smithi169.stdout: osd deep scrub update digest min age = 30 2024-04-24T23:34:00.848 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-24T23:34:00.848 INFO:teuthology.orchestra.run.smithi169.stdout: osd map max advance = 10 2024-04-24T23:34:00.848 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-24T23:34:00.848 INFO:teuthology.orchestra.run.smithi169.stdout: journal zero on create = true 2024-04-24T23:34:00.848 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-24T23:34:00.848 INFO:teuthology.orchestra.run.smithi169.stdout: filestore ondisk finisher threads = 3 2024-04-24T23:34:00.848 INFO:teuthology.orchestra.run.smithi169.stdout: filestore apply finisher threads = 3 2024-04-24T23:34:00.848 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-24T23:34:00.848 INFO:teuthology.orchestra.run.smithi169.stdout: bdev debug aio = true 2024-04-24T23:34:00.848 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug misdirected ops = true 2024-04-24T23:34:00.849 INFO:teuthology.orchestra.run.smithi169.stdout: bluestore block size = 96636764160 2024-04-24T23:34:00.849 INFO:teuthology.orchestra.run.smithi169.stdout: bluestore compression mode = aggressive 2024-04-24T23:34:00.849 INFO:teuthology.orchestra.run.smithi169.stdout: bluestore fsck on mount = True 2024-04-24T23:34:00.849 INFO:teuthology.orchestra.run.smithi169.stdout: debug bluefs = 20 2024-04-24T23:34:00.849 INFO:teuthology.orchestra.run.smithi169.stdout: debug bluestore = 20 2024-04-24T23:34:00.849 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2024-04-24T23:34:00.849 INFO:teuthology.orchestra.run.smithi169.stdout: debug osd = 20 2024-04-24T23:34:00.849 INFO:teuthology.orchestra.run.smithi169.stdout: debug rocksdb = 10 2024-04-24T23:34:00.849 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd backfillfull_ratio = 0.85 2024-04-24T23:34:00.849 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd full ratio = 0.9 2024-04-24T23:34:00.849 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd nearfull ratio = 0.8 2024-04-24T23:34:00.849 INFO:teuthology.orchestra.run.smithi169.stdout: osd failsafe full ratio = 0.95 2024-04-24T23:34:00.849 INFO:teuthology.orchestra.run.smithi169.stdout: osd objectstore = bluestore 2024-04-24T23:34:00.850 INFO:teuthology.orchestra.run.smithi169.stdout: osd op complaint time = 180 2024-04-24T23:34:00.850 INFO:teuthology.orchestra.run.smithi169.stdout: osd shutdown pgref assert = True 2024-04-24T23:34:00.850 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-24T23:34:00.850 INFO:teuthology.orchestra.run.smithi169.stdout:[mgr] 2024-04-24T23:34:00.850 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2024-04-24T23:34:00.850 INFO:teuthology.orchestra.run.smithi169.stdout: debug mgr = 20 2024-04-24T23:34:00.850 INFO:teuthology.orchestra.run.smithi169.stdout: debug mon = 20 2024-04-24T23:34:00.850 INFO:teuthology.orchestra.run.smithi169.stdout: debug auth = 20 2024-04-24T23:34:00.850 INFO:teuthology.orchestra.run.smithi169.stdout: mon reweight min pgs per osd = 4 2024-04-24T23:34:00.850 INFO:teuthology.orchestra.run.smithi169.stdout: mon reweight min bytes per osd = 10 2024-04-24T23:34:00.850 INFO:teuthology.orchestra.run.smithi169.stdout: mgr/telemetry/nag = false 2024-04-24T23:34:00.850 INFO:teuthology.orchestra.run.smithi169.stdout: debug client = 20 2024-04-24T23:34:00.851 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-24T23:34:00.851 INFO:teuthology.orchestra.run.smithi169.stdout:[mon] 2024-04-24T23:34:00.851 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2024-04-24T23:34:00.851 INFO:teuthology.orchestra.run.smithi169.stdout: debug mon = 20 2024-04-24T23:34:00.851 INFO:teuthology.orchestra.run.smithi169.stdout: debug paxos = 20 2024-04-24T23:34:00.851 INFO:teuthology.orchestra.run.smithi169.stdout: debug auth = 20 2024-04-24T23:34:00.851 INFO:teuthology.orchestra.run.smithi169.stdout: mon data avail warn = 5 2024-04-24T23:34:00.851 INFO:teuthology.orchestra.run.smithi169.stdout: mon mgr mkfs grace = 240 2024-04-24T23:34:00.851 INFO:teuthology.orchestra.run.smithi169.stdout: mon reweight min pgs per osd = 4 2024-04-24T23:34:00.851 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd reporter subtree level = osd 2024-04-24T23:34:00.851 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd prime pg temp = true 2024-04-24T23:34:00.851 INFO:teuthology.orchestra.run.smithi169.stdout: mon reweight min bytes per osd = 10 2024-04-24T23:34:00.851 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-24T23:34:00.851 INFO:teuthology.orchestra.run.smithi169.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-24T23:34:00.852 INFO:teuthology.orchestra.run.smithi169.stdout: auth mon ticket ttl = 660 # 11m 2024-04-24T23:34:00.852 INFO:teuthology.orchestra.run.smithi169.stdout: auth service ticket ttl = 240 # 4m 2024-04-24T23:34:00.852 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-24T23:34:00.852 INFO:teuthology.orchestra.run.smithi169.stdout: # don't complain about insecure global_id in the test suite 2024-04-24T23:34:00.852 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-24T23:34:00.852 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-24T23:34:00.852 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-24T23:34:00.852 INFO:teuthology.orchestra.run.smithi169.stdout: # 1m isn't quite enough 2024-04-24T23:34:00.852 INFO:teuthology.orchestra.run.smithi169.stdout: mon_down_mkfs_grace = 2m 2024-04-24T23:34:00.852 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-24T23:34:00.852 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_filestore_osds = false 2024-04-24T23:34:00.852 INFO:teuthology.orchestra.run.smithi169.stdout: mon down mkfs grace = 300 2024-04-24T23:34:00.852 INFO:teuthology.orchestra.run.smithi169.stdout: mon op complaint time = 120 2024-04-24T23:34:00.853 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-24T23:34:00.853 INFO:teuthology.orchestra.run.smithi169.stdout:[client] 2024-04-24T23:34:00.853 INFO:teuthology.orchestra.run.smithi169.stdout: rgw cache enabled = true 2024-04-24T23:34:00.853 INFO:teuthology.orchestra.run.smithi169.stdout: rgw enable ops log = true 2024-04-24T23:34:00.853 INFO:teuthology.orchestra.run.smithi169.stdout: rgw enable usage log = true 2024-04-24T23:34:00.853 INFO:teuthology.orchestra.run.smithi169.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-24T23:34:00.853 INFO:teuthology.orchestra.run.smithi169.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-24T23:34:00.853 INFO:teuthology.orchestra.run.smithi169.stdout: client mount timeout = 600 2024-04-24T23:34:00.853 INFO:teuthology.orchestra.run.smithi169.stdout: debug client = 20 2024-04-24T23:34:00.853 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2024-04-24T23:34:00.853 INFO:teuthology.orchestra.run.smithi169.stdout: rados mon op timeout = 900 2024-04-24T23:34:00.853 INFO:teuthology.orchestra.run.smithi169.stdout: rados osd op timeout = 900 2024-04-24T23:34:00.853 INFO:teuthology.orchestra.run.smithi169.stdout:[mon.a] 2024-04-24T23:34:00.854 INFO:teuthology.orchestra.run.smithi169.stdout:[mon.b] 2024-04-24T23:34:00.854 INFO:teuthology.orchestra.run.smithi169.stdout:[mon.c] 2024-04-24T23:34:00.854 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.a] 2024-04-24T23:34:00.854 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.c] 2024-04-24T23:34:00.854 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.b] 2024-04-24T23:34:00.854 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.d] 2024-04-24T23:34:00.854 INFO:teuthology.orchestra.run.smithi169.stdout:[mds] 2024-04-24T23:34:00.854 INFO:teuthology.orchestra.run.smithi169.stdout: debug mds = 20 2024-04-24T23:34:00.854 INFO:teuthology.orchestra.run.smithi169.stdout: debug mds balancer = 20 2024-04-24T23:34:00.854 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2024-04-24T23:34:00.854 INFO:teuthology.orchestra.run.smithi169.stdout: mds debug frag = True 2024-04-24T23:34:00.854 INFO:teuthology.orchestra.run.smithi169.stdout: mds debug scatterstat = True 2024-04-24T23:34:00.854 INFO:teuthology.orchestra.run.smithi169.stdout: mds op complaint time = 180 2024-04-24T23:34:00.855 INFO:teuthology.orchestra.run.smithi169.stdout: mds verify scatter = True 2024-04-24T23:34:00.855 INFO:teuthology.orchestra.run.smithi169.stdout: osd op complaint time = 180 2024-04-24T23:34:00.855 INFO:teuthology.orchestra.run.smithi169.stdout: rados mon op timeout = 900 2024-04-24T23:34:00.855 INFO:teuthology.orchestra.run.smithi169.stdout: rados osd op timeout = 900 2024-04-24T23:34:00.856 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-24T23:34:00.856 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-24T23:34:00.947 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-24T23:34:00.947 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-24T23:34:00.947 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-24T23:34:00.961 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-24T23:34:00.962 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-24T23:34:01.016 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi157.front.sepia.ceph.com 2024-04-24T23:34:01.016 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-24T23:34:01.016 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-24T23:34:01.016 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-24T23:34:01.090 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-24T23:34:01.090 DEBUG:teuthology.orchestra.run.smithi157:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-24T23:34:01.145 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi169.front.sepia.ceph.com 2024-04-24T23:34:01.146 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-24T23:34:01.146 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-24T23:34:01.146 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-24T23:34:01.179 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-24T23:34:01.179 DEBUG:teuthology.orchestra.run.smithi169:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-24T23:34:01.235 INFO:tasks.ceph:Setting up mon nodes... 2024-04-24T23:34:01.235 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-24T23:34:01.235 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-24T23:34:01.284 INFO:teuthology.orchestra.run.smithi157.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-24T23:34:01.287 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-24T23:34:01.344 INFO:teuthology.orchestra.run.smithi169.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-24T23:34:01.347 INFO:tasks.ceph:Setting up mds nodes... 2024-04-24T23:34:01.347 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-24T23:34:01.396 INFO:teuthology.orchestra.run.smithi157.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-24T23:34:01.399 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-24T23:34:01.422 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-24T23:34:01.513 INFO:teuthology.orchestra.run.smithi157.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-24T23:34:01.515 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-24T23:34:01.539 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-24T23:34:01.587 INFO:teuthology.orchestra.run.smithi169.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-24T23:34:01.589 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-24T23:34:01.612 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-24T23:34:01.700 INFO:teuthology.orchestra.run.smithi169.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-24T23:34:01.703 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-24T23:34:01.728 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-24T23:34:01.728 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-24T23:34:01.768 INFO:teuthology.orchestra.run.smithi157.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-24T23:34:01.780 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-24T23:34:01.818 INFO:teuthology.orchestra.run.smithi169.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-24T23:34:01.830 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-24T23:34:01.831 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi157.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi169.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-24T23:34:01.831 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-24T23:34:01.854 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-24T23:34:01.854 INFO:tasks.ceph:role: osd.0 2024-04-24T23:34:01.854 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi157.front.sepia.ceph.com 2024-04-24T23:34:01.854 DEBUG:teuthology.orchestra.run.smithi157:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-24T23:34:01.917 INFO:teuthology.orchestra.run.smithi157.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T23:34:01.918 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T23:34:01.918 INFO:teuthology.orchestra.run.smithi157.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T23:34:01.918 INFO:teuthology.orchestra.run.smithi157.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-24T23:34:01.918 INFO:teuthology.orchestra.run.smithi157.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T23:34:01.918 INFO:teuthology.orchestra.run.smithi157.stdout: = sunit=0 swidth=0 blks 2024-04-24T23:34:01.918 INFO:teuthology.orchestra.run.smithi157.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T23:34:01.918 INFO:teuthology.orchestra.run.smithi157.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-24T23:34:01.918 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T23:34:01.918 INFO:teuthology.orchestra.run.smithi157.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T23:34:01.950 INFO:teuthology.orchestra.run.smithi157.stdout:Discarding blocks...Done. 2024-04-24T23:34:01.952 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi157.front.sepia.ceph.com -o noatime 2024-04-24T23:34:01.952 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-24T23:34:02.025 DEBUG:teuthology.orchestra.run.smithi157:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-24T23:34:02.095 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-24T23:34:02.159 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-24T23:34:02.159 INFO:tasks.ceph:role: osd.1 2024-04-24T23:34:02.159 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi157.front.sepia.ceph.com 2024-04-24T23:34:02.159 DEBUG:teuthology.orchestra.run.smithi157:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-24T23:34:02.222 INFO:teuthology.orchestra.run.smithi157.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T23:34:02.222 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T23:34:02.222 INFO:teuthology.orchestra.run.smithi157.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T23:34:02.222 INFO:teuthology.orchestra.run.smithi157.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-24T23:34:02.222 INFO:teuthology.orchestra.run.smithi157.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T23:34:02.222 INFO:teuthology.orchestra.run.smithi157.stdout: = sunit=0 swidth=0 blks 2024-04-24T23:34:02.222 INFO:teuthology.orchestra.run.smithi157.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T23:34:02.222 INFO:teuthology.orchestra.run.smithi157.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-24T23:34:02.222 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T23:34:02.222 INFO:teuthology.orchestra.run.smithi157.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T23:34:02.254 INFO:teuthology.orchestra.run.smithi157.stdout:Discarding blocks...Done. 2024-04-24T23:34:02.256 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi157.front.sepia.ceph.com -o noatime 2024-04-24T23:34:02.256 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-24T23:34:02.334 DEBUG:teuthology.orchestra.run.smithi157:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-24T23:34:02.402 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-24T23:34:02.465 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-24T23:34:02.466 INFO:tasks.ceph:role: osd.2 2024-04-24T23:34:02.466 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi157.front.sepia.ceph.com 2024-04-24T23:34:02.466 DEBUG:teuthology.orchestra.run.smithi157:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-24T23:34:02.528 INFO:teuthology.orchestra.run.smithi157.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T23:34:02.528 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T23:34:02.528 INFO:teuthology.orchestra.run.smithi157.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T23:34:02.528 INFO:teuthology.orchestra.run.smithi157.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-24T23:34:02.528 INFO:teuthology.orchestra.run.smithi157.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T23:34:02.528 INFO:teuthology.orchestra.run.smithi157.stdout: = sunit=0 swidth=0 blks 2024-04-24T23:34:02.528 INFO:teuthology.orchestra.run.smithi157.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T23:34:02.529 INFO:teuthology.orchestra.run.smithi157.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-24T23:34:02.529 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T23:34:02.529 INFO:teuthology.orchestra.run.smithi157.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T23:34:02.560 INFO:teuthology.orchestra.run.smithi157.stdout:Discarding blocks...Done. 2024-04-24T23:34:02.562 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi157.front.sepia.ceph.com -o noatime 2024-04-24T23:34:02.562 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-24T23:34:02.629 DEBUG:teuthology.orchestra.run.smithi157:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-24T23:34:02.700 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-24T23:34:02.763 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-24T23:34:02.763 INFO:tasks.ceph:role: osd.3 2024-04-24T23:34:02.763 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi157.front.sepia.ceph.com 2024-04-24T23:34:02.763 DEBUG:teuthology.orchestra.run.smithi157:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-24T23:34:02.826 INFO:teuthology.orchestra.run.smithi157.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T23:34:02.827 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T23:34:02.827 INFO:teuthology.orchestra.run.smithi157.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T23:34:02.827 INFO:teuthology.orchestra.run.smithi157.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-24T23:34:02.827 INFO:teuthology.orchestra.run.smithi157.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T23:34:02.827 INFO:teuthology.orchestra.run.smithi157.stdout: = sunit=0 swidth=0 blks 2024-04-24T23:34:02.827 INFO:teuthology.orchestra.run.smithi157.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T23:34:02.827 INFO:teuthology.orchestra.run.smithi157.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-24T23:34:02.827 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T23:34:02.827 INFO:teuthology.orchestra.run.smithi157.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T23:34:02.859 INFO:teuthology.orchestra.run.smithi157.stdout:Discarding blocks...Done. 2024-04-24T23:34:02.861 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi157.front.sepia.ceph.com -o noatime 2024-04-24T23:34:02.861 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-24T23:34:02.928 DEBUG:teuthology.orchestra.run.smithi157:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-24T23:34:02.996 DEBUG:teuthology.orchestra.run.smithi157:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T23:34:03.095 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:03.094+0000 7f2f02c0c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-24T23:34:03.095 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:03.094+0000 7f2f02c0c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-24T23:34:03.095 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:03.094+0000 7f2f02c0c640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-24T23:34:03.095 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:03.094+0000 7f2f02c0c640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-24T23:34:03.095 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:03.094+0000 7f2f02c0c640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-24T23:34:03.095 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:03.094+0000 7f2f02c0c640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-24T23:34:05.940 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-24T23:34:05.968 DEBUG:teuthology.orchestra.run.smithi157:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T23:34:06.057 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:06.056+0000 7f334a301640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-24T23:34:06.057 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:06.056+0000 7f334a301640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-24T23:34:06.057 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:06.056+0000 7f334a301640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-24T23:34:06.057 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:06.056+0000 7f334a301640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-24T23:34:06.057 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:06.056+0000 7f334a301640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-24T23:34:06.058 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:06.056+0000 7f334a301640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-24T23:34:08.918 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-24T23:34:08.946 DEBUG:teuthology.orchestra.run.smithi157:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T23:34:09.035 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:09.034+0000 7f8e28029640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-24T23:34:09.035 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:09.034+0000 7f8e28029640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-24T23:34:09.035 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:09.034+0000 7f8e28029640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-24T23:34:09.035 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:09.034+0000 7f8e28029640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-24T23:34:09.035 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:09.034+0000 7f8e28029640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-24T23:34:09.035 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:09.035+0000 7f8e28029640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-24T23:34:11.874 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-24T23:34:11.898 DEBUG:teuthology.orchestra.run.smithi157:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T23:34:11.981 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:11.980+0000 7fdbe9a0f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-24T23:34:11.982 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:11.980+0000 7fdbe9a0f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-24T23:34:11.982 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:11.980+0000 7fdbe9a0f640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-24T23:34:11.982 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:11.980+0000 7fdbe9a0f640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-24T23:34:11.982 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:11.980+0000 7fdbe9a0f640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-24T23:34:11.982 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:11.981+0000 7fdbe9a0f640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-24T23:34:14.823 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-24T23:34:14.891 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-24T23:34:14.920 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-24T23:34:14.920 INFO:tasks.ceph:role: osd.4 2024-04-24T23:34:14.920 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi169.front.sepia.ceph.com 2024-04-24T23:34:14.921 DEBUG:teuthology.orchestra.run.smithi169:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-24T23:34:14.995 INFO:teuthology.orchestra.run.smithi169.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T23:34:14.996 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T23:34:14.996 INFO:teuthology.orchestra.run.smithi169.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T23:34:14.996 INFO:teuthology.orchestra.run.smithi169.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-24T23:34:14.996 INFO:teuthology.orchestra.run.smithi169.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T23:34:14.996 INFO:teuthology.orchestra.run.smithi169.stdout: = sunit=0 swidth=0 blks 2024-04-24T23:34:14.996 INFO:teuthology.orchestra.run.smithi169.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T23:34:14.996 INFO:teuthology.orchestra.run.smithi169.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-24T23:34:14.996 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T23:34:14.996 INFO:teuthology.orchestra.run.smithi169.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T23:34:15.521 INFO:teuthology.orchestra.run.smithi169.stdout:Discarding blocks...Done. 2024-04-24T23:34:15.524 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi169.front.sepia.ceph.com -o noatime 2024-04-24T23:34:15.524 DEBUG:teuthology.orchestra.run.smithi169:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-24T23:34:15.622 DEBUG:teuthology.orchestra.run.smithi169:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-24T23:34:15.654 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-24T23:34:15.719 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-24T23:34:15.719 INFO:tasks.ceph:role: osd.5 2024-04-24T23:34:15.719 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi169.front.sepia.ceph.com 2024-04-24T23:34:15.719 DEBUG:teuthology.orchestra.run.smithi169:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-24T23:34:15.789 INFO:teuthology.orchestra.run.smithi169.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T23:34:15.789 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T23:34:15.789 INFO:teuthology.orchestra.run.smithi169.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T23:34:15.789 INFO:teuthology.orchestra.run.smithi169.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-24T23:34:15.789 INFO:teuthology.orchestra.run.smithi169.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T23:34:15.789 INFO:teuthology.orchestra.run.smithi169.stdout: = sunit=0 swidth=0 blks 2024-04-24T23:34:15.790 INFO:teuthology.orchestra.run.smithi169.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T23:34:15.790 INFO:teuthology.orchestra.run.smithi169.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-24T23:34:15.790 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T23:34:15.790 INFO:teuthology.orchestra.run.smithi169.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T23:34:16.249 INFO:teuthology.orchestra.run.smithi169.stdout:Discarding blocks...Done. 2024-04-24T23:34:16.251 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi169.front.sepia.ceph.com -o noatime 2024-04-24T23:34:16.251 DEBUG:teuthology.orchestra.run.smithi169:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-24T23:34:16.329 DEBUG:teuthology.orchestra.run.smithi169:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-24T23:34:16.399 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-24T23:34:16.462 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-24T23:34:16.462 INFO:tasks.ceph:role: osd.6 2024-04-24T23:34:16.462 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi169.front.sepia.ceph.com 2024-04-24T23:34:16.463 DEBUG:teuthology.orchestra.run.smithi169:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-24T23:34:16.534 INFO:teuthology.orchestra.run.smithi169.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T23:34:16.534 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T23:34:16.534 INFO:teuthology.orchestra.run.smithi169.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T23:34:16.534 INFO:teuthology.orchestra.run.smithi169.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-24T23:34:16.534 INFO:teuthology.orchestra.run.smithi169.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T23:34:16.534 INFO:teuthology.orchestra.run.smithi169.stdout: = sunit=0 swidth=0 blks 2024-04-24T23:34:16.535 INFO:teuthology.orchestra.run.smithi169.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T23:34:16.535 INFO:teuthology.orchestra.run.smithi169.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-24T23:34:16.535 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T23:34:16.535 INFO:teuthology.orchestra.run.smithi169.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T23:34:17.028 INFO:teuthology.orchestra.run.smithi169.stdout:Discarding blocks...Done. 2024-04-24T23:34:17.030 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi169.front.sepia.ceph.com -o noatime 2024-04-24T23:34:17.030 DEBUG:teuthology.orchestra.run.smithi169:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-24T23:34:17.132 DEBUG:teuthology.orchestra.run.smithi169:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-24T23:34:17.160 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-24T23:34:17.224 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-24T23:34:17.224 INFO:tasks.ceph:role: osd.7 2024-04-24T23:34:17.224 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi169.front.sepia.ceph.com 2024-04-24T23:34:17.225 DEBUG:teuthology.orchestra.run.smithi169:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-24T23:34:17.296 INFO:teuthology.orchestra.run.smithi169.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T23:34:17.297 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T23:34:17.297 INFO:teuthology.orchestra.run.smithi169.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T23:34:17.297 INFO:teuthology.orchestra.run.smithi169.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-24T23:34:17.297 INFO:teuthology.orchestra.run.smithi169.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T23:34:17.297 INFO:teuthology.orchestra.run.smithi169.stdout: = sunit=0 swidth=0 blks 2024-04-24T23:34:17.297 INFO:teuthology.orchestra.run.smithi169.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T23:34:17.297 INFO:teuthology.orchestra.run.smithi169.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-24T23:34:17.297 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T23:34:17.298 INFO:teuthology.orchestra.run.smithi169.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T23:34:17.807 INFO:teuthology.orchestra.run.smithi169.stdout:Discarding blocks...Done. 2024-04-24T23:34:17.809 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi169.front.sepia.ceph.com -o noatime 2024-04-24T23:34:17.810 DEBUG:teuthology.orchestra.run.smithi169:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-24T23:34:17.961 DEBUG:teuthology.orchestra.run.smithi169:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-24T23:34:17.993 DEBUG:teuthology.orchestra.run.smithi169:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T23:34:18.096 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:18.096+0000 7fb615619640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-24T23:34:18.099 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:18.099+0000 7fb615619640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-24T23:34:18.099 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:18.099+0000 7fb615619640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-24T23:34:18.100 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:18.099+0000 7fb615619640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-24T23:34:18.100 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:18.099+0000 7fb615619640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-24T23:34:18.107 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:18.107+0000 7fb615619640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-24T23:34:20.952 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-24T23:34:20.977 DEBUG:teuthology.orchestra.run.smithi169:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T23:34:21.063 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:21.062+0000 7fa9bf40f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-24T23:34:21.063 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:21.063+0000 7fa9bf40f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-24T23:34:21.063 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:21.063+0000 7fa9bf40f640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-24T23:34:21.063 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:21.063+0000 7fa9bf40f640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-24T23:34:21.063 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:21.063+0000 7fa9bf40f640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-24T23:34:21.063 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:21.063+0000 7fa9bf40f640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-24T23:34:23.907 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-24T23:34:23.933 DEBUG:teuthology.orchestra.run.smithi169:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T23:34:24.018 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:24.017+0000 7fa299126640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-24T23:34:24.019 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:24.017+0000 7fa299126640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-24T23:34:24.019 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:24.017+0000 7fa299126640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-24T23:34:24.019 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:24.017+0000 7fa299126640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-24T23:34:24.019 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:24.017+0000 7fa299126640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-24T23:34:24.019 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:24.018+0000 7fa299126640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-24T23:34:26.865 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-24T23:34:26.889 DEBUG:teuthology.orchestra.run.smithi169:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T23:34:26.974 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:26.972+0000 7fecbe21a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-24T23:34:26.974 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:26.973+0000 7fecbe21a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-24T23:34:26.974 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:26.973+0000 7fecbe21a640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-24T23:34:26.974 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:26.973+0000 7fecbe21a640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-24T23:34:26.974 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:26.973+0000 7fecbe21a640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-24T23:34:26.975 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:26.973+0000 7fecbe21a640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-24T23:34:28.879 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:34:28.882 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:34:29.827 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-24T23:34:29.855 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-24T23:34:29.855 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-24T23:34:29.855 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-24T23:34:29.884 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-24T23:34:29.884 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-24T23:34:29.953 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-24T23:34:29.953 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-24T23:34:30.020 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-24T23:34:30.020 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-24T23:34:30.086 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-24T23:34:30.086 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-24T23:34:30.152 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-24T23:34:30.152 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-24T23:34:30.218 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-24T23:34:30.218 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-24T23:34:30.282 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-24T23:34:30.282 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-24T23:34:30.310 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-24T23:34:30.310 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-24T23:34:30.378 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-24T23:34:30.378 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-24T23:34:30.444 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-24T23:34:30.444 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-24T23:34:30.508 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-24T23:34:30.509 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-24T23:34:30.573 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-24T23:34:30.573 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-24T23:34:30.638 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-24T23:34:30.638 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-24T23:34:30.702 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-24T23:34:30.702 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-24T23:34:30.719 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-24T23:34:30.719 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-24T23:34:30.757 INFO:tasks.ceph:Adding keys to all mons... 2024-04-24T23:34:30.757 DEBUG:teuthology.orchestra.run.smithi157:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-24T23:34:30.762 DEBUG:teuthology.orchestra.run.smithi169:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-24T23:34:30.800 INFO:teuthology.orchestra.run.smithi157.stdout:[mgr.y] 2024-04-24T23:34:30.800 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDplilmblADERAARwmWPUmnajRZ97kL9cE31Q== 2024-04-24T23:34:30.800 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.a] 2024-04-24T23:34:30.800 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDplilm0L2wFxAATEjESxuhrF6T5oCrK1Javg== 2024-04-24T23:34:30.800 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.c] 2024-04-24T23:34:30.800 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDplilmCtKkHhAAX76strlhhzzlxFGblIMKLw== 2024-04-24T23:34:30.800 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.0] 2024-04-24T23:34:30.800 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDrlilmZ6WyBRAA3UbYHjjwl6WCEcD/lzgwSQ== 2024-04-24T23:34:30.801 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.1] 2024-04-24T23:34:30.801 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDulilm0DltAxAALGdoYe++sNr3Q5oyWGtVuw== 2024-04-24T23:34:30.801 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.2] 2024-04-24T23:34:30.801 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDxlilmKTwjAhAAyU+iScDfBat4Le9lWpQAOw== 2024-04-24T23:34:30.801 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.3] 2024-04-24T23:34:30.801 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDzlilm73CIOhAApIePpncoyVB+KRv6VZ2QQA== 2024-04-24T23:34:30.801 INFO:teuthology.orchestra.run.smithi157.stdout:[mgr.x] 2024-04-24T23:34:30.801 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDplilmWDWZFBAArHaOkitjNkZJGVHffZuyOQ== 2024-04-24T23:34:30.801 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.b] 2024-04-24T23:34:30.801 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDplilmV8AQIxAALN1mrQBTFgBlwHAFNcMr9g== 2024-04-24T23:34:30.801 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.d] 2024-04-24T23:34:30.801 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDplilmnLvSKRAArEV2YFiCVXwtJ0Snf6Vd/w== 2024-04-24T23:34:30.802 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.4] 2024-04-24T23:34:30.802 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQD6lilmPBHSBRAAErstD+Xo+A3HAdp444jzDg== 2024-04-24T23:34:30.802 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.5] 2024-04-24T23:34:30.802 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQD9lilmqhrLAxAAiDRZ1AgibXN96RDsa6kVTA== 2024-04-24T23:34:30.802 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.6] 2024-04-24T23:34:30.802 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQAAlylmjIYoARAAkeTchr5QWEBrS6LqIC217g== 2024-04-24T23:34:30.802 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.7] 2024-04-24T23:34:30.802 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQAClylmgV0XOhAA0oI807pOjD507I2oOuIbbg== 2024-04-24T23:34:30.802 INFO:teuthology.orchestra.run.smithi157.stdout:[client.0] 2024-04-24T23:34:30.802 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDplilmdlbeLRAA/NRgMe/sy7IgOw7K048o+g== 2024-04-24T23:34:30.802 INFO:teuthology.orchestra.run.smithi157.stdout:[client.1] 2024-04-24T23:34:30.803 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDplilm/Z3cMBAA9Kz3D/4TtmOgujVXY8KvPw== 2024-04-24T23:34:30.819 INFO:teuthology.orchestra.run.smithi169.stdout:[mgr.y] 2024-04-24T23:34:30.819 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQDplilmblADERAARwmWPUmnajRZ97kL9cE31Q== 2024-04-24T23:34:30.819 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.a] 2024-04-24T23:34:30.819 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQDplilm0L2wFxAATEjESxuhrF6T5oCrK1Javg== 2024-04-24T23:34:30.819 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.c] 2024-04-24T23:34:30.820 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQDplilmCtKkHhAAX76strlhhzzlxFGblIMKLw== 2024-04-24T23:34:30.820 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.0] 2024-04-24T23:34:30.820 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQDrlilmZ6WyBRAA3UbYHjjwl6WCEcD/lzgwSQ== 2024-04-24T23:34:30.820 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.1] 2024-04-24T23:34:30.820 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQDulilm0DltAxAALGdoYe++sNr3Q5oyWGtVuw== 2024-04-24T23:34:30.820 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.2] 2024-04-24T23:34:30.820 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQDxlilmKTwjAhAAyU+iScDfBat4Le9lWpQAOw== 2024-04-24T23:34:30.820 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.3] 2024-04-24T23:34:30.820 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQDzlilm73CIOhAApIePpncoyVB+KRv6VZ2QQA== 2024-04-24T23:34:30.820 INFO:teuthology.orchestra.run.smithi169.stdout:[mgr.x] 2024-04-24T23:34:30.820 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQDplilmWDWZFBAArHaOkitjNkZJGVHffZuyOQ== 2024-04-24T23:34:30.820 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.b] 2024-04-24T23:34:30.820 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQDplilmV8AQIxAALN1mrQBTFgBlwHAFNcMr9g== 2024-04-24T23:34:30.821 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.d] 2024-04-24T23:34:30.821 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQDplilmnLvSKRAArEV2YFiCVXwtJ0Snf6Vd/w== 2024-04-24T23:34:30.821 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.4] 2024-04-24T23:34:30.821 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQD6lilmPBHSBRAAErstD+Xo+A3HAdp444jzDg== 2024-04-24T23:34:30.821 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.5] 2024-04-24T23:34:30.821 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQD9lilmqhrLAxAAiDRZ1AgibXN96RDsa6kVTA== 2024-04-24T23:34:30.821 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.6] 2024-04-24T23:34:30.821 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQAAlylmjIYoARAAkeTchr5QWEBrS6LqIC217g== 2024-04-24T23:34:30.821 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.7] 2024-04-24T23:34:30.821 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQAClylmgV0XOhAA0oI807pOjD507I2oOuIbbg== 2024-04-24T23:34:30.821 INFO:teuthology.orchestra.run.smithi169.stdout:[client.0] 2024-04-24T23:34:30.821 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQDplilmdlbeLRAA/NRgMe/sy7IgOw7K048o+g== 2024-04-24T23:34:30.822 INFO:teuthology.orchestra.run.smithi169.stdout:[client.1] 2024-04-24T23:34:30.822 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQDplilm/Z3cMBAA9Kz3D/4TtmOgujVXY8KvPw== 2024-04-24T23:34:30.823 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-24T23:34:30.846 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-24T23:34:30.912 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T23:34:30.914 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T23:34:30.965 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T23:34:30.967 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T23:34:31.018 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T23:34:31.020 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T23:34:31.068 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T23:34:31.070 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T23:34:31.119 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T23:34:31.121 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T23:34:31.169 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T23:34:31.171 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T23:34:31.220 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-24T23:34:31.222 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-24T23:34:31.270 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T23:34:31.272 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T23:34:31.324 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T23:34:31.327 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T23:34:31.378 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T23:34:31.381 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T23:34:31.429 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T23:34:31.431 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T23:34:31.480 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T23:34:31.482 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T23:34:31.530 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T23:34:31.533 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T23:34:31.581 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-24T23:34:31.583 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-24T23:34:31.635 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-24T23:34:31.638 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-24T23:34:31.689 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-24T23:34:31.689 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-24T23:34:31.712 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-24T23:34:31.806 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-24T23:34:31.829 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-24T23:34:31.852 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-24T23:34:31.945 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-24T23:34:31.968 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-24T23:34:32.032 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-24T23:34:32.125 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-24T23:34:32.148 DEBUG:teuthology.orchestra.run.smithi157:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-24T23:34:32.150 DEBUG:teuthology.orchestra.run.smithi169:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-24T23:34:32.202 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-24T23:34:32.203 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-24T23:34:32.203 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mon -f --cluster ceph -i a 2024-04-24T23:34:32.205 INFO:tasks.ceph.mon.a:Started 2024-04-24T23:34:32.206 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-24T23:34:32.206 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mon -f --cluster ceph -i b 2024-04-24T23:34:32.245 INFO:tasks.ceph.mon.b:Started 2024-04-24T23:34:32.245 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-24T23:34:32.245 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mon -f --cluster ceph -i c 2024-04-24T23:34:32.247 INFO:tasks.ceph.mon.c:Started 2024-04-24T23:34:32.247 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-24T23:34:32.247 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-24T23:34:32.248 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mgr -f --cluster ceph -i y 2024-04-24T23:34:32.250 INFO:tasks.ceph.mgr.y:Started 2024-04-24T23:34:32.250 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-24T23:34:32.250 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mgr -f --cluster ceph -i x 2024-04-24T23:34:32.252 INFO:tasks.ceph.mgr.x:Started 2024-04-24T23:34:32.252 INFO:tasks.ceph:Setting crush tunables to default 2024-04-24T23:34:32.252 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd crush tunables default 2024-04-24T23:34:32.350 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:32.350+0000 7f702edc1640 1 Processor -- start 2024-04-24T23:34:32.351 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:32.350+0000 7f702edc1640 1 -- start start 2024-04-24T23:34:32.352 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:32.350+0000 7f702edc1640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70280c8710 0x7f7028155340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:32.352 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:32.350+0000 7f702edc1640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7028070a80 0x7f70280c4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:32.353 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:32.351+0000 7f702edc1640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7028156330 con 0x7f70280706a0 2024-04-24T23:34:32.353 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:32.351+0000 7f702edc1640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f70280c5170 con 0x7f7028070a80 2024-04-24T23:34:32.353 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:32.351+0000 7f702edc1640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7028158930 con 0x7f70280c8710 2024-04-24T23:34:32.353 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:32.351+0000 7f702d35e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7028070a80 0x7f70280c4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:32.353 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:32.351+0000 7f702d35e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7028070a80 0x7f70280c4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:49064/0 (socket says 172.21.15.157:49064) 2024-04-24T23:34:32.353 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:32.351+0000 7f702d35e640 1 -- 172.21.15.157:0/259469813 learned_addr learned my addr 172.21.15.157:0/259469813 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:34:32.353 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:32.351+0000 7f702d35e640 1 -- 172.21.15.157:0/259469813 >> v2:172.21.15.157:3300/0 conn(0x7f7028070a80 msgr2=0x7f70280c4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-24T23:34:32.353 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:32.351+0000 7f702d35e640 1 -- 172.21.15.157:0/259469813 >> v2:172.21.15.157:3300/0 conn(0x7f7028070a80 msgr2=0x7f70280c4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T23:34:32.353 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:32.351+0000 7f702d35e640 1 --2- 172.21.15.157:0/259469813 >> v2:172.21.15.157:3300/0 conn(0x7f7028070a80 0x7f70280c4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T23:34:32.353 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:32.351+0000 7f702d35e640 1 --2- 172.21.15.157:0/259469813 >> v2:172.21.15.157:3300/0 conn(0x7f7028070a80 0x7f70280c4c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T23:34:32.353 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:32.351+0000 7f7027fff640 1 --2- 172.21.15.157:0/259469813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70280c8710 0x7f7028155340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:32.354 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:32.351+0000 7f7027fff640 1 -- 172.21.15.157:0/259469813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70280c8710 msgr2=0x7f7028155340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-24T23:34:32.354 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:32.351+0000 7f7027fff640 1 -- 172.21.15.157:0/259469813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70280c8710 msgr2=0x7f7028155340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T23:34:32.354 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:32.351+0000 7f7027fff640 1 --2- 172.21.15.157:0/259469813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70280c8710 0x7f7028155340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T23:34:32.354 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:32.351+0000 7f7027fff640 1 --2- 172.21.15.157:0/259469813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70280c8710 0x7f7028155340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T23:34:32.553 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:32.552+0000 7f702d35e640 1 --2- 172.21.15.157:0/259469813 >> v2:172.21.15.157:3300/0 conn(0x7f7028070a80 0x7f70280c4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:32.553 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:32.552+0000 7f702d35e640 1 -- 172.21.15.157:0/259469813 >> v2:172.21.15.157:3300/0 conn(0x7f7028070a80 msgr2=0x7f70280c4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-24T23:34:32.553 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:32.552+0000 7f702d35e640 1 -- 172.21.15.157:0/259469813 >> v2:172.21.15.157:3300/0 conn(0x7f7028070a80 msgr2=0x7f70280c4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T23:34:32.553 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:32.552+0000 7f702d35e640 1 --2- 172.21.15.157:0/259469813 >> v2:172.21.15.157:3300/0 conn(0x7f7028070a80 0x7f70280c4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T23:34:32.553 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:32.552+0000 7f7027fff640 1 --2- 172.21.15.157:0/259469813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70280c8710 0x7f7028155340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:32.553 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:32.552+0000 7f702d35e640 1 --2- 172.21.15.157:0/259469813 >> v2:172.21.15.157:3300/0 conn(0x7f7028070a80 0x7f70280c4c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T23:34:32.553 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:32.552+0000 7f7027fff640 1 -- 172.21.15.157:0/259469813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70280c8710 msgr2=0x7f7028155340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-24T23:34:32.554 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:32.552+0000 7f7027fff640 1 -- 172.21.15.157:0/259469813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70280c8710 msgr2=0x7f7028155340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T23:34:32.554 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:32.552+0000 7f7027fff640 1 --2- 172.21.15.157:0/259469813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70280c8710 0x7f7028155340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T23:34:32.554 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:32.552+0000 7f7027fff640 1 --2- 172.21.15.157:0/259469813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70280c8710 0x7f7028155340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T23:34:32.954 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:32.953+0000 7f702d35e640 1 --2- 172.21.15.157:0/259469813 >> v2:172.21.15.157:3300/0 conn(0x7f7028070a80 0x7f70280c4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:32.954 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:32.953+0000 7f702d35e640 1 -- 172.21.15.157:0/259469813 >> v2:172.21.15.157:3300/0 conn(0x7f7028070a80 msgr2=0x7f70280c4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-24T23:34:32.954 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:32.953+0000 7f702d35e640 1 -- 172.21.15.157:0/259469813 >> v2:172.21.15.157:3300/0 conn(0x7f7028070a80 msgr2=0x7f70280c4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T23:34:32.954 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:32.953+0000 7f702d35e640 1 --2- 172.21.15.157:0/259469813 >> v2:172.21.15.157:3300/0 conn(0x7f7028070a80 0x7f70280c4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T23:34:32.954 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:32.953+0000 7f702d35e640 1 --2- 172.21.15.157:0/259469813 >> v2:172.21.15.157:3300/0 conn(0x7f7028070a80 0x7f70280c4c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-24T23:34:32.954 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:32.953+0000 7f7027fff640 1 --2- 172.21.15.157:0/259469813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70280c8710 0x7f7028155340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:32.954 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:32.953+0000 7f7027fff640 1 -- 172.21.15.157:0/259469813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70280c8710 msgr2=0x7f7028155340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-24T23:34:32.954 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:32.953+0000 7f7027fff640 1 -- 172.21.15.157:0/259469813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70280c8710 msgr2=0x7f7028155340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T23:34:32.955 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:32.953+0000 7f7027fff640 1 --2- 172.21.15.157:0/259469813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70280c8710 0x7f7028155340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T23:34:32.955 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:32.953+0000 7f7027fff640 1 --2- 172.21.15.157:0/259469813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70280c8710 0x7f7028155340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-24T23:34:33.755 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:33.755+0000 7f702d35e640 1 --2- 172.21.15.157:0/259469813 >> v2:172.21.15.157:3300/0 conn(0x7f7028070a80 0x7f70280c4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:33.755 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:33.755+0000 7f702d35e640 1 -- 172.21.15.157:0/259469813 >> v2:172.21.15.157:3300/0 conn(0x7f7028070a80 msgr2=0x7f70280c4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-24T23:34:33.755 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:33.755+0000 7f702d35e640 1 -- 172.21.15.157:0/259469813 >> v2:172.21.15.157:3300/0 conn(0x7f7028070a80 msgr2=0x7f70280c4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T23:34:33.755 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:33.755+0000 7f702d35e640 1 --2- 172.21.15.157:0/259469813 >> v2:172.21.15.157:3300/0 conn(0x7f7028070a80 0x7f70280c4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T23:34:33.755 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:33.755+0000 7f702d35e640 1 --2- 172.21.15.157:0/259469813 >> v2:172.21.15.157:3300/0 conn(0x7f7028070a80 0x7f70280c4c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-24T23:34:33.756 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:33.755+0000 7f7027fff640 1 --2- 172.21.15.157:0/259469813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70280c8710 0x7f7028155340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:33.756 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:33.755+0000 7f7027fff640 1 -- 172.21.15.157:0/259469813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70280c8710 msgr2=0x7f7028155340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-24T23:34:33.756 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:33.755+0000 7f7027fff640 1 -- 172.21.15.157:0/259469813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70280c8710 msgr2=0x7f7028155340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T23:34:33.756 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:33.755+0000 7f7027fff640 1 --2- 172.21.15.157:0/259469813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70280c8710 0x7f7028155340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T23:34:33.756 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:33.755+0000 7f7027fff640 1 --2- 172.21.15.157:0/259469813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70280c8710 0x7f7028155340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-24T23:34:35.352 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:35.352+0000 7f70267fc640 1 -- 172.21.15.157:0/259469813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70280c8710 msgr2=0x7f7028155340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T23:34:35.352 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:35.352+0000 7f70267fc640 1 --2- 172.21.15.157:0/259469813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70280c8710 0x7f7028155340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:35.352 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:35.352+0000 7f70267fc640 1 -- 172.21.15.157:0/259469813 >> v2:172.21.15.157:3300/0 conn(0x7f7028070a80 msgr2=0x7f70280c4c30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T23:34:35.352 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:35.352+0000 7f70267fc640 1 --2- 172.21.15.157:0/259469813 >> v2:172.21.15.157:3300/0 conn(0x7f7028070a80 0x7f70280c4c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:35.352 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:35.352+0000 7f70267fc640 1 -- 172.21.15.157:0/259469813 >> v1:172.21.15.169:6789/0 conn(0x7f70280706a0 legacy=0x7f70281590a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:35.352 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:35.352+0000 7f70267fc640 1 --2- 172.21.15.157:0/259469813 >> v2:172.21.15.169:3300/0 conn(0x7f700c0036a0 0x7f700c005b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:35.353 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:35.352+0000 7f70267fc640 1 -- 172.21.15.157:0/259469813 --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f700c009ea0 con 0x7f700c006170 2024-04-24T23:34:35.353 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:35.352+0000 7f70267fc640 1 -- 172.21.15.157:0/259469813 --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f70280c5170 con 0x7f700c0036a0 2024-04-24T23:34:35.353 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:35.352+0000 7f702cb5d640 1 --2- 172.21.15.157:0/259469813 >> v2:172.21.15.169:3300/0 conn(0x7f700c0036a0 0x7f700c005b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:35.353 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:35.352+0000 7f702cb5d640 1 -- 172.21.15.157:0/259469813 >> v2:172.21.15.169:3300/0 conn(0x7f700c0036a0 msgr2=0x7f700c005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-24T23:34:35.353 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:35.352+0000 7f702cb5d640 1 -- 172.21.15.157:0/259469813 >> v2:172.21.15.169:3300/0 conn(0x7f700c0036a0 msgr2=0x7f700c005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T23:34:35.353 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:35.352+0000 7f702cb5d640 1 --2- 172.21.15.157:0/259469813 >> v2:172.21.15.169:3300/0 conn(0x7f700c0036a0 0x7f700c005b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T23:34:35.353 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:35.352+0000 7f702cb5d640 1 --2- 172.21.15.157:0/259469813 >> v2:172.21.15.169:3300/0 conn(0x7f700c0036a0 0x7f700c005b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T23:34:35.553 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:35.553+0000 7f702cb5d640 1 --2- 172.21.15.157:0/259469813 >> v2:172.21.15.169:3300/0 conn(0x7f700c0036a0 0x7f700c005b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:35.553 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:35.553+0000 7f702cb5d640 1 -- 172.21.15.157:0/259469813 >> v2:172.21.15.169:3300/0 conn(0x7f700c0036a0 msgr2=0x7f700c005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-24T23:34:35.553 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:35.553+0000 7f702cb5d640 1 -- 172.21.15.157:0/259469813 >> v2:172.21.15.169:3300/0 conn(0x7f700c0036a0 msgr2=0x7f700c005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T23:34:35.553 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:35.553+0000 7f702cb5d640 1 --2- 172.21.15.157:0/259469813 >> v2:172.21.15.169:3300/0 conn(0x7f700c0036a0 0x7f700c005b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T23:34:35.553 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:35.553+0000 7f702cb5d640 1 --2- 172.21.15.157:0/259469813 >> v2:172.21.15.169:3300/0 conn(0x7f700c0036a0 0x7f700c005b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T23:34:35.954 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:35.954+0000 7f702cb5d640 1 --2- 172.21.15.157:0/259469813 >> v2:172.21.15.169:3300/0 conn(0x7f700c0036a0 0x7f700c005b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:35.954 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:35.954+0000 7f702cb5d640 1 -- 172.21.15.157:0/259469813 >> v2:172.21.15.169:3300/0 conn(0x7f700c0036a0 msgr2=0x7f700c005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-24T23:34:35.954 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:35.954+0000 7f702cb5d640 1 -- 172.21.15.157:0/259469813 >> v2:172.21.15.169:3300/0 conn(0x7f700c0036a0 msgr2=0x7f700c005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T23:34:35.954 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:35.954+0000 7f702cb5d640 1 --2- 172.21.15.157:0/259469813 >> v2:172.21.15.169:3300/0 conn(0x7f700c0036a0 0x7f700c005b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T23:34:35.954 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:35.954+0000 7f702cb5d640 1 --2- 172.21.15.157:0/259469813 >> v2:172.21.15.169:3300/0 conn(0x7f700c0036a0 0x7f700c005b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-24T23:34:36.755 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:36.755+0000 7f702cb5d640 1 --2- 172.21.15.157:0/259469813 >> v2:172.21.15.169:3300/0 conn(0x7f700c0036a0 0x7f700c005b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:36.756 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:36.756+0000 7f702cb5d640 1 -- 172.21.15.157:0/259469813 >> v2:172.21.15.169:3300/0 conn(0x7f700c0036a0 msgr2=0x7f700c005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-24T23:34:36.756 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:36.756+0000 7f702cb5d640 1 -- 172.21.15.157:0/259469813 >> v2:172.21.15.169:3300/0 conn(0x7f700c0036a0 msgr2=0x7f700c005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T23:34:36.756 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:36.756+0000 7f702cb5d640 1 --2- 172.21.15.157:0/259469813 >> v2:172.21.15.169:3300/0 conn(0x7f700c0036a0 0x7f700c005b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T23:34:36.756 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:36.756+0000 7f702cb5d640 1 --2- 172.21.15.157:0/259469813 >> v2:172.21.15.169:3300/0 conn(0x7f700c0036a0 0x7f700c005b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-24T23:34:37.529 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.528+0000 7f70277fe640 1 -- 172.21.15.157:0/259469813 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1798607815 0 0) 0x7f700c009ea0 con 0x7f700c006170 2024-04-24T23:34:37.529 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.528+0000 7f70277fe640 1 -- 172.21.15.157:0/259469813 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7018003520 con 0x7f700c006170 2024-04-24T23:34:37.529 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.528+0000 7f70277fe640 1 -- 172.21.15.157:0/259469813 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4038816637 0 0) 0x7f7018003520 con 0x7f700c006170 2024-04-24T23:34:37.529 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.528+0000 7f70277fe640 1 -- 172.21.15.157:0/259469813 >> v2:172.21.15.169:3300/0 conn(0x7f700c0036a0 msgr2=0x7f700c005b20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T23:34:37.530 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.528+0000 7f70277fe640 1 --2- 172.21.15.157:0/259469813 >> v2:172.21.15.169:3300/0 conn(0x7f700c0036a0 0x7f700c005b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:37.530 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.528+0000 7f70277fe640 1 -- 172.21.15.157:0/259469813 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f700c00b150 con 0x7f700c006170 2024-04-24T23:34:37.530 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.528+0000 7f70277fe640 1 -- 172.21.15.157:0/259469813 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1480525340 0 0) 0x7f701400d580 con 0x7f700c006170 2024-04-24T23:34:37.530 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.528+0000 7f70277fe640 1 -- 172.21.15.157:0/259469813 >> v1:172.21.15.157:6789/0 conn(0x7f700c006170 legacy=0x7f700c008630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:37.530 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.528+0000 7f702d35e640 1 -- 172.21.15.157:0/259469813 reap_dead start 2024-04-24T23:34:37.530 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.528+0000 7f70277fe640 1 --2- 172.21.15.157:0/259469813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70180039b0 0x7f7018003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:37.530 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.528+0000 7f70277fe640 1 --2- 172.21.15.157:0/259469813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7018004800 0x7f7018004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:37.530 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.528+0000 7f70277fe640 1 --2- 172.21.15.157:0/259469813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70180052e0 0x7f701800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:37.530 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.528+0000 7f70277fe640 1 -- 172.21.15.157:0/259469813 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f70280c5170 con 0x7f70180052e0 2024-04-24T23:34:37.530 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.528+0000 7f70277fe640 1 -- 172.21.15.157:0/259469813 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f701800c230 con 0x7f70180039b0 2024-04-24T23:34:37.530 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.528+0000 7f70277fe640 1 -- 172.21.15.157:0/259469813 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f701800c430 con 0x7f7018004800 2024-04-24T23:34:37.530 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.528+0000 7f702d35e640 1 --2- 172.21.15.157:0/259469813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70180052e0 0x7f701800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:37.531 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.528+0000 7f7027fff640 1 --2- 172.21.15.157:0/259469813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7018004800 0x7f7018004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:37.531 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.528+0000 7f702cb5d640 1 --2- 172.21.15.157:0/259469813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70180039b0 0x7f7018003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:37.531 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.529+0000 7f702d35e640 1 -- 172.21.15.157:0/259469813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7018004800 msgr2=0x7f7018004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:37.531 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.529+0000 7f702d35e640 1 --2- 172.21.15.157:0/259469813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7018004800 0x7f7018004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:37.531 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.529+0000 7f702d35e640 1 -- 172.21.15.157:0/259469813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70180039b0 msgr2=0x7f7018003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:37.531 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.529+0000 7f702d35e640 1 --2- 172.21.15.157:0/259469813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70180039b0 0x7f7018003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:37.531 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.529+0000 7f702d35e640 1 -- 172.21.15.157:0/259469813 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7018003520 con 0x7f70180052e0 2024-04-24T23:34:37.531 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.529+0000 7f702d35e640 1 --2- 172.21.15.157:0/259469813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70180052e0 0x7f701800b9c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f70280c3b90 tx=0x7f7018005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=690aadf2541ad1f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:37.531 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.530+0000 7f70277fe640 1 -- 172.21.15.157:0/259469813 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7020009700 con 0x7f70180052e0 2024-04-24T23:34:37.531 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.530+0000 7f70277fe640 1 -- 172.21.15.157:0/259469813 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f702002c4b0 con 0x7f70180052e0 2024-04-24T23:34:37.531 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.530+0000 7f70277fe640 1 -- 172.21.15.157:0/259469813 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702002c8a0 con 0x7f70180052e0 2024-04-24T23:34:37.535 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.530+0000 7f702edc1640 1 -- 172.21.15.157:0/259469813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70180052e0 msgr2=0x7f701800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:37.535 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.530+0000 7f702edc1640 1 --2- 172.21.15.157:0/259469813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70180052e0 0x7f701800b9c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f70280c3b90 tx=0x7f7018005830 comp rx=0 tx=0).stop 2024-04-24T23:34:37.535 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.530+0000 7f702edc1640 1 -- 172.21.15.157:0/259469813 shutdown_connections 2024-04-24T23:34:37.535 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.530+0000 7f702edc1640 1 --2- 172.21.15.157:0/259469813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70180052e0 0x7f701800b9c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:37.535 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.530+0000 7f702edc1640 1 --2- 172.21.15.157:0/259469813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7018004800 0x7f7018004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:37.535 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.530+0000 7f702edc1640 1 --2- 172.21.15.157:0/259469813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70180039b0 0x7f7018003db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:37.535 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.530+0000 7f702edc1640 1 -- 172.21.15.157:0/259469813 >> 172.21.15.157:0/259469813 conn(0x7f7028066ea0 msgr2=0x7f7028067280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:37.535 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.530+0000 7f702edc1640 1 -- 172.21.15.157:0/259469813 shutdown_connections 2024-04-24T23:34:37.535 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.530+0000 7f702edc1640 1 -- 172.21.15.157:0/259469813 wait complete. 2024-04-24T23:34:37.535 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.532+0000 7f702edc1640 1 Processor -- start 2024-04-24T23:34:37.535 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.532+0000 7f702edc1640 1 -- start start 2024-04-24T23:34:37.535 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.533+0000 7f702edc1640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70180039b0 0x7f7028153710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:37.536 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.533+0000 7f702edc1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7018004800 0x7f70280ae410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:37.536 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.533+0000 7f702edc1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70180052e0 0x7f70280abc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:37.536 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.533+0000 7f702edc1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7028152d80 con 0x7f7018004800 2024-04-24T23:34:37.536 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.533+0000 7f702edc1640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7028158c10 con 0x7f70180052e0 2024-04-24T23:34:37.536 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.533+0000 7f702edc1640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f70280c8fe0 con 0x7f70180039b0 2024-04-24T23:34:37.536 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.533+0000 7f7027fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7018004800 0x7f70280ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:37.536 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.533+0000 7f7027fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7018004800 0x7f70280ae410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58152/0 (socket says 172.21.15.157:58152) 2024-04-24T23:34:37.536 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.533+0000 7f7027fff640 1 -- 172.21.15.157:0/778490251 learned_addr learned my addr 172.21.15.157:0/778490251 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:34:37.536 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.533+0000 7f702d35e640 1 --2- 172.21.15.157:0/778490251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70180052e0 0x7f70280abc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:37.536 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.533+0000 7f702cb5d640 1 --2- 172.21.15.157:0/778490251 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70180039b0 0x7f7028153710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:37.536 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.533+0000 7f7027fff640 1 -- 172.21.15.157:0/778490251 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70180039b0 msgr2=0x7f7028153710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:37.536 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.533+0000 7f7027fff640 1 --2- 172.21.15.157:0/778490251 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70180039b0 0x7f7028153710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:37.536 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.533+0000 7f7027fff640 1 -- 172.21.15.157:0/778490251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70180052e0 msgr2=0x7f70280abc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:37.536 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.533+0000 7f7027fff640 1 --2- 172.21.15.157:0/778490251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70180052e0 0x7f70280abc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:37.536 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.533+0000 7f7027fff640 1 -- 172.21.15.157:0/778490251 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7028155300 con 0x7f7018004800 2024-04-24T23:34:37.537 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.534+0000 7f7027fff640 1 --2- 172.21.15.157:0/778490251 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7018004800 0x7f70280ae410 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7014016940 tx=0x7f7014016d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:37.537 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.534+0000 7f70257fa640 1 -- 172.21.15.157:0/778490251 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7014004320 con 0x7f7018004800 2024-04-24T23:34:37.537 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.534+0000 7f70257fa640 1 -- 172.21.15.157:0/778490251 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f701401b030 con 0x7f7018004800 2024-04-24T23:34:37.537 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.534+0000 7f702edc1640 1 -- 172.21.15.157:0/778490251 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70281555e0 con 0x7f7018004800 2024-04-24T23:34:37.537 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.534+0000 7f70257fa640 1 -- 172.21.15.157:0/778490251 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70140148d0 con 0x7f7018004800 2024-04-24T23:34:37.537 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.534+0000 7f702edc1640 1 -- 172.21.15.157:0/778490251 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70281eaf10 con 0x7f7018004800 2024-04-24T23:34:37.537 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.534+0000 7f70257fa640 1 -- 172.21.15.157:0/778490251 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7014003610 con 0x7f7018004800 2024-04-24T23:34:37.537 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.534+0000 7f70257fa640 1 -- 172.21.15.157:0/778490251 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7014017cf0 con 0x7f7018004800 2024-04-24T23:34:37.537 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.535+0000 7f702edc1640 1 -- 172.21.15.157:0/778490251 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70280c30d0 con 0x7f7018004800 2024-04-24T23:34:37.537 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.537+0000 7f70257fa640 1 -- 172.21.15.157:0/778490251 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f70280c30d0 con 0x7f7018004800 2024-04-24T23:34:37.603 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.604+0000 7f702edc1640 1 -- 172.21.15.157:0/778490251 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f702806ff30 con 0x7f7018004800 2024-04-24T23:34:37.605 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.605+0000 7f70257fa640 1 -- 172.21.15.157:0/778490251 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f7014014e00 con 0x7f7018004800 2024-04-24T23:34:37.605 INFO:teuthology.orchestra.run.smithi157.stderr:adjusted tunables profile to default 2024-04-24T23:34:37.606 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.606+0000 7f702edc1640 1 -- 172.21.15.157:0/778490251 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7018004800 msgr2=0x7f70280ae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:37.606 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.606+0000 7f702edc1640 1 --2- 172.21.15.157:0/778490251 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7018004800 0x7f70280ae410 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7014016940 tx=0x7f7014016d40 comp rx=0 tx=0).stop 2024-04-24T23:34:37.606 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.606+0000 7f702edc1640 1 -- 172.21.15.157:0/778490251 shutdown_connections 2024-04-24T23:34:37.607 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.606+0000 7f702edc1640 1 --2- 172.21.15.157:0/778490251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70180052e0 0x7f70280abc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:37.607 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.606+0000 7f702edc1640 1 --2- 172.21.15.157:0/778490251 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7018004800 0x7f70280ae410 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:37.607 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.606+0000 7f702edc1640 1 --2- 172.21.15.157:0/778490251 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70180039b0 0x7f7028153710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:37.607 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.607+0000 7f702edc1640 1 -- 172.21.15.157:0/778490251 >> 172.21.15.157:0/778490251 conn(0x7f7028066ea0 msgr2=0x7f70280c6860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:37.607 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.607+0000 7f702edc1640 1 -- 172.21.15.157:0/778490251 shutdown_connections 2024-04-24T23:34:37.607 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:37.607+0000 7f702edc1640 1 -- 172.21.15.157:0/778490251 wait complete. 2024-04-24T23:34:37.617 INFO:tasks.ceph:check_enable_crimson: False 2024-04-24T23:34:37.617 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-24T23:34:37.618 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-24T23:34:37.618 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-24T23:34:37.642 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-24T23:34:37.643 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-24T23:34:37.654 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-24T23:34:37.654+0000 7f590581a040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T23:34:37.706 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-24T23:34:37.707 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-24T23:34:37.769 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-24T23:34:37.769 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-24T23:34:37.806 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-24T23:34:37.806+0000 7f590581a040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:34:37.832 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-24T23:34:37.832 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-24T23:34:37.857 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-24T23:34:37.857 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-24T23:34:37.862 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-24T23:34:37.862+0000 7f590581a040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T23:34:37.884 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-24T23:34:37.884 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-24T23:34:37.949 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-24T23:34:37.949 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-24T23:34:38.013 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new 95d14ac8-fccf-4261-be8e-edafa746f629 0 2024-04-24T23:34:38.114 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-24T23:34:38.114+0000 7f590581a040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T23:34:38.151 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.151+0000 7f9a2faa3640 1 Processor -- start 2024-04-24T23:34:38.151 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.151+0000 7f9a2faa3640 1 -- start start 2024-04-24T23:34:38.153 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.152+0000 7f9a2faa3640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a28150a40 0x7f9a280a7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:38.153 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.152+0000 7f9a2faa3640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9a280a81b0 0x7f9a280c7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:38.153 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.152+0000 7f9a2faa3640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a28150e20 con 0x7f9a280c8210 2024-04-24T23:34:38.153 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.152+0000 7f9a2faa3640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f9a280c86a0 con 0x7f9a280a81b0 2024-04-24T23:34:38.153 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.152+0000 7f9a2faa3640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9a281568d0 con 0x7f9a28150a40 2024-04-24T23:34:38.154 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.152+0000 7f9a2d83f640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a28150a40 0x7f9a280a7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:38.154 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.152+0000 7f9a2d03e640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9a280a81b0 0x7f9a280c7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:38.154 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.152+0000 7f9a2d03e640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9a280a81b0 0x7f9a280c7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:49824/0 (socket says 172.21.15.169:49824) 2024-04-24T23:34:38.154 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.152+0000 7f9a2d83f640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a28150a40 0x7f9a280a7c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.169:51334/0 (socket says 172.21.15.169:51334) 2024-04-24T23:34:38.154 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.152+0000 7f9a2d03e640 1 -- 172.21.15.169:0/3384170246 learned_addr learned my addr 172.21.15.169:0/3384170246 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-24T23:34:38.154 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.152+0000 7f9a2d83f640 1 -- 172.21.15.169:0/3384170246 >> v2:172.21.15.169:3300/0 conn(0x7f9a280a81b0 msgr2=0x7f9a280c7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:38.154 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.152+0000 7f9a2d83f640 1 --2- 172.21.15.169:0/3384170246 >> v2:172.21.15.169:3300/0 conn(0x7f9a280a81b0 0x7f9a280c7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:38.154 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.152+0000 7f9a2d83f640 1 -- 172.21.15.169:0/3384170246 >> v1:172.21.15.169:6789/0 conn(0x7f9a280c8210 legacy=0x7f9a28153620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:38.154 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.152+0000 7f9a2d83f640 1 -- 172.21.15.169:0/3384170246 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a28156b70 con 0x7f9a28150a40 2024-04-24T23:34:38.154 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.152+0000 7f9a2d03e640 1 --2- 172.21.15.169:0/3384170246 >> v2:172.21.15.169:3300/0 conn(0x7f9a280a81b0 0x7f9a280c7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:34:38.154 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.153+0000 7f9a2d83f640 1 --2- 172.21.15.169:0/3384170246 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a28150a40 0x7f9a280a7c70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9a240097a0 tx=0x7f9a2402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f996d5fb551d5f5a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:38.154 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.153+0000 7f9a2c83d640 1 -- 172.21.15.169:0/3384170246 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a24037070 con 0x7f9a28150a40 2024-04-24T23:34:38.155 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.153+0000 7f9a2c83d640 1 -- 172.21.15.169:0/3384170246 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a2402ae50 con 0x7f9a28150a40 2024-04-24T23:34:38.155 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.153+0000 7f9a2c83d640 1 -- 172.21.15.169:0/3384170246 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a24032660 con 0x7f9a28150a40 2024-04-24T23:34:38.156 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.154+0000 7f9a2faa3640 1 -- 172.21.15.169:0/3384170246 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a28150a40 msgr2=0x7f9a280a7c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:38.156 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.154+0000 7f9a2faa3640 1 --2- 172.21.15.169:0/3384170246 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a28150a40 0x7f9a280a7c70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9a240097a0 tx=0x7f9a2402a100 comp rx=0 tx=0).stop 2024-04-24T23:34:38.156 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.154+0000 7f9a2faa3640 1 -- 172.21.15.169:0/3384170246 shutdown_connections 2024-04-24T23:34:38.156 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.154+0000 7f9a2faa3640 1 --2- 172.21.15.169:0/3384170246 >> v2:172.21.15.169:3300/0 conn(0x7f9a280a81b0 0x7f9a280c7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:38.156 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.154+0000 7f9a2faa3640 1 --2- 172.21.15.169:0/3384170246 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a28150a40 0x7f9a280a7c70 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:38.156 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.154+0000 7f9a2faa3640 1 -- 172.21.15.169:0/3384170246 >> 172.21.15.169:0/3384170246 conn(0x7f9a280aa7b0 msgr2=0x7f9a280aab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:38.156 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.154+0000 7f9a2faa3640 1 -- 172.21.15.169:0/3384170246 shutdown_connections 2024-04-24T23:34:38.156 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.154+0000 7f9a2faa3640 1 -- 172.21.15.169:0/3384170246 wait complete. 2024-04-24T23:34:38.157 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.155+0000 7f9a2faa3640 1 Processor -- start 2024-04-24T23:34:38.157 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.156+0000 7f9a2faa3640 1 -- start start 2024-04-24T23:34:38.159 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.157+0000 7f9a2faa3640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a280a81b0 0x7f9a280c4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:38.159 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.157+0000 7f9a2faa3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a280c8210 0x7f9a280c5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:38.159 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.157+0000 7f9a2faa3640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a28150a40 0x7f9a281557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:38.160 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.157+0000 7f9a2faa3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9a28157410 con 0x7f9a280c8210 2024-04-24T23:34:38.160 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.157+0000 7f9a2faa3640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9a28157130 con 0x7f9a28150a40 2024-04-24T23:34:38.160 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.157+0000 7f9a2faa3640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9a281576f0 con 0x7f9a280a81b0 2024-04-24T23:34:38.160 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.157+0000 7f9a2d83f640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a280a81b0 0x7f9a280c4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:38.160 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.157+0000 7f9a2d83f640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a280a81b0 0x7f9a280c4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.169:51344/0 (socket says 172.21.15.169:51344) 2024-04-24T23:34:38.160 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.157+0000 7f9a2d83f640 1 -- 172.21.15.169:0/2151756509 learned_addr learned my addr 172.21.15.169:0/2151756509 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-24T23:34:38.160 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.157+0000 7f9a2e040640 1 --2- 172.21.15.169:0/2151756509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a28150a40 0x7f9a281557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:38.160 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.157+0000 7f9a2d03e640 1 --2- 172.21.15.169:0/2151756509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a280c8210 0x7f9a280c5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:38.160 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.157+0000 7f9a2d83f640 1 -- 172.21.15.169:0/2151756509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a28150a40 msgr2=0x7f9a281557c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:38.160 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.157+0000 7f9a2d83f640 1 --2- 172.21.15.169:0/2151756509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a28150a40 0x7f9a281557c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:38.160 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.157+0000 7f9a2d83f640 1 -- 172.21.15.169:0/2151756509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a280c8210 msgr2=0x7f9a280c5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:38.160 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.157+0000 7f9a2d83f640 1 --2- 172.21.15.169:0/2151756509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a280c8210 0x7f9a280c5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:38.161 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.157+0000 7f9a2d83f640 1 -- 172.21.15.169:0/2151756509 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a280c3300 con 0x7f9a280a81b0 2024-04-24T23:34:38.161 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.157+0000 7f9a2e040640 1 --2- 172.21.15.169:0/2151756509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a28150a40 0x7f9a281557c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:34:38.161 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.157+0000 7f9a2d03e640 1 --2- 172.21.15.169:0/2151756509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a280c8210 0x7f9a280c5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:34:38.161 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.157+0000 7f9a2d83f640 1 --2- 172.21.15.169:0/2151756509 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a280a81b0 0x7f9a280c4330 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9a24000c00 tx=0x7f9a240046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:38.162 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.157+0000 7f9a1e7fc640 1 -- 172.21.15.169:0/2151756509 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a24042070 con 0x7f9a280a81b0 2024-04-24T23:34:38.162 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.157+0000 7f9a1e7fc640 1 -- 172.21.15.169:0/2151756509 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a2403e460 con 0x7f9a280a81b0 2024-04-24T23:34:38.162 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.157+0000 7f9a2faa3640 1 -- 172.21.15.169:0/2151756509 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a280c35e0 con 0x7f9a280a81b0 2024-04-24T23:34:38.162 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.158+0000 7f9a1e7fc640 1 -- 172.21.15.169:0/2151756509 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a24037040 con 0x7f9a280a81b0 2024-04-24T23:34:38.162 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.158+0000 7f9a2faa3640 1 -- 172.21.15.169:0/2151756509 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a28154f60 con 0x7f9a280a81b0 2024-04-24T23:34:38.162 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.158+0000 7f9a1e7fc640 1 -- 172.21.15.169:0/2151756509 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9a24045070 con 0x7f9a280a81b0 2024-04-24T23:34:38.162 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.158+0000 7f9a1e7fc640 1 -- 172.21.15.169:0/2151756509 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f9a24047b50 con 0x7f9a280a81b0 2024-04-24T23:34:38.162 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.159+0000 7f9a2faa3640 1 -- 172.21.15.169:0/2151756509 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a280681c0 con 0x7f9a280a81b0 2024-04-24T23:34:38.162 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.161+0000 7f9a1e7fc640 1 -- 172.21.15.169:0/2151756509 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f9a2403e850 con 0x7f9a280a81b0 2024-04-24T23:34:38.163 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-24T23:34:38.163+0000 7f590581a040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T23:34:38.232 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.233+0000 7f9a2faa3640 1 -- 172.21.15.169:0/2151756509 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd new", "uuid": "95d14ac8-fccf-4261-be8e-edafa746f629", "id": 0} v 0) -- 0x7f9a2814a4c0 con 0x7f9a280a81b0 2024-04-24T23:34:38.236 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.236+0000 7f9a1e7fc640 1 -- 172.21.15.169:0/2151756509 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "95d14ac8-fccf-4261-be8e-edafa746f629", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f9a2403ea30 con 0x7f9a280a81b0 2024-04-24T23:34:38.236 INFO:teuthology.orchestra.run.smithi169.stdout:0 2024-04-24T23:34:38.237 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.237+0000 7f9a2faa3640 1 -- 172.21.15.169:0/2151756509 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a280a81b0 msgr2=0x7f9a280c4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:38.237 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.237+0000 7f9a2faa3640 1 --2- 172.21.15.169:0/2151756509 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a280a81b0 0x7f9a280c4330 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9a24000c00 tx=0x7f9a240046c0 comp rx=0 tx=0).stop 2024-04-24T23:34:38.237 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.237+0000 7f9a2faa3640 1 -- 172.21.15.169:0/2151756509 shutdown_connections 2024-04-24T23:34:38.237 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.237+0000 7f9a2faa3640 1 --2- 172.21.15.169:0/2151756509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a28150a40 0x7f9a281557c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:38.237 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.237+0000 7f9a2faa3640 1 --2- 172.21.15.169:0/2151756509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a280c8210 0x7f9a280c5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:38.237 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.237+0000 7f9a2faa3640 1 --2- 172.21.15.169:0/2151756509 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a280a81b0 0x7f9a280c4330 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:38.237 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.237+0000 7f9a2faa3640 1 -- 172.21.15.169:0/2151756509 >> 172.21.15.169:0/2151756509 conn(0x7f9a280aa7b0 msgr2=0x7f9a280a8840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:38.238 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.237+0000 7f9a2faa3640 1 -- 172.21.15.169:0/2151756509 shutdown_connections 2024-04-24T23:34:38.238 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.237+0000 7f9a2faa3640 1 -- 172.21.15.169:0/2151756509 wait complete. 2024-04-24T23:34:38.247 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new 7be79da9-8eec-4990-9178-71d602e39d9b 1 2024-04-24T23:34:38.345 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.345+0000 7f4ab214f640 1 Processor -- start 2024-04-24T23:34:38.346 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.345+0000 7f4ab214f640 1 -- start start 2024-04-24T23:34:38.348 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.346+0000 7f4ab214f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4aac1562b0 0x7f4aac0c6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:38.348 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.346+0000 7f4ab214f640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4aac073500 0x7f4aac1591a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:38.348 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.346+0000 7f4ab214f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4aac0c1b50 con 0x7f4aac072b70 2024-04-24T23:34:38.348 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.346+0000 7f4ab214f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f4aac156690 con 0x7f4aac1562b0 2024-04-24T23:34:38.348 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.346+0000 7f4ab214f640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f4aac073050 con 0x7f4aac073500 2024-04-24T23:34:38.348 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.346+0000 7f4aaaffd640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4aac073500 0x7f4aac1591a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:38.348 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.346+0000 7f4aaaffd640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4aac073500 0x7f4aac1591a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:49836/0 (socket says 172.21.15.169:49836) 2024-04-24T23:34:38.348 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.346+0000 7f4aaaffd640 1 -- 172.21.15.169:0/273361182 learned_addr learned my addr 172.21.15.169:0/273361182 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-24T23:34:38.348 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.346+0000 7f4aab7fe640 1 --2- 172.21.15.169:0/273361182 >> v2:172.21.15.157:3300/0 conn(0x7f4aac1562b0 0x7f4aac0c6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:38.348 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.347+0000 7f4aaaffd640 1 -- 172.21.15.169:0/273361182 >> v2:172.21.15.157:3300/0 conn(0x7f4aac1562b0 msgr2=0x7f4aac0c6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:38.348 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.347+0000 7f4aaaffd640 1 --2- 172.21.15.169:0/273361182 >> v2:172.21.15.157:3300/0 conn(0x7f4aac1562b0 0x7f4aac0c6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:38.348 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.347+0000 7f4aaaffd640 1 -- 172.21.15.169:0/273361182 >> v1:172.21.15.157:6789/0 conn(0x7f4aac072b70 legacy=0x7f4aac06b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:38.348 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.347+0000 7f4aaaffd640 1 -- 172.21.15.169:0/273361182 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aac0c1e30 con 0x7f4aac073500 2024-04-24T23:34:38.349 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.347+0000 7f4aaaffd640 1 --2- 172.21.15.169:0/273361182 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4aac073500 0x7f4aac1591a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4aac06c1a0 tx=0x7f4a9c02a150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47e2c56fd2967110 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:38.349 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.347+0000 7f4aaa7fc640 1 -- 172.21.15.169:0/273361182 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c037070 con 0x7f4aac073500 2024-04-24T23:34:38.349 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.347+0000 7f4aaa7fc640 1 -- 172.21.15.169:0/273361182 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a9c032440 con 0x7f4aac073500 2024-04-24T23:34:38.349 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.347+0000 7f4aaa7fc640 1 -- 172.21.15.169:0/273361182 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c032810 con 0x7f4aac073500 2024-04-24T23:34:38.352 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.348+0000 7f4ab214f640 1 -- 172.21.15.169:0/273361182 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4aac073500 msgr2=0x7f4aac1591a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:38.352 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.348+0000 7f4ab214f640 1 --2- 172.21.15.169:0/273361182 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4aac073500 0x7f4aac1591a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4aac06c1a0 tx=0x7f4a9c02a150 comp rx=0 tx=0).stop 2024-04-24T23:34:38.352 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.348+0000 7f4ab214f640 1 -- 172.21.15.169:0/273361182 shutdown_connections 2024-04-24T23:34:38.352 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.348+0000 7f4ab214f640 1 --2- 172.21.15.169:0/273361182 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4aac073500 0x7f4aac1591a0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:38.352 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.348+0000 7f4ab214f640 1 --2- 172.21.15.169:0/273361182 >> v2:172.21.15.157:3300/0 conn(0x7f4aac1562b0 0x7f4aac0c6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:38.352 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.348+0000 7f4ab214f640 1 -- 172.21.15.169:0/273361182 >> 172.21.15.169:0/273361182 conn(0x7f4aac0c44f0 msgr2=0x7f4aac0c4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:38.352 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.348+0000 7f4ab214f640 1 -- 172.21.15.169:0/273361182 shutdown_connections 2024-04-24T23:34:38.352 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.348+0000 7f4ab214f640 1 -- 172.21.15.169:0/273361182 wait complete. 2024-04-24T23:34:38.352 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.349+0000 7f4ab214f640 1 Processor -- start 2024-04-24T23:34:38.352 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.349+0000 7f4ab214f640 1 -- start start 2024-04-24T23:34:38.352 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.350+0000 7f4ab214f640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4aac072b70 0x7f4aac14d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:38.352 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.350+0000 7f4ab214f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4aac073500 0x7f4aac1e1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:38.353 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.350+0000 7f4ab214f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4aac1562b0 0x7f4aac06dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:38.353 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.350+0000 7f4ab214f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4aac0c6fc0 con 0x7f4aac073500 2024-04-24T23:34:38.353 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.350+0000 7f4ab214f640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4aac0c6e40 con 0x7f4aac1562b0 2024-04-24T23:34:38.353 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.350+0000 7f4ab214f640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4aac0c7140 con 0x7f4aac072b70 2024-04-24T23:34:38.353 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.350+0000 7f4aab7fe640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4aac072b70 0x7f4aac14d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:38.353 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.350+0000 7f4aabfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4aac1562b0 0x7f4aac06dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:38.353 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.350+0000 7f4aab7fe640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4aac072b70 0x7f4aac14d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.169:51350/0 (socket says 172.21.15.169:51350) 2024-04-24T23:34:38.353 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.350+0000 7f4aab7fe640 1 -- 172.21.15.169:0/526390761 learned_addr learned my addr 172.21.15.169:0/526390761 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-24T23:34:38.353 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.350+0000 7f4aaaffd640 1 --2- 172.21.15.169:0/526390761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4aac073500 0x7f4aac1e1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:38.353 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.350+0000 7f4aab7fe640 1 -- 172.21.15.169:0/526390761 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4aac1562b0 msgr2=0x7f4aac06dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:38.353 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.350+0000 7f4aab7fe640 1 --2- 172.21.15.169:0/526390761 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4aac1562b0 0x7f4aac06dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:38.353 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.350+0000 7f4aab7fe640 1 -- 172.21.15.169:0/526390761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4aac073500 msgr2=0x7f4aac1e1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:38.353 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.350+0000 7f4aab7fe640 1 --2- 172.21.15.169:0/526390761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4aac073500 0x7f4aac1e1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:38.353 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.350+0000 7f4aab7fe640 1 -- 172.21.15.169:0/526390761 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aac1df4b0 con 0x7f4aac072b70 2024-04-24T23:34:38.354 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.350+0000 7f4aabfff640 1 --2- 172.21.15.169:0/526390761 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4aac1562b0 0x7f4aac06dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:34:38.354 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.351+0000 7f4aab7fe640 1 --2- 172.21.15.169:0/526390761 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4aac072b70 0x7f4aac14d100 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4aa000e9e0 tx=0x7f4aa000eea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:38.354 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.351+0000 7f4a93fff640 1 -- 172.21.15.169:0/526390761 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa0017070 con 0x7f4aac072b70 2024-04-24T23:34:38.354 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.351+0000 7f4a93fff640 1 -- 172.21.15.169:0/526390761 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4aa000c9d0 con 0x7f4aac072b70 2024-04-24T23:34:38.354 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.351+0000 7f4ab214f640 1 -- 172.21.15.169:0/526390761 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4aac14d640 con 0x7f4aac072b70 2024-04-24T23:34:38.354 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.351+0000 7f4a93fff640 1 -- 172.21.15.169:0/526390761 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa000cdc0 con 0x7f4aac072b70 2024-04-24T23:34:38.354 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.351+0000 7f4ab214f640 1 -- 172.21.15.169:0/526390761 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4aac06e8b0 con 0x7f4aac072b70 2024-04-24T23:34:38.354 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.351+0000 7f4a93fff640 1 -- 172.21.15.169:0/526390761 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4aa001a070 con 0x7f4aac072b70 2024-04-24T23:34:38.354 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.351+0000 7f4a93fff640 1 -- 172.21.15.169:0/526390761 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f4aa0013820 con 0x7f4aac072b70 2024-04-24T23:34:38.354 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.352+0000 7f4ab214f640 1 -- 172.21.15.169:0/526390761 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4aac0adb50 con 0x7f4aac072b70 2024-04-24T23:34:38.354 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.354+0000 7f4a93fff640 1 -- 172.21.15.169:0/526390761 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f4aac14d640 con 0x7f4aac072b70 2024-04-24T23:34:38.426 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.426+0000 7f4ab214f640 1 -- 172.21.15.169:0/526390761 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd new", "uuid": "7be79da9-8eec-4990-9178-71d602e39d9b", "id": 1} v 0) -- 0x7f4aac0c7450 con 0x7f4aac072b70 2024-04-24T23:34:38.429 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.430+0000 7f4a93fff640 1 -- 172.21.15.169:0/526390761 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7be79da9-8eec-4990-9178-71d602e39d9b", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f4aac1df4b0 con 0x7f4aac072b70 2024-04-24T23:34:38.430 INFO:teuthology.orchestra.run.smithi169.stdout:1 2024-04-24T23:34:38.431 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.430+0000 7f4ab214f640 1 -- 172.21.15.169:0/526390761 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4aac072b70 msgr2=0x7f4aac14d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:38.431 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.430+0000 7f4ab214f640 1 --2- 172.21.15.169:0/526390761 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4aac072b70 0x7f4aac14d100 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4aa000e9e0 tx=0x7f4aa000eea0 comp rx=0 tx=0).stop 2024-04-24T23:34:38.431 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.430+0000 7f4ab214f640 1 -- 172.21.15.169:0/526390761 shutdown_connections 2024-04-24T23:34:38.431 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.430+0000 7f4ab214f640 1 --2- 172.21.15.169:0/526390761 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4aac1562b0 0x7f4aac06dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:38.431 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.430+0000 7f4ab214f640 1 --2- 172.21.15.169:0/526390761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4aac073500 0x7f4aac1e1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:38.431 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.430+0000 7f4ab214f640 1 --2- 172.21.15.169:0/526390761 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4aac072b70 0x7f4aac14d100 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:38.431 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.430+0000 7f4ab214f640 1 -- 172.21.15.169:0/526390761 >> 172.21.15.169:0/526390761 conn(0x7f4aac0c44f0 msgr2=0x7f4aac153a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:38.431 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.430+0000 7f4ab214f640 1 -- 172.21.15.169:0/526390761 shutdown_connections 2024-04-24T23:34:38.431 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.431+0000 7f4ab214f640 1 -- 172.21.15.169:0/526390761 wait complete. 2024-04-24T23:34:38.440 INFO:tasks.ceph.mgr.y.smithi157.stderr:2024-04-24T23:34:38.440+0000 7fc502f8b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T23:34:38.441 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new 0127ed00-b7c3-466d-bfb0-01b0fd315f49 2 2024-04-24T23:34:38.540 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.540+0000 7fefbfa76640 1 Processor -- start 2024-04-24T23:34:38.540 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.540+0000 7fefbfa76640 1 -- start start 2024-04-24T23:34:38.542 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.541+0000 7fefbfa76640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fefb81562b0 0x7fefb80c6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:38.542 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.541+0000 7fefbfa76640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fefb8072b70 0x7fefb815fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:38.542 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.541+0000 7fefbfa76640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefb80c1b50 con 0x7fefb8073500 2024-04-24T23:34:38.542 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.541+0000 7fefbfa76640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fefb8156690 con 0x7fefb81562b0 2024-04-24T23:34:38.542 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.541+0000 7fefbfa76640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fefb8073050 con 0x7fefb8072b70 2024-04-24T23:34:38.543 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.541+0000 7fefbe013640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fefb8072b70 0x7fefb815fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:38.543 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.541+0000 7fefbe013640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fefb8072b70 0x7fefb815fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.169:51352/0 (socket says 172.21.15.169:51352) 2024-04-24T23:34:38.543 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.541+0000 7fefbe013640 1 -- 172.21.15.169:0/3788238528 learned_addr learned my addr 172.21.15.169:0/3788238528 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-24T23:34:38.543 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.541+0000 7fefbd812640 1 --2- 172.21.15.169:0/3788238528 >> v2:172.21.15.157:3300/0 conn(0x7fefb81562b0 0x7fefb80c6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:38.543 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.541+0000 7fefbc810640 1 -- 172.21.15.169:0/3788238528 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2642858732 0 0) 0x7fefb80c1b50 con 0x7fefb8073500 2024-04-24T23:34:38.543 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.541+0000 7fefbc810640 1 -- 172.21.15.169:0/3788238528 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fefa4003520 con 0x7fefb8073500 2024-04-24T23:34:38.543 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.541+0000 7fefbd812640 1 -- 172.21.15.169:0/3788238528 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fefb8072b70 msgr2=0x7fefb815fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:38.543 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.541+0000 7fefbd812640 1 --2- 172.21.15.169:0/3788238528 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fefb8072b70 0x7fefb815fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:38.543 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.541+0000 7fefbd812640 1 -- 172.21.15.169:0/3788238528 >> v1:172.21.15.157:6789/0 conn(0x7fefb8073500 legacy=0x7fefb81591a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:38.543 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.541+0000 7fefbd812640 1 -- 172.21.15.169:0/3788238528 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefb80c1e30 con 0x7fefb81562b0 2024-04-24T23:34:38.543 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.541+0000 7fefbe013640 1 --2- 172.21.15.169:0/3788238528 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fefb8072b70 0x7fefb815fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:34:38.543 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.542+0000 7fefbd812640 1 --2- 172.21.15.169:0/3788238528 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefb81562b0 0x7fefb80c6900 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fefb40097a0 tx=0x7fefb402a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75a17675a04c1f5e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:38.543 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.542+0000 7fefbc810640 1 -- 172.21.15.169:0/3788238528 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb4037070 con 0x7fefb81562b0 2024-04-24T23:34:38.543 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.542+0000 7fefbc810640 1 -- 172.21.15.169:0/3788238528 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefb402adb0 con 0x7fefb81562b0 2024-04-24T23:34:38.543 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.542+0000 7fefbc810640 1 -- 172.21.15.169:0/3788238528 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb4032660 con 0x7fefb81562b0 2024-04-24T23:34:38.546 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.543+0000 7fefbfa76640 1 -- 172.21.15.169:0/3788238528 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefb81562b0 msgr2=0x7fefb80c6900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:38.547 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.543+0000 7fefbfa76640 1 --2- 172.21.15.169:0/3788238528 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefb81562b0 0x7fefb80c6900 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fefb40097a0 tx=0x7fefb402a060 comp rx=0 tx=0).stop 2024-04-24T23:34:38.547 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.543+0000 7fefbfa76640 1 -- 172.21.15.169:0/3788238528 shutdown_connections 2024-04-24T23:34:38.547 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.543+0000 7fefbfa76640 1 --2- 172.21.15.169:0/3788238528 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fefb8072b70 0x7fefb815fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:38.547 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.543+0000 7fefbfa76640 1 --2- 172.21.15.169:0/3788238528 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefb81562b0 0x7fefb80c6900 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:38.547 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.543+0000 7fefbfa76640 1 -- 172.21.15.169:0/3788238528 >> 172.21.15.169:0/3788238528 conn(0x7fefb80c44f0 msgr2=0x7fefb80c4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:38.547 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.543+0000 7fefbfa76640 1 -- 172.21.15.169:0/3788238528 shutdown_connections 2024-04-24T23:34:38.547 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.543+0000 7fefbfa76640 1 -- 172.21.15.169:0/3788238528 wait complete. 2024-04-24T23:34:38.547 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.544+0000 7fefbfa76640 1 Processor -- start 2024-04-24T23:34:38.547 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.544+0000 7fefbfa76640 1 -- start start 2024-04-24T23:34:38.547 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.544+0000 7fefbfa76640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fefb8072b70 0x7fefb8157860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:38.547 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.545+0000 7fefbfa76640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefb8073500 0x7fefb81e1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:38.547 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.545+0000 7fefbfa76640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fefb81562b0 0x7fefb81e2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:38.547 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.545+0000 7fefbfa76640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fefb8159a30 con 0x7fefb8073500 2024-04-24T23:34:38.547 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.545+0000 7fefbfa76640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fefb81598b0 con 0x7fefb81562b0 2024-04-24T23:34:38.548 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.545+0000 7fefbfa76640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fefb80c6e40 con 0x7fefb8072b70 2024-04-24T23:34:38.548 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.545+0000 7fefbd812640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fefb8072b70 0x7fefb8157860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:38.548 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.545+0000 7fefbd812640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fefb8072b70 0x7fefb8157860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.169:51368/0 (socket says 172.21.15.169:51368) 2024-04-24T23:34:38.548 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.545+0000 7fefbe013640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fefb81562b0 0x7fefb81e2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:38.548 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.545+0000 7fefbd812640 1 -- 172.21.15.169:0/2602409055 learned_addr learned my addr 172.21.15.169:0/2602409055 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-24T23:34:38.548 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.545+0000 7fefbd011640 1 --2- 172.21.15.169:0/2602409055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefb8073500 0x7fefb81e1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:38.548 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.545+0000 7fefbd812640 1 -- 172.21.15.169:0/2602409055 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fefb81562b0 msgr2=0x7fefb81e2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:38.548 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.545+0000 7fefbd812640 1 --2- 172.21.15.169:0/2602409055 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fefb81562b0 0x7fefb81e2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:38.548 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.545+0000 7fefbd812640 1 -- 172.21.15.169:0/2602409055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefb8073500 msgr2=0x7fefb81e1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:38.548 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.545+0000 7fefbd812640 1 --2- 172.21.15.169:0/2602409055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefb8073500 0x7fefb81e1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:38.548 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.545+0000 7fefbd812640 1 -- 172.21.15.169:0/2602409055 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefb81ee580 con 0x7fefb8072b70 2024-04-24T23:34:38.548 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.545+0000 7fefbe013640 1 --2- 172.21.15.169:0/2602409055 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fefb81562b0 0x7fefb81e2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:34:38.548 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.545+0000 7fefbd011640 1 --2- 172.21.15.169:0/2602409055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefb8073500 0x7fefb81e1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:34:38.549 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.545+0000 7fefbd812640 1 --2- 172.21.15.169:0/2602409055 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fefb8072b70 0x7fefb8157860 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fefb40098c0 tx=0x7fefb40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:38.549 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.545+0000 7fefae7fc640 1 -- 172.21.15.169:0/2602409055 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb4041030 con 0x7fefb8072b70 2024-04-24T23:34:38.549 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.545+0000 7fefae7fc640 1 -- 172.21.15.169:0/2602409055 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefb402abb0 con 0x7fefb8072b70 2024-04-24T23:34:38.549 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.545+0000 7fefbfa76640 1 -- 172.21.15.169:0/2602409055 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefb81ee860 con 0x7fefb8072b70 2024-04-24T23:34:38.549 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.545+0000 7fefae7fc640 1 -- 172.21.15.169:0/2602409055 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb4037040 con 0x7fefb8072b70 2024-04-24T23:34:38.549 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.545+0000 7fefbfa76640 1 -- 172.21.15.169:0/2602409055 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fefb81eee20 con 0x7fefb8072b70 2024-04-24T23:34:38.549 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.546+0000 7fefae7fc640 1 -- 172.21.15.169:0/2602409055 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fefb4044070 con 0x7fefb8072b70 2024-04-24T23:34:38.549 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.546+0000 7fefae7fc640 1 -- 172.21.15.169:0/2602409055 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7fefb4046a10 con 0x7fefb8072b70 2024-04-24T23:34:38.549 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.546+0000 7fefbfa76640 1 -- 172.21.15.169:0/2602409055 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefb80688d0 con 0x7fefb8072b70 2024-04-24T23:34:38.549 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.548+0000 7fefae7fc640 1 -- 172.21.15.169:0/2602409055 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fefb402ad60 con 0x7fefb8072b70 2024-04-24T23:34:38.583 INFO:tasks.ceph.mgr.y.smithi157.stderr:2024-04-24T23:34:38.583+0000 7fc502f8b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:34:38.620 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.620+0000 7fefbfa76640 1 -- 172.21.15.169:0/2602409055 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd new", "uuid": "0127ed00-b7c3-466d-bfb0-01b0fd315f49", "id": 2} v 0) -- 0x7fefb80c7450 con 0x7fefb8072b70 2024-04-24T23:34:38.623 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.624+0000 7fefae7fc640 1 -- 172.21.15.169:0/2602409055 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0127ed00-b7c3-466d-bfb0-01b0fd315f49", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7fefb4046d80 con 0x7fefb8072b70 2024-04-24T23:34:38.624 INFO:teuthology.orchestra.run.smithi169.stdout:2 2024-04-24T23:34:38.625 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.624+0000 7fefbfa76640 1 -- 172.21.15.169:0/2602409055 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fefb8072b70 msgr2=0x7fefb8157860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:38.625 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.624+0000 7fefbfa76640 1 --2- 172.21.15.169:0/2602409055 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fefb8072b70 0x7fefb8157860 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fefb40098c0 tx=0x7fefb40046c0 comp rx=0 tx=0).stop 2024-04-24T23:34:38.625 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.624+0000 7fefbfa76640 1 -- 172.21.15.169:0/2602409055 shutdown_connections 2024-04-24T23:34:38.625 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.624+0000 7fefbfa76640 1 --2- 172.21.15.169:0/2602409055 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fefb81562b0 0x7fefb81e2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:38.625 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.624+0000 7fefbfa76640 1 --2- 172.21.15.169:0/2602409055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefb8073500 0x7fefb81e1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:38.625 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.624+0000 7fefbfa76640 1 --2- 172.21.15.169:0/2602409055 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fefb8072b70 0x7fefb8157860 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:38.625 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.624+0000 7fefbfa76640 1 -- 172.21.15.169:0/2602409055 >> 172.21.15.169:0/2602409055 conn(0x7fefb80c44f0 msgr2=0x7fefb8074130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:38.625 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.625+0000 7fefbfa76640 1 -- 172.21.15.169:0/2602409055 shutdown_connections 2024-04-24T23:34:38.625 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.625+0000 7fefbfa76640 1 -- 172.21.15.169:0/2602409055 wait complete. 2024-04-24T23:34:38.635 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new f2331acc-aa37-433e-868e-3561b6039562 3 2024-04-24T23:34:38.635 INFO:tasks.ceph.mgr.y.smithi157.stderr:2024-04-24T23:34:38.636+0000 7fc502f8b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T23:34:38.743 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.743+0000 7f7978064640 1 Processor -- start 2024-04-24T23:34:38.743 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.743+0000 7f7978064640 1 -- start start 2024-04-24T23:34:38.745 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.744+0000 7f7978064640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f79701566e0 0x7f797006b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:38.745 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.744+0000 7f7978064640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79700c41d0 con 0x7f797006b160 2024-04-24T23:34:38.745 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.744+0000 7f7978064640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79700c6b10 con 0x7f79700c6220 2024-04-24T23:34:38.745 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.744+0000 7f7978064640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f79700c6df0 con 0x7f79701566e0 2024-04-24T23:34:38.745 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.744+0000 7f79755ff640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f79701566e0 0x7f797006b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:38.745 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.744+0000 7f79755ff640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f79701566e0 0x7f797006b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:49852/0 (socket says 172.21.15.169:49852) 2024-04-24T23:34:38.745 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.744+0000 7f79755ff640 1 -- 172.21.15.169:0/392654742 learned_addr learned my addr 172.21.15.169:0/392654742 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-24T23:34:38.745 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.744+0000 7f79755ff640 1 -- 172.21.15.169:0/392654742 >> v1:172.21.15.169:6789/0 conn(0x7f79700c6220 legacy=0x7f797006f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:38.745 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.744+0000 7f79755ff640 1 -- 172.21.15.169:0/392654742 >> v1:172.21.15.157:6789/0 conn(0x7f797006b160 legacy=0x7f7970153820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:38.745 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.744+0000 7f79755ff640 1 -- 172.21.15.169:0/392654742 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79700c70d0 con 0x7f79701566e0 2024-04-24T23:34:38.747 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.745+0000 7f79755ff640 1 --2- 172.21.15.169:0/392654742 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79701566e0 0x7f797006b540 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7960009870 tx=0x7f796002a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8e86e72a3b8353a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:38.747 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.745+0000 7f7974dfe640 1 -- 172.21.15.169:0/392654742 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7960037070 con 0x7f79701566e0 2024-04-24T23:34:38.747 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.745+0000 7f7974dfe640 1 -- 172.21.15.169:0/392654742 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7960032440 con 0x7f79701566e0 2024-04-24T23:34:38.747 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.745+0000 7f7974dfe640 1 -- 172.21.15.169:0/392654742 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7960032810 con 0x7f79701566e0 2024-04-24T23:34:38.747 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.746+0000 7f7978064640 1 -- 172.21.15.169:0/392654742 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79701566e0 msgr2=0x7f797006b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:38.747 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.746+0000 7f7978064640 1 --2- 172.21.15.169:0/392654742 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79701566e0 0x7f797006b540 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7960009870 tx=0x7f796002a270 comp rx=0 tx=0).stop 2024-04-24T23:34:38.747 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.746+0000 7f7978064640 1 -- 172.21.15.169:0/392654742 shutdown_connections 2024-04-24T23:34:38.747 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.746+0000 7f7978064640 1 --2- 172.21.15.169:0/392654742 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79701566e0 0x7f797006b540 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:38.747 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.746+0000 7f7978064640 1 -- 172.21.15.169:0/392654742 >> 172.21.15.169:0/392654742 conn(0x7f79700c43f0 msgr2=0x7f79700c47d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:38.747 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.746+0000 7f7978064640 1 -- 172.21.15.169:0/392654742 shutdown_connections 2024-04-24T23:34:38.747 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.746+0000 7f7978064640 1 -- 172.21.15.169:0/392654742 wait complete. 2024-04-24T23:34:38.748 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.748+0000 7f7978064640 1 Processor -- start 2024-04-24T23:34:38.748 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.748+0000 7f7978064640 1 -- start start 2024-04-24T23:34:38.751 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.749+0000 7f7978064640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f797006b160 0x7f79701ec4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:38.751 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.749+0000 7f7978064640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79700c6220 0x7f79701eca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:38.751 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.749+0000 7f7978064640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79701566e0 0x7f797014cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:38.751 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.749+0000 7f7978064640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7970153f30 con 0x7f79700c6220 2024-04-24T23:34:38.751 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.749+0000 7f7978064640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f79700c3520 con 0x7f79701566e0 2024-04-24T23:34:38.751 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.749+0000 7f7978064640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f79701540b0 con 0x7f797006b160 2024-04-24T23:34:38.751 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.749+0000 7f7976601640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79701566e0 0x7f797014cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:38.751 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.749+0000 7f7975e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f797006b160 0x7f79701ec4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:38.751 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.749+0000 7f7976601640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79701566e0 0x7f797014cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:49862/0 (socket says 172.21.15.169:49862) 2024-04-24T23:34:38.751 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.749+0000 7f7976601640 1 -- 172.21.15.169:0/2051973529 learned_addr learned my addr 172.21.15.169:0/2051973529 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-24T23:34:38.751 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.749+0000 7f79755ff640 1 --2- 172.21.15.169:0/2051973529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79700c6220 0x7f79701eca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:38.751 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.749+0000 7f7976601640 1 -- 172.21.15.169:0/2051973529 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f797006b160 msgr2=0x7f79701ec4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:38.752 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.749+0000 7f7976601640 1 --2- 172.21.15.169:0/2051973529 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f797006b160 0x7f79701ec4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:38.752 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.749+0000 7f7976601640 1 -- 172.21.15.169:0/2051973529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79700c6220 msgr2=0x7f79701eca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:38.752 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.749+0000 7f7976601640 1 --2- 172.21.15.169:0/2051973529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79700c6220 0x7f79701eca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:38.752 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.749+0000 7f7976601640 1 -- 172.21.15.169:0/2051973529 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79701ecf40 con 0x7f79701566e0 2024-04-24T23:34:38.752 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.749+0000 7f7975e00640 1 --2- 172.21.15.169:0/2051973529 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f797006b160 0x7f79701ec4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:34:38.752 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.750+0000 7f7976601640 1 --2- 172.21.15.169:0/2051973529 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79701566e0 0x7f797014cb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f796800c7a0 tx=0x7f796800cc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:38.752 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.750+0000 7f79667fc640 1 -- 172.21.15.169:0/2051973529 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7968003b70 con 0x7f79701566e0 2024-04-24T23:34:38.752 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.750+0000 7f79667fc640 1 -- 172.21.15.169:0/2051973529 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7968003d30 con 0x7f79701566e0 2024-04-24T23:34:38.752 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.750+0000 7f7978064640 1 -- 172.21.15.169:0/2051973529 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79701dc8d0 con 0x7f79701566e0 2024-04-24T23:34:38.752 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.750+0000 7f79667fc640 1 -- 172.21.15.169:0/2051973529 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796800aa90 con 0x7f79701566e0 2024-04-24T23:34:38.752 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.750+0000 7f7978064640 1 -- 172.21.15.169:0/2051973529 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f797014d940 con 0x7f79701566e0 2024-04-24T23:34:38.752 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.750+0000 7f79667fc640 1 -- 172.21.15.169:0/2051973529 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f796800ac30 con 0x7f79701566e0 2024-04-24T23:34:38.752 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.750+0000 7f79667fc640 1 -- 172.21.15.169:0/2051973529 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f7968012a90 con 0x7f79701566e0 2024-04-24T23:34:38.753 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.751+0000 7f7978064640 1 -- 172.21.15.169:0/2051973529 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79700681c0 con 0x7f79701566e0 2024-04-24T23:34:38.753 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.753+0000 7f79667fc640 1 -- 172.21.15.169:0/2051973529 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f7968012ce0 con 0x7f79701566e0 2024-04-24T23:34:38.843 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.843+0000 7f7978064640 1 -- 172.21.15.169:0/2051973529 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f2331acc-aa37-433e-868e-3561b6039562", "id": 3} v 0) -- 0x7f797006bef0 con 0x7f79701566e0 2024-04-24T23:34:38.846 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.846+0000 7f79667fc640 1 -- 172.21.15.169:0/2051973529 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f2331acc-aa37-433e-868e-3561b6039562", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f7968012600 con 0x7f79701566e0 2024-04-24T23:34:38.846 INFO:teuthology.orchestra.run.smithi169.stdout:3 2024-04-24T23:34:38.847 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.847+0000 7f7978064640 1 -- 172.21.15.169:0/2051973529 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79701566e0 msgr2=0x7f797014cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:38.847 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.847+0000 7f7978064640 1 --2- 172.21.15.169:0/2051973529 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79701566e0 0x7f797014cb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f796800c7a0 tx=0x7f796800cc60 comp rx=0 tx=0).stop 2024-04-24T23:34:38.847 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.847+0000 7f7978064640 1 -- 172.21.15.169:0/2051973529 shutdown_connections 2024-04-24T23:34:38.847 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.847+0000 7f7978064640 1 --2- 172.21.15.169:0/2051973529 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79701566e0 0x7f797014cb70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:38.847 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.847+0000 7f7978064640 1 --2- 172.21.15.169:0/2051973529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79700c6220 0x7f79701eca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:38.847 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.847+0000 7f7978064640 1 --2- 172.21.15.169:0/2051973529 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f797006b160 0x7f79701ec4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:38.847 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.847+0000 7f7978064640 1 -- 172.21.15.169:0/2051973529 >> 172.21.15.169:0/2051973529 conn(0x7f79700c43f0 msgr2=0x7f79700c36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:38.847 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.847+0000 7f7978064640 1 -- 172.21.15.169:0/2051973529 shutdown_connections 2024-04-24T23:34:38.847 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.847+0000 7f7978064640 1 -- 172.21.15.169:0/2051973529 wait complete. 2024-04-24T23:34:38.858 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new 1998854b-7398-43f5-ab7d-79fa0debcf97 4 2024-04-24T23:34:38.864 INFO:tasks.ceph.mgr.y.smithi157.stderr:2024-04-24T23:34:38.865+0000 7fc502f8b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T23:34:38.909 INFO:tasks.ceph.mgr.y.smithi157.stderr:2024-04-24T23:34:38.909+0000 7fc502f8b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T23:34:38.941 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-24T23:34:38.941+0000 7f590581a040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T23:34:38.955 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.955+0000 7f8c47b14640 1 Processor -- start 2024-04-24T23:34:38.955 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.955+0000 7f8c47b14640 1 -- start start 2024-04-24T23:34:38.958 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.957+0000 7f8c47b14640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8c400c37c0 0x7f8c400a8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:38.958 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.957+0000 7f8c47b14640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c40070b90 con 0x7f8c40070720 2024-04-24T23:34:38.958 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.957+0000 7f8c47b14640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c4006f100 con 0x7f8c400a7c70 2024-04-24T23:34:38.958 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.957+0000 7f8c47b14640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f8c40070e20 con 0x7f8c400c37c0 2024-04-24T23:34:38.958 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.957+0000 7f8c458b0640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f8c400a7c70 0x7f8c40157160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.169:40916/0 (socket says 172.21.15.169:40916) 2024-04-24T23:34:38.958 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.957+0000 7f8c458b0640 1 -- 172.21.15.169:0/1437034665 learned_addr learned my addr 172.21.15.169:0/1437034665 (peer_addr_for_me v1:172.21.15.169:0/0) 2024-04-24T23:34:38.959 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.957+0000 7f8c450af640 1 --2- 172.21.15.169:0/1437034665 >> v2:172.21.15.157:3300/0 conn(0x7f8c400c37c0 0x7f8c400a8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:38.959 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.957+0000 7f8c448ae640 1 -- 172.21.15.169:0/1437034665 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 785624755 0 0) 0x7f8c4006f100 con 0x7f8c400a7c70 2024-04-24T23:34:38.959 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.957+0000 7f8c448ae640 1 -- 172.21.15.169:0/1437034665 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8c2c003520 con 0x7f8c400a7c70 2024-04-24T23:34:38.959 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.957+0000 7f8c448ae640 1 -- 172.21.15.169:0/1437034665 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2252656283 0 0) 0x7f8c40070b90 con 0x7f8c40070720 2024-04-24T23:34:38.959 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.957+0000 7f8c448ae640 1 -- 172.21.15.169:0/1437034665 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8c4006f100 con 0x7f8c40070720 2024-04-24T23:34:38.959 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.957+0000 7f8c450af640 1 -- 172.21.15.169:0/1437034665 >> v1:172.21.15.169:6789/0 conn(0x7f8c400a7c70 legacy=0x7f8c40157160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:38.959 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.957+0000 7f8c450af640 1 -- 172.21.15.169:0/1437034665 >> v1:172.21.15.157:6789/0 conn(0x7f8c40070720 legacy=0x7f8c401531d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:38.959 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.957+0000 7f8c450af640 1 -- 172.21.15.169:0/1437034665 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c40071100 con 0x7f8c400c37c0 2024-04-24T23:34:38.959 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.958+0000 7f8c450af640 1 --2- 172.21.15.169:0/1437034665 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c400c37c0 0x7f8c400a8050 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c0097a0 tx=0x7f8c3c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b51cfd312ff44bc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:38.959 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.958+0000 7f8c448ae640 1 -- 172.21.15.169:0/1437034665 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c037070 con 0x7f8c400c37c0 2024-04-24T23:34:38.959 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.958+0000 7f8c448ae640 1 -- 172.21.15.169:0/1437034665 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c3c032440 con 0x7f8c400c37c0 2024-04-24T23:34:38.959 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.958+0000 7f8c448ae640 1 -- 172.21.15.169:0/1437034665 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c032810 con 0x7f8c400c37c0 2024-04-24T23:34:38.960 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.959+0000 7f8c47b14640 1 -- 172.21.15.169:0/1437034665 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c400c37c0 msgr2=0x7f8c400a8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:38.960 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.959+0000 7f8c47b14640 1 --2- 172.21.15.169:0/1437034665 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c400c37c0 0x7f8c400a8050 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c0097a0 tx=0x7f8c3c02a230 comp rx=0 tx=0).stop 2024-04-24T23:34:38.960 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.959+0000 7f8c47b14640 1 -- 172.21.15.169:0/1437034665 shutdown_connections 2024-04-24T23:34:38.960 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.959+0000 7f8c47b14640 1 --2- 172.21.15.169:0/1437034665 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c400c37c0 0x7f8c400a8050 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:38.960 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.959+0000 7f8c47b14640 1 -- 172.21.15.169:0/1437034665 >> 172.21.15.169:0/1437034665 conn(0x7f8c400c8ba0 msgr2=0x7f8c400c8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:38.961 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.959+0000 7f8c47b14640 1 -- 172.21.15.169:0/1437034665 shutdown_connections 2024-04-24T23:34:38.961 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.959+0000 7f8c47b14640 1 -- 172.21.15.169:0/1437034665 wait complete. 2024-04-24T23:34:38.961 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.961+0000 7f8c47b14640 1 Processor -- start 2024-04-24T23:34:38.961 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.961+0000 7f8c47b14640 1 -- start start 2024-04-24T23:34:38.964 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.962+0000 7f8c47b14640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c40070720 0x7f8c400a8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:38.964 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.962+0000 7f8c47b14640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c400a7c70 0x7f8c400aa410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:38.964 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.962+0000 7f8c47b14640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c400c37c0 0x7f8c400aa950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:38.964 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.962+0000 7f8c47b14640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8c400c5d20 con 0x7f8c400c37c0 2024-04-24T23:34:38.964 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.962+0000 7f8c47b14640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8c40156c40 con 0x7f8c40070720 2024-04-24T23:34:38.964 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.962+0000 7f8c47b14640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8c400c5ea0 con 0x7f8c400a7c70 2024-04-24T23:34:38.964 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.962+0000 7f8c458b0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c40070720 0x7f8c400a8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:38.964 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.962+0000 7f8c450af640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c400a7c70 0x7f8c400aa410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:38.964 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.962+0000 7f8c450af640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c400a7c70 0x7f8c400aa410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.169:51390/0 (socket says 172.21.15.169:51390) 2024-04-24T23:34:38.964 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.962+0000 7f8c450af640 1 -- 172.21.15.169:0/3831855713 learned_addr learned my addr 172.21.15.169:0/3831855713 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-24T23:34:38.964 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.962+0000 7f8c460b1640 1 --2- 172.21.15.169:0/3831855713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c400c37c0 0x7f8c400aa950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:38.964 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.962+0000 7f8c458b0640 1 -- 172.21.15.169:0/3831855713 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c400a7c70 msgr2=0x7f8c400aa410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:38.964 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.962+0000 7f8c458b0640 1 --2- 172.21.15.169:0/3831855713 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c400a7c70 0x7f8c400aa410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:38.964 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.962+0000 7f8c458b0640 1 -- 172.21.15.169:0/3831855713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c400c37c0 msgr2=0x7f8c400aa950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:38.964 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.962+0000 7f8c458b0640 1 --2- 172.21.15.169:0/3831855713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c400c37c0 0x7f8c400aa950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:38.965 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.962+0000 7f8c458b0640 1 -- 172.21.15.169:0/3831855713 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c400a9cc0 con 0x7f8c40070720 2024-04-24T23:34:38.965 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.962+0000 7f8c450af640 1 --2- 172.21.15.169:0/3831855713 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c400a7c70 0x7f8c400aa410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:34:38.965 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.962+0000 7f8c460b1640 1 --2- 172.21.15.169:0/3831855713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c400c37c0 0x7f8c400aa950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:34:38.965 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.963+0000 7f8c458b0640 1 --2- 172.21.15.169:0/3831855713 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c40070720 0x7f8c400a8fb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8c3000c3e0 tx=0x7f8c3000c8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:38.965 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.963+0000 7f8c367fc640 1 -- 172.21.15.169:0/3831855713 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c30017070 con 0x7f8c40070720 2024-04-24T23:34:38.965 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.963+0000 7f8c367fc640 1 -- 172.21.15.169:0/3831855713 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c30005ce0 con 0x7f8c40070720 2024-04-24T23:34:38.965 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.963+0000 7f8c47b14640 1 -- 172.21.15.169:0/3831855713 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c400a9fa0 con 0x7f8c40070720 2024-04-24T23:34:38.965 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.963+0000 7f8c367fc640 1 -- 172.21.15.169:0/3831855713 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c300072d0 con 0x7f8c40070720 2024-04-24T23:34:38.965 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.963+0000 7f8c47b14640 1 -- 172.21.15.169:0/3831855713 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c40153c80 con 0x7f8c40070720 2024-04-24T23:34:38.965 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.963+0000 7f8c367fc640 1 -- 172.21.15.169:0/3831855713 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8c3001a070 con 0x7f8c40070720 2024-04-24T23:34:38.965 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.963+0000 7f8c367fc640 1 -- 172.21.15.169:0/3831855713 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f8c3001cc60 con 0x7f8c40070720 2024-04-24T23:34:38.965 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.964+0000 7f8c47b14640 1 -- 172.21.15.169:0/3831855713 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c400688d0 con 0x7f8c40070720 2024-04-24T23:34:38.966 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:38.966+0000 7f8c367fc640 1 -- 172.21.15.169:0/3831855713 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f8c30007470 con 0x7f8c40070720 2024-04-24T23:34:39.035 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-24T23:34:39.035+0000 7f590581a040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T23:34:39.039 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.039+0000 7f8c47b14640 1 -- 172.21.15.169:0/3831855713 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd new", "uuid": "1998854b-7398-43f5-ab7d-79fa0debcf97", "id": 4} v 0) -- 0x7f8c400c8770 con 0x7f8c40070720 2024-04-24T23:34:39.042 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.041+0000 7f8c367fc640 1 -- 172.21.15.169:0/3831855713 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1998854b-7398-43f5-ab7d-79fa0debcf97", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f8c30007650 con 0x7f8c40070720 2024-04-24T23:34:39.042 INFO:teuthology.orchestra.run.smithi169.stdout:4 2024-04-24T23:34:39.043 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.042+0000 7f8c47b14640 1 -- 172.21.15.169:0/3831855713 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c40070720 msgr2=0x7f8c400a8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:39.043 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.042+0000 7f8c47b14640 1 --2- 172.21.15.169:0/3831855713 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c40070720 0x7f8c400a8fb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8c3000c3e0 tx=0x7f8c3000c8a0 comp rx=0 tx=0).stop 2024-04-24T23:34:39.043 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.042+0000 7f8c47b14640 1 -- 172.21.15.169:0/3831855713 shutdown_connections 2024-04-24T23:34:39.043 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.042+0000 7f8c47b14640 1 --2- 172.21.15.169:0/3831855713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c400c37c0 0x7f8c400aa950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:39.043 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.042+0000 7f8c47b14640 1 --2- 172.21.15.169:0/3831855713 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c400a7c70 0x7f8c400aa410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:39.043 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.042+0000 7f8c47b14640 1 --2- 172.21.15.169:0/3831855713 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c40070720 0x7f8c400a8fb0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:39.043 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.042+0000 7f8c47b14640 1 -- 172.21.15.169:0/3831855713 >> 172.21.15.169:0/3831855713 conn(0x7f8c400c8ba0 msgr2=0x7f8c400c8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:39.044 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.043+0000 7f8c47b14640 1 -- 172.21.15.169:0/3831855713 shutdown_connections 2024-04-24T23:34:39.044 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.043+0000 7f8c47b14640 1 -- 172.21.15.169:0/3831855713 wait complete. 2024-04-24T23:34:39.053 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new c37153ff-9b2b-48b9-a684-2c82b6f39f15 5 2024-04-24T23:34:39.082 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-24T23:34:39.082+0000 7f590581a040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T23:34:39.149 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.149+0000 7f9275091640 1 Processor -- start 2024-04-24T23:34:39.150 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.149+0000 7f9275091640 1 -- start start 2024-04-24T23:34:39.151 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.150+0000 7f9275091640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9270156340 0x7f92700c8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:39.151 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.150+0000 7f9275091640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f92700c7330 0x7f92700c3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:39.151 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.150+0000 7f9275091640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92700c8740 con 0x7f92700c3970 2024-04-24T23:34:39.151 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.150+0000 7f9275091640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f9270156720 con 0x7f9270156340 2024-04-24T23:34:39.151 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.150+0000 7f9275091640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f92700c3ee0 con 0x7f92700c7330 2024-04-24T23:34:39.151 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.150+0000 7f926ed9d640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9270156340 0x7f92700c8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:39.151 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.150+0000 7f926e59c640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f92700c7330 0x7f92700c3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:39.151 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.150+0000 7f926ed9d640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9270156340 0x7f92700c8200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:49892/0 (socket says 172.21.15.169:49892) 2024-04-24T23:34:39.152 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.150+0000 7f926ed9d640 1 -- 172.21.15.169:0/2091057088 learned_addr learned my addr 172.21.15.169:0/2091057088 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-24T23:34:39.152 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.150+0000 7f926f59e640 1 --1- 172.21.15.169:0/2091057088 >> v1:172.21.15.169:6789/0 conn(0x7f92700c3970 0x7f9270158ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.169:40930/0 (socket says 172.21.15.169:40930) 2024-04-24T23:34:39.152 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.151+0000 7f926e59c640 1 -- 172.21.15.169:0/2091057088 >> v2:172.21.15.169:3300/0 conn(0x7f9270156340 msgr2=0x7f92700c8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:39.152 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.151+0000 7f926e59c640 1 --2- 172.21.15.169:0/2091057088 >> v2:172.21.15.169:3300/0 conn(0x7f9270156340 0x7f92700c8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:39.152 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.151+0000 7f926e59c640 1 -- 172.21.15.169:0/2091057088 >> v1:172.21.15.169:6789/0 conn(0x7f92700c3970 legacy=0x7f9270158ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:39.152 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.151+0000 7f926e59c640 1 -- 172.21.15.169:0/2091057088 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92700c41c0 con 0x7f92700c7330 2024-04-24T23:34:39.152 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.151+0000 7f926ed9d640 1 --2- 172.21.15.169:0/2091057088 >> v2:172.21.15.169:3300/0 conn(0x7f9270156340 0x7f92700c8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:34:39.153 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.151+0000 7f926e59c640 1 --2- 172.21.15.169:0/2091057088 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f92700c7330 0x7f92700c3430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f92640097a0 tx=0x7f926402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5766847039693a45 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:39.153 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.151+0000 7f926dd9b640 1 -- 172.21.15.169:0/2091057088 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9264037070 con 0x7f92700c7330 2024-04-24T23:34:39.153 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.152+0000 7f926dd9b640 1 -- 172.21.15.169:0/2091057088 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f926402ae50 con 0x7f92700c7330 2024-04-24T23:34:39.154 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.152+0000 7f926dd9b640 1 -- 172.21.15.169:0/2091057088 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9264032660 con 0x7f92700c7330 2024-04-24T23:34:39.154 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.152+0000 7f9275091640 1 -- 172.21.15.169:0/2091057088 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f92700c7330 msgr2=0x7f92700c3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:39.154 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.152+0000 7f9275091640 1 --2- 172.21.15.169:0/2091057088 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f92700c7330 0x7f92700c3430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f92640097a0 tx=0x7f926402a100 comp rx=0 tx=0).stop 2024-04-24T23:34:39.154 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.152+0000 7f9275091640 1 -- 172.21.15.169:0/2091057088 shutdown_connections 2024-04-24T23:34:39.154 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.152+0000 7f9275091640 1 --2- 172.21.15.169:0/2091057088 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f92700c7330 0x7f92700c3430 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:39.154 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.152+0000 7f9275091640 1 --2- 172.21.15.169:0/2091057088 >> v2:172.21.15.169:3300/0 conn(0x7f9270156340 0x7f92700c8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:39.154 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.152+0000 7f9275091640 1 -- 172.21.15.169:0/2091057088 >> 172.21.15.169:0/2091057088 conn(0x7f92700aa7e0 msgr2=0x7f92700aabf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:39.154 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.152+0000 7f9275091640 1 -- 172.21.15.169:0/2091057088 shutdown_connections 2024-04-24T23:34:39.154 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.152+0000 7f9275091640 1 -- 172.21.15.169:0/2091057088 wait complete. 2024-04-24T23:34:39.154 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.154+0000 7f9275091640 1 Processor -- start 2024-04-24T23:34:39.154 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.154+0000 7f9275091640 1 -- start start 2024-04-24T23:34:39.157 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.154+0000 7f9275091640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92700c3970 0x7f927014dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:39.157 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.154+0000 7f9275091640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f92700c7330 0x7f9270154100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:39.157 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.154+0000 7f9275091640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9270156340 0x7f9270157040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:39.157 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.154+0000 7f9275091640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f92700c4a60 con 0x7f92700c3970 2024-04-24T23:34:39.157 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.154+0000 7f9275091640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f92700c4780 con 0x7f9270156340 2024-04-24T23:34:39.157 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.154+0000 7f9275091640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f92701595d0 con 0x7f92700c7330 2024-04-24T23:34:39.157 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.154+0000 7f926f59e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9270156340 0x7f9270157040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:39.157 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.154+0000 7f926e59c640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f92700c7330 0x7f9270154100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:39.157 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.154+0000 7f926f59e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9270156340 0x7f9270157040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:49906/0 (socket says 172.21.15.169:49906) 2024-04-24T23:34:39.157 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.154+0000 7f926f59e640 1 -- 172.21.15.169:0/2518603216 learned_addr learned my addr 172.21.15.169:0/2518603216 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-24T23:34:39.157 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.155+0000 7f926ed9d640 1 --2- 172.21.15.169:0/2518603216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92700c3970 0x7f927014dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:39.157 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.155+0000 7f926f59e640 1 -- 172.21.15.169:0/2518603216 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f92700c7330 msgr2=0x7f9270154100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:39.157 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.155+0000 7f926f59e640 1 --2- 172.21.15.169:0/2518603216 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f92700c7330 0x7f9270154100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:39.158 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.155+0000 7f926f59e640 1 -- 172.21.15.169:0/2518603216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92700c3970 msgr2=0x7f927014dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:39.158 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.155+0000 7f926f59e640 1 --2- 172.21.15.169:0/2518603216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92700c3970 0x7f927014dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:39.158 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.155+0000 7f926f59e640 1 -- 172.21.15.169:0/2518603216 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f927014eda0 con 0x7f9270156340 2024-04-24T23:34:39.158 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.155+0000 7f926e59c640 1 --2- 172.21.15.169:0/2518603216 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f92700c7330 0x7f9270154100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:34:39.158 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.155+0000 7f926f59e640 1 --2- 172.21.15.169:0/2518603216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9270156340 0x7f9270157040 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f926800c780 tx=0x7f926800cc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:39.158 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.155+0000 7f92577fe640 1 -- 172.21.15.169:0/2518603216 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9268017070 con 0x7f9270156340 2024-04-24T23:34:39.158 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.155+0000 7f92577fe640 1 -- 172.21.15.169:0/2518603216 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f926800a640 con 0x7f9270156340 2024-04-24T23:34:39.158 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.155+0000 7f9275091640 1 -- 172.21.15.169:0/2518603216 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f927014eac0 con 0x7f9270156340 2024-04-24T23:34:39.158 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.155+0000 7f92577fe640 1 -- 172.21.15.169:0/2518603216 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926800a9f0 con 0x7f9270156340 2024-04-24T23:34:39.158 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.155+0000 7f9275091640 1 -- 172.21.15.169:0/2518603216 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f927014d430 con 0x7f9270156340 2024-04-24T23:34:39.158 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.155+0000 7f92577fe640 1 -- 172.21.15.169:0/2518603216 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f926801a070 con 0x7f9270156340 2024-04-24T23:34:39.158 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.155+0000 7f92577fe640 1 -- 172.21.15.169:0/2518603216 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f9268007910 con 0x7f9270156340 2024-04-24T23:34:39.158 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.156+0000 7f9275091640 1 -- 172.21.15.169:0/2518603216 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92700688d0 con 0x7f9270156340 2024-04-24T23:34:39.159 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.158+0000 7f92577fe640 1 -- 172.21.15.169:0/2518603216 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f926800ab90 con 0x7f9270156340 2024-04-24T23:34:39.231 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.231+0000 7f9275091640 1 -- 172.21.15.169:0/2518603216 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c37153ff-9b2b-48b9-a684-2c82b6f39f15", "id": 5} v 0) -- 0x7f92700c7110 con 0x7f9270156340 2024-04-24T23:34:39.234 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.234+0000 7f92577fe640 1 -- 172.21.15.169:0/2518603216 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c37153ff-9b2b-48b9-a684-2c82b6f39f15", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f9268007070 con 0x7f9270156340 2024-04-24T23:34:39.234 INFO:teuthology.orchestra.run.smithi169.stdout:5 2024-04-24T23:34:39.235 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.235+0000 7f9275091640 1 -- 172.21.15.169:0/2518603216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9270156340 msgr2=0x7f9270157040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:39.235 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.235+0000 7f9275091640 1 --2- 172.21.15.169:0/2518603216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9270156340 0x7f9270157040 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f926800c780 tx=0x7f926800cc40 comp rx=0 tx=0).stop 2024-04-24T23:34:39.235 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.235+0000 7f9275091640 1 -- 172.21.15.169:0/2518603216 shutdown_connections 2024-04-24T23:34:39.235 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.235+0000 7f9275091640 1 --2- 172.21.15.169:0/2518603216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9270156340 0x7f9270157040 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:39.235 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.235+0000 7f9275091640 1 --2- 172.21.15.169:0/2518603216 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f92700c7330 0x7f9270154100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:39.235 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.235+0000 7f9275091640 1 --2- 172.21.15.169:0/2518603216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92700c3970 0x7f927014dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:39.235 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.235+0000 7f9275091640 1 -- 172.21.15.169:0/2518603216 >> 172.21.15.169:0/2518603216 conn(0x7f92700aa7e0 msgr2=0x7f92700aabc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:39.235 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.235+0000 7f9275091640 1 -- 172.21.15.169:0/2518603216 shutdown_connections 2024-04-24T23:34:39.236 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.235+0000 7f9275091640 1 -- 172.21.15.169:0/2518603216 wait complete. 2024-04-24T23:34:39.245 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new 8db79e1c-f36b-49cb-8d8a-a20c2aec57c8 6 2024-04-24T23:34:39.257 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-24T23:34:39.258+0000 7f590581a040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T23:34:39.316 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-24T23:34:39.316+0000 7f590581a040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T23:34:39.355 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.355+0000 7fd925e31640 1 Processor -- start 2024-04-24T23:34:39.355 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.355+0000 7fd925e31640 1 -- start start 2024-04-24T23:34:39.357 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.356+0000 7fd925e31640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd9200b03e0 0x7fd9201567e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:39.357 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.356+0000 7fd925e31640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd9201592f0 0x7fd920155c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:39.357 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.356+0000 7fd925e31640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9200cb330 con 0x7fd9200c55d0 2024-04-24T23:34:39.357 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.356+0000 7fd925e31640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fd9200b07c0 con 0x7fd9200b03e0 2024-04-24T23:34:39.358 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.356+0000 7fd925e31640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fd9200cb610 con 0x7fd9201592f0 2024-04-24T23:34:39.358 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.356+0000 7fd917fff640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd9201592f0 0x7fd920155c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:39.358 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.356+0000 7fd91ffff640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fd9200c55d0 0x7fd9200c75f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.169:40946/0 (socket says 172.21.15.169:40946) 2024-04-24T23:34:39.358 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.356+0000 7fd91ffff640 1 -- 172.21.15.169:0/2261819368 learned_addr learned my addr 172.21.15.169:0/2261819368 (peer_addr_for_me v1:172.21.15.169:0/0) 2024-04-24T23:34:39.358 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.356+0000 7fd91f7fe640 1 --2- 172.21.15.169:0/2261819368 >> v2:172.21.15.157:3300/0 conn(0x7fd9200b03e0 0x7fd9201567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:39.358 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.357+0000 7fd917fff640 1 -- 172.21.15.169:0/2261819368 >> v2:172.21.15.157:3300/0 conn(0x7fd9200b03e0 msgr2=0x7fd9201567e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:39.358 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.357+0000 7fd917fff640 1 --2- 172.21.15.169:0/2261819368 >> v2:172.21.15.157:3300/0 conn(0x7fd9200b03e0 0x7fd9201567e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:39.358 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.357+0000 7fd917fff640 1 -- 172.21.15.169:0/2261819368 >> v1:172.21.15.169:6789/0 conn(0x7fd9200c55d0 legacy=0x7fd9200c75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:39.358 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.357+0000 7fd917fff640 1 -- 172.21.15.169:0/2261819368 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9200cb8f0 con 0x7fd9201592f0 2024-04-24T23:34:39.358 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.357+0000 7fd91effd640 1 -- 172.21.15.169:0/2261819368 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2240482542 0 0) 0x7fd9200cb330 con 0x7fd9200c55d0 2024-04-24T23:34:39.358 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.357+0000 7fd91f7fe640 1 --2- 172.21.15.169:0/2261819368 >> v2:172.21.15.157:3300/0 conn(0x7fd9200b03e0 0x7fd9201567e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:34:39.358 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.357+0000 7fd917fff640 1 --2- 172.21.15.169:0/2261819368 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9201592f0 0x7fd920155c00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd920074d00 tx=0x7fd90802a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3852e55eb356a789 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:39.358 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.357+0000 7fd91effd640 1 -- 172.21.15.169:0/2261819368 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd908037070 con 0x7fd9201592f0 2024-04-24T23:34:39.358 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.357+0000 7fd91effd640 1 -- 172.21.15.169:0/2261819368 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd908032440 con 0x7fd9201592f0 2024-04-24T23:34:39.359 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.357+0000 7fd91effd640 1 -- 172.21.15.169:0/2261819368 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90802ae60 con 0x7fd9201592f0 2024-04-24T23:34:39.361 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.358+0000 7fd925e31640 1 -- 172.21.15.169:0/2261819368 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9201592f0 msgr2=0x7fd920155c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:39.362 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.358+0000 7fd925e31640 1 --2- 172.21.15.169:0/2261819368 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9201592f0 0x7fd920155c00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd920074d00 tx=0x7fd90802a110 comp rx=0 tx=0).stop 2024-04-24T23:34:39.362 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.358+0000 7fd925e31640 1 -- 172.21.15.169:0/2261819368 shutdown_connections 2024-04-24T23:34:39.362 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.358+0000 7fd925e31640 1 --2- 172.21.15.169:0/2261819368 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9201592f0 0x7fd920155c00 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:39.362 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.358+0000 7fd925e31640 1 --2- 172.21.15.169:0/2261819368 >> v2:172.21.15.157:3300/0 conn(0x7fd9200b03e0 0x7fd9201567e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:39.362 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.358+0000 7fd925e31640 1 -- 172.21.15.169:0/2261819368 >> 172.21.15.169:0/2261819368 conn(0x7fd920067080 msgr2=0x7fd920067490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:39.362 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.358+0000 7fd925e31640 1 -- 172.21.15.169:0/2261819368 shutdown_connections 2024-04-24T23:34:39.362 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.358+0000 7fd925e31640 1 -- 172.21.15.169:0/2261819368 wait complete. 2024-04-24T23:34:39.362 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.359+0000 7fd925e31640 1 Processor -- start 2024-04-24T23:34:39.362 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.359+0000 7fd925e31640 1 -- start start 2024-04-24T23:34:39.362 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.360+0000 7fd925e31640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9200b03e0 0x7fd9201546e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:39.362 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.360+0000 7fd925e31640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9200c55d0 0x7fd92014c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:39.363 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.360+0000 7fd925e31640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9201592f0 0x7fd92014c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:39.363 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.360+0000 7fd925e31640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd920159780 con 0x7fd9200c55d0 2024-04-24T23:34:39.363 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.360+0000 7fd925e31640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd9201556e0 con 0x7fd9201592f0 2024-04-24T23:34:39.363 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.360+0000 7fd925e31640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd9200cafc0 con 0x7fd9200b03e0 2024-04-24T23:34:39.363 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.360+0000 7fd91ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9201592f0 0x7fd92014c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:39.363 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.360+0000 7fd91f7fe640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9200b03e0 0x7fd9201546e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:39.363 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.360+0000 7fd91ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9201592f0 0x7fd92014c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:49928/0 (socket says 172.21.15.169:49928) 2024-04-24T23:34:39.363 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.360+0000 7fd91ffff640 1 -- 172.21.15.169:0/3851288126 learned_addr learned my addr 172.21.15.169:0/3851288126 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-24T23:34:39.363 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.360+0000 7fd917fff640 1 --2- 172.21.15.169:0/3851288126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9200c55d0 0x7fd92014c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:39.363 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.360+0000 7fd91ffff640 1 -- 172.21.15.169:0/3851288126 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9200b03e0 msgr2=0x7fd9201546e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:39.363 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.360+0000 7fd91ffff640 1 --2- 172.21.15.169:0/3851288126 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9200b03e0 0x7fd9201546e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:39.363 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.360+0000 7fd91ffff640 1 -- 172.21.15.169:0/3851288126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9200c55d0 msgr2=0x7fd92014c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:39.363 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.360+0000 7fd91ffff640 1 --2- 172.21.15.169:0/3851288126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9200c55d0 0x7fd92014c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:39.363 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.360+0000 7fd91ffff640 1 -- 172.21.15.169:0/3851288126 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9200c9e50 con 0x7fd9201592f0 2024-04-24T23:34:39.364 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.360+0000 7fd91f7fe640 1 --2- 172.21.15.169:0/3851288126 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9200b03e0 0x7fd9201546e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:34:39.364 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.361+0000 7fd91ffff640 1 --2- 172.21.15.169:0/3851288126 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9201592f0 0x7fd92014c650 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd91800c7a0 tx=0x7fd91800cc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:39.364 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.361+0000 7fd91cff9640 1 -- 172.21.15.169:0/3851288126 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd918018070 con 0x7fd9201592f0 2024-04-24T23:34:39.364 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.361+0000 7fd91cff9640 1 -- 172.21.15.169:0/3851288126 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd91800a640 con 0x7fd9201592f0 2024-04-24T23:34:39.364 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.361+0000 7fd925e31640 1 -- 172.21.15.169:0/3851288126 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9200ca0c0 con 0x7fd9201592f0 2024-04-24T23:34:39.364 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.361+0000 7fd91cff9640 1 -- 172.21.15.169:0/3851288126 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd91800aa70 con 0x7fd9201592f0 2024-04-24T23:34:39.364 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.361+0000 7fd925e31640 1 -- 172.21.15.169:0/3851288126 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd920154c20 con 0x7fd9201592f0 2024-04-24T23:34:39.364 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.361+0000 7fd91cff9640 1 -- 172.21.15.169:0/3851288126 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd91801b070 con 0x7fd9201592f0 2024-04-24T23:34:39.364 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.361+0000 7fd91cff9640 1 -- 172.21.15.169:0/3851288126 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7fd918007850 con 0x7fd9201592f0 2024-04-24T23:34:39.364 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.362+0000 7fd925e31640 1 -- 172.21.15.169:0/3851288126 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9200681c0 con 0x7fd9201592f0 2024-04-24T23:34:39.364 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.364+0000 7fd91cff9640 1 -- 172.21.15.169:0/3851288126 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fd91800ac10 con 0x7fd9201592f0 2024-04-24T23:34:39.373 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-24T23:34:39.374+0000 7f590581a040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T23:34:39.436 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.436+0000 7fd925e31640 1 -- 172.21.15.169:0/3851288126 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd new", "uuid": "8db79e1c-f36b-49cb-8d8a-a20c2aec57c8", "id": 6} v 0) -- 0x7fd920155020 con 0x7fd9201592f0 2024-04-24T23:34:39.439 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.439+0000 7fd91cff9640 1 -- 172.21.15.169:0/3851288126 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8db79e1c-f36b-49cb-8d8a-a20c2aec57c8", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7fd91800adf0 con 0x7fd9201592f0 2024-04-24T23:34:39.439 INFO:teuthology.orchestra.run.smithi169.stdout:6 2024-04-24T23:34:39.440 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.440+0000 7fd925e31640 1 -- 172.21.15.169:0/3851288126 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9201592f0 msgr2=0x7fd92014c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:39.440 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.440+0000 7fd925e31640 1 --2- 172.21.15.169:0/3851288126 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9201592f0 0x7fd92014c650 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd91800c7a0 tx=0x7fd91800cc60 comp rx=0 tx=0).stop 2024-04-24T23:34:39.440 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.440+0000 7fd925e31640 1 -- 172.21.15.169:0/3851288126 shutdown_connections 2024-04-24T23:34:39.440 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.440+0000 7fd925e31640 1 --2- 172.21.15.169:0/3851288126 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9201592f0 0x7fd92014c650 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:39.440 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.440+0000 7fd925e31640 1 --2- 172.21.15.169:0/3851288126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9200c55d0 0x7fd92014c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:39.440 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.440+0000 7fd925e31640 1 --2- 172.21.15.169:0/3851288126 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9200b03e0 0x7fd9201546e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:39.441 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.440+0000 7fd925e31640 1 -- 172.21.15.169:0/3851288126 >> 172.21.15.169:0/3851288126 conn(0x7fd920067080 msgr2=0x7fd920067460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:39.441 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.440+0000 7fd925e31640 1 -- 172.21.15.169:0/3851288126 shutdown_connections 2024-04-24T23:34:39.441 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.440+0000 7fd925e31640 1 -- 172.21.15.169:0/3851288126 wait complete. 2024-04-24T23:34:39.450 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new 928586e9-296c-4b8b-9b27-42f9f155dbdc 7 2024-04-24T23:34:39.494 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-24T23:34:39.494+0000 7f590581a040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:34:39.549 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.549+0000 7f89d02ee640 1 Processor -- start 2024-04-24T23:34:39.549 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.549+0000 7f89d02ee640 1 -- start start 2024-04-24T23:34:39.551 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.550+0000 7f89d02ee640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89c81562b0 0x7f89c80c6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:39.551 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.550+0000 7f89d02ee640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f89c8072b70 0x7f89c815fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:39.551 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.550+0000 7f89d02ee640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89c80c1b50 con 0x7f89c8073500 2024-04-24T23:34:39.551 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.550+0000 7f89d02ee640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f89c8156690 con 0x7f89c8072b70 2024-04-24T23:34:39.551 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.550+0000 7f89d02ee640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f89c8073050 con 0x7f89c81562b0 2024-04-24T23:34:39.551 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.550+0000 7f89ce08a640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89c81562b0 0x7f89c80c6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:39.551 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.550+0000 7f89ce08a640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89c81562b0 0x7f89c80c6900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.169:41030/0 (socket says 172.21.15.169:41030) 2024-04-24T23:34:39.551 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.550+0000 7f89ce08a640 1 -- 172.21.15.169:0/4104944681 learned_addr learned my addr 172.21.15.169:0/4104944681 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-24T23:34:39.551 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.550+0000 7f89ce88b640 1 --2- 172.21.15.169:0/4104944681 >> v2:172.21.15.157:3300/0 conn(0x7f89c8072b70 0x7f89c815fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:39.551 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.550+0000 7f89ce08a640 1 -- 172.21.15.169:0/4104944681 >> v2:172.21.15.157:3300/0 conn(0x7f89c8072b70 msgr2=0x7f89c815fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:39.552 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.550+0000 7f89ce08a640 1 --2- 172.21.15.169:0/4104944681 >> v2:172.21.15.157:3300/0 conn(0x7f89c8072b70 0x7f89c815fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:39.552 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.550+0000 7f89ce08a640 1 -- 172.21.15.169:0/4104944681 >> v1:172.21.15.157:6789/0 conn(0x7f89c8073500 legacy=0x7f89c81591a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:39.552 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.550+0000 7f89ce08a640 1 -- 172.21.15.169:0/4104944681 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89c80c1e30 con 0x7f89c81562b0 2024-04-24T23:34:39.552 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.550+0000 7f89ce88b640 1 --2- 172.21.15.169:0/4104944681 >> v2:172.21.15.157:3300/0 conn(0x7f89c8072b70 0x7f89c815fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:34:39.552 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.551+0000 7f89ce08a640 1 --2- 172.21.15.169:0/4104944681 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89c81562b0 0x7f89c80c6900 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f89c4009870 tx=0x7f89c402a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0c39c0ef16a2ff7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:39.552 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.551+0000 7f89cd088640 1 -- 172.21.15.169:0/4104944681 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c4037070 con 0x7f89c81562b0 2024-04-24T23:34:39.552 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.551+0000 7f89cd088640 1 -- 172.21.15.169:0/4104944681 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89c4032440 con 0x7f89c81562b0 2024-04-24T23:34:39.552 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.551+0000 7f89cd088640 1 -- 172.21.15.169:0/4104944681 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c4032810 con 0x7f89c81562b0 2024-04-24T23:34:39.555 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.551+0000 7f89d02ee640 1 -- 172.21.15.169:0/4104944681 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89c81562b0 msgr2=0x7f89c80c6900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:39.555 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.551+0000 7f89d02ee640 1 --2- 172.21.15.169:0/4104944681 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89c81562b0 0x7f89c80c6900 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f89c4009870 tx=0x7f89c402a130 comp rx=0 tx=0).stop 2024-04-24T23:34:39.555 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.552+0000 7f89d02ee640 1 -- 172.21.15.169:0/4104944681 shutdown_connections 2024-04-24T23:34:39.555 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.552+0000 7f89d02ee640 1 --2- 172.21.15.169:0/4104944681 >> v2:172.21.15.157:3300/0 conn(0x7f89c8072b70 0x7f89c815fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:39.555 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.552+0000 7f89d02ee640 1 --2- 172.21.15.169:0/4104944681 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89c81562b0 0x7f89c80c6900 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:39.555 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.552+0000 7f89d02ee640 1 -- 172.21.15.169:0/4104944681 >> 172.21.15.169:0/4104944681 conn(0x7f89c80c44f0 msgr2=0x7f89c80c4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:39.555 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.552+0000 7f89d02ee640 1 -- 172.21.15.169:0/4104944681 shutdown_connections 2024-04-24T23:34:39.555 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.552+0000 7f89d02ee640 1 -- 172.21.15.169:0/4104944681 wait complete. 2024-04-24T23:34:39.555 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.553+0000 7f89d02ee640 1 Processor -- start 2024-04-24T23:34:39.555 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.553+0000 7f89d02ee640 1 -- start start 2024-04-24T23:34:39.555 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.553+0000 7f89d02ee640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89c8072b70 0x7f89c8157b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:39.556 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.553+0000 7f89d02ee640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89c8073500 0x7f89c80abb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:39.556 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.553+0000 7f89d02ee640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89c81562b0 0x7f89c80ac080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:39.556 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.553+0000 7f89d02ee640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f89c8159a30 con 0x7f89c8073500 2024-04-24T23:34:39.556 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.553+0000 7f89d02ee640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f89c81598b0 con 0x7f89c8072b70 2024-04-24T23:34:39.556 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.553+0000 7f89d02ee640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f89c80c6e40 con 0x7f89c81562b0 2024-04-24T23:34:39.556 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.553+0000 7f89ce88b640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89c81562b0 0x7f89c80ac080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:39.556 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.553+0000 7f89ce08a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89c8072b70 0x7f89c8157b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:39.556 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.553+0000 7f89ce88b640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89c81562b0 0x7f89c80ac080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.169:41038/0 (socket says 172.21.15.169:41038) 2024-04-24T23:34:39.556 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.554+0000 7f89ce88b640 1 -- 172.21.15.169:0/2652035608 learned_addr learned my addr 172.21.15.169:0/2652035608 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-24T23:34:39.556 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.554+0000 7f89cd889640 1 --2- 172.21.15.169:0/2652035608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89c8073500 0x7f89c80abb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:39.556 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.554+0000 7f89ce88b640 1 -- 172.21.15.169:0/2652035608 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89c8072b70 msgr2=0x7f89c8157b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:39.556 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.554+0000 7f89ce88b640 1 --2- 172.21.15.169:0/2652035608 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89c8072b70 0x7f89c8157b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:39.556 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.554+0000 7f89ce88b640 1 -- 172.21.15.169:0/2652035608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89c8073500 msgr2=0x7f89c80abb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:39.556 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.554+0000 7f89ce88b640 1 --2- 172.21.15.169:0/2652035608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89c8073500 0x7f89c80abb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:39.557 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.554+0000 7f89ce88b640 1 -- 172.21.15.169:0/2652035608 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89c80ad500 con 0x7f89c81562b0 2024-04-24T23:34:39.557 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.554+0000 7f89cd889640 1 --2- 172.21.15.169:0/2652035608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89c8073500 0x7f89c80abb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:34:39.557 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.554+0000 7f89ce88b640 1 --2- 172.21.15.169:0/2652035608 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89c81562b0 0x7f89c80ac080 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f89c0007e30 tx=0x7f89c000d4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:39.557 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.554+0000 7f89beffd640 1 -- 172.21.15.169:0/2652035608 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c001a070 con 0x7f89c81562b0 2024-04-24T23:34:39.557 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.554+0000 7f89beffd640 1 -- 172.21.15.169:0/2652035608 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89c0003b90 con 0x7f89c81562b0 2024-04-24T23:34:39.557 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.554+0000 7f89beffd640 1 -- 172.21.15.169:0/2652035608 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c000de80 con 0x7f89c81562b0 2024-04-24T23:34:39.557 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.554+0000 7f89d02ee640 1 -- 172.21.15.169:0/2652035608 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89c81ec0e0 con 0x7f89c81562b0 2024-04-24T23:34:39.557 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.554+0000 7f89d02ee640 1 -- 172.21.15.169:0/2652035608 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f89c806ee10 con 0x7f89c81562b0 2024-04-24T23:34:39.557 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.554+0000 7f89beffd640 1 -- 172.21.15.169:0/2652035608 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f89c001d070 con 0x7f89c81562b0 2024-04-24T23:34:39.557 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.554+0000 7f89beffd640 1 -- 172.21.15.169:0/2652035608 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f89c0004690 con 0x7f89c81562b0 2024-04-24T23:34:39.557 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.555+0000 7f89d02ee640 1 -- 172.21.15.169:0/2652035608 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89c80688d0 con 0x7f89c81562b0 2024-04-24T23:34:39.559 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.559+0000 7f89beffd640 1 -- 172.21.15.169:0/2652035608 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f89c0024020 con 0x7f89c81562b0 2024-04-24T23:34:39.602 INFO:tasks.ceph.mgr.y.smithi157.stderr:2024-04-24T23:34:39.603+0000 7fc502f8b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T23:34:39.631 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.631+0000 7f89d02ee640 1 -- 172.21.15.169:0/2652035608 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd new", "uuid": "928586e9-296c-4b8b-9b27-42f9f155dbdc", "id": 7} v 0) -- 0x7f89c80c7450 con 0x7f89c81562b0 2024-04-24T23:34:39.639 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.639+0000 7f89beffd640 1 -- 172.21.15.169:0/2652035608 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "928586e9-296c-4b8b-9b27-42f9f155dbdc", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f89c80ad500 con 0x7f89c81562b0 2024-04-24T23:34:39.640 INFO:teuthology.orchestra.run.smithi169.stdout:7 2024-04-24T23:34:39.640 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.640+0000 7f89d02ee640 1 -- 172.21.15.169:0/2652035608 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89c81562b0 msgr2=0x7f89c80ac080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:39.641 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.640+0000 7f89d02ee640 1 --2- 172.21.15.169:0/2652035608 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89c81562b0 0x7f89c80ac080 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f89c0007e30 tx=0x7f89c000d4a0 comp rx=0 tx=0).stop 2024-04-24T23:34:39.641 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.640+0000 7f89d02ee640 1 -- 172.21.15.169:0/2652035608 shutdown_connections 2024-04-24T23:34:39.641 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.640+0000 7f89d02ee640 1 --2- 172.21.15.169:0/2652035608 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89c81562b0 0x7f89c80ac080 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:39.641 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.640+0000 7f89d02ee640 1 --2- 172.21.15.169:0/2652035608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89c8073500 0x7f89c80abb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:39.641 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.640+0000 7f89d02ee640 1 --2- 172.21.15.169:0/2652035608 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89c8072b70 0x7f89c8157b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:39.641 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.640+0000 7f89d02ee640 1 -- 172.21.15.169:0/2652035608 >> 172.21.15.169:0/2652035608 conn(0x7f89c80c44f0 msgr2=0x7f89c80c48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:39.641 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.640+0000 7f89d02ee640 1 -- 172.21.15.169:0/2652035608 shutdown_connections 2024-04-24T23:34:39.641 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:34:39.640+0000 7f89d02ee640 1 -- 172.21.15.169:0/2652035608 wait complete. 2024-04-24T23:34:39.651 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-24T23:34:39.651 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-osd -f --cluster ceph -i 0 2024-04-24T23:34:39.653 INFO:tasks.ceph.osd.0:Started 2024-04-24T23:34:39.653 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-24T23:34:39.653 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-osd -f --cluster ceph -i 1 2024-04-24T23:34:39.655 INFO:tasks.ceph.osd.1:Started 2024-04-24T23:34:39.655 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-24T23:34:39.656 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-osd -f --cluster ceph -i 2 2024-04-24T23:34:39.657 INFO:tasks.ceph.osd.2:Started 2024-04-24T23:34:39.658 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-24T23:34:39.658 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-osd -f --cluster ceph -i 3 2024-04-24T23:34:39.659 INFO:tasks.ceph.osd.3:Started 2024-04-24T23:34:39.660 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-24T23:34:39.660 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-osd -f --cluster ceph -i 4 2024-04-24T23:34:39.693 INFO:tasks.ceph.osd.4:Started 2024-04-24T23:34:39.693 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-24T23:34:39.693 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-osd -f --cluster ceph -i 5 2024-04-24T23:34:39.695 INFO:tasks.ceph.osd.5:Started 2024-04-24T23:34:39.695 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-24T23:34:39.695 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-osd -f --cluster ceph -i 6 2024-04-24T23:34:39.697 INFO:tasks.ceph.osd.6:Started 2024-04-24T23:34:39.698 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-24T23:34:39.698 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-osd -f --cluster ceph -i 7 2024-04-24T23:34:39.700 INFO:tasks.ceph.mgr.y.smithi157.stderr:2024-04-24T23:34:39.700+0000 7fc502f8b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T23:34:39.702 INFO:tasks.ceph.osd.7:Started 2024-04-24T23:34:39.702 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:34:39.757 INFO:tasks.ceph.mgr.y.smithi157.stderr:2024-04-24T23:34:39.758+0000 7fc502f8b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T23:34:39.814 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.815+0000 7f11fa5ed640 1 Processor -- start 2024-04-24T23:34:39.815 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.815+0000 7f11fa5ed640 1 -- start start 2024-04-24T23:34:39.816 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.816+0000 7f11fa5ed640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f11f41562d0 0x7f11f406b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:39.816 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.816+0000 7f11fa5ed640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11f40c3e20 0x7f11f4159310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:39.816 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.816+0000 7f11fa5ed640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11f40c2c00 con 0x7f11f406b6a0 2024-04-24T23:34:39.816 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.816+0000 7f11fa5ed640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f11f41566b0 con 0x7f11f41562d0 2024-04-24T23:34:39.816 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.816+0000 7f11fa5ed640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f11f40c2ee0 con 0x7f11f40c3e20 2024-04-24T23:34:39.816 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.816+0000 7f11f3fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f11f41562d0 0x7f11f406b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:39.816 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.816+0000 7f11f3fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f11f41562d0 0x7f11f406b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58200/0 (socket says 172.21.15.157:58200) 2024-04-24T23:34:39.817 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.816+0000 7f11f3fff640 1 -- 172.21.15.157:0/2035303514 learned_addr learned my addr 172.21.15.157:0/2035303514 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:34:39.817 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.816+0000 7f11f8b8a640 1 --2- 172.21.15.157:0/2035303514 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11f40c3e20 0x7f11f4159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:39.817 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.816+0000 7f11f3fff640 1 -- 172.21.15.157:0/2035303514 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11f40c3e20 msgr2=0x7f11f4159310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:39.817 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.816+0000 7f11f3fff640 1 --2- 172.21.15.157:0/2035303514 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11f40c3e20 0x7f11f4159310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:39.817 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.816+0000 7f11f3fff640 1 -- 172.21.15.157:0/2035303514 >> v1:172.21.15.169:6789/0 conn(0x7f11f406b6a0 legacy=0x7f11f4158c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:39.817 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.816+0000 7f11f3fff640 1 -- 172.21.15.157:0/2035303514 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11f40c31c0 con 0x7f11f41562d0 2024-04-24T23:34:39.817 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.817+0000 7f11f3fff640 1 --2- 172.21.15.157:0/2035303514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11f41562d0 0x7f11f406b160 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f11e8009870 tx=0x7f11e802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bbcd504c59790f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:39.818 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.817+0000 7f11f2ffd640 1 -- 172.21.15.157:0/2035303514 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e8037070 con 0x7f11f41562d0 2024-04-24T23:34:39.818 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.817+0000 7f11f2ffd640 1 -- 172.21.15.157:0/2035303514 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11e8032440 con 0x7f11f41562d0 2024-04-24T23:34:39.818 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.817+0000 7f11f2ffd640 1 -- 172.21.15.157:0/2035303514 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e8032810 con 0x7f11f41562d0 2024-04-24T23:34:39.818 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.817+0000 7f11fa5ed640 1 -- 172.21.15.157:0/2035303514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11f41562d0 msgr2=0x7f11f406b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:39.819 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.817+0000 7f11fa5ed640 1 --2- 172.21.15.157:0/2035303514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11f41562d0 0x7f11f406b160 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f11e8009870 tx=0x7f11e802a1c0 comp rx=0 tx=0).stop 2024-04-24T23:34:39.819 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.817+0000 7f11fa5ed640 1 -- 172.21.15.157:0/2035303514 shutdown_connections 2024-04-24T23:34:39.819 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.818+0000 7f11fa5ed640 1 --2- 172.21.15.157:0/2035303514 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11f40c3e20 0x7f11f4159310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:39.819 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.818+0000 7f11fa5ed640 1 --2- 172.21.15.157:0/2035303514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11f41562d0 0x7f11f406b160 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:39.819 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.818+0000 7f11fa5ed640 1 -- 172.21.15.157:0/2035303514 >> 172.21.15.157:0/2035303514 conn(0x7f11f40a97a0 msgr2=0x7f11f40a9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:39.819 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.818+0000 7f11fa5ed640 1 -- 172.21.15.157:0/2035303514 shutdown_connections 2024-04-24T23:34:39.819 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.818+0000 7f11fa5ed640 1 -- 172.21.15.157:0/2035303514 wait complete. 2024-04-24T23:34:39.819 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.819+0000 7f11fa5ed640 1 Processor -- start 2024-04-24T23:34:39.819 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.819+0000 7f11fa5ed640 1 -- start start 2024-04-24T23:34:39.821 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.819+0000 7f11fa5ed640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11f406b6a0 0x7f11f40aa890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:39.821 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.819+0000 7f11fa5ed640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11f40c3e20 0x7f11f4158090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:39.821 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.819+0000 7f11fa5ed640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11f41562d0 0x7f11f41e1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:39.821 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.819+0000 7f11fa5ed640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f11f4159850 con 0x7f11f41562d0 2024-04-24T23:34:39.822 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.819+0000 7f11fa5ed640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f11f40c3780 con 0x7f11f40c3e20 2024-04-24T23:34:39.822 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.819+0000 7f11fa5ed640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f11f4072e30 con 0x7f11f406b6a0 2024-04-24T23:34:39.822 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.820+0000 7f11f8b8a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11f41562d0 0x7f11f41e1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:39.822 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.820+0000 7f11f8b8a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11f41562d0 0x7f11f41e1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58204/0 (socket says 172.21.15.157:58204) 2024-04-24T23:34:39.822 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.820+0000 7f11f8b8a640 1 -- 172.21.15.157:0/2419298148 learned_addr learned my addr 172.21.15.157:0/2419298148 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:34:39.822 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.820+0000 7f11f3fff640 1 --2- 172.21.15.157:0/2419298148 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11f406b6a0 0x7f11f40aa890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:39.822 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.820+0000 7f11f37fe640 1 --2- 172.21.15.157:0/2419298148 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11f40c3e20 0x7f11f4158090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:39.822 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.820+0000 7f11f8b8a640 1 -- 172.21.15.157:0/2419298148 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11f406b6a0 msgr2=0x7f11f40aa890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:39.822 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.820+0000 7f11f8b8a640 1 --2- 172.21.15.157:0/2419298148 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11f406b6a0 0x7f11f40aa890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:39.822 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.820+0000 7f11f8b8a640 1 -- 172.21.15.157:0/2419298148 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11f40c3e20 msgr2=0x7f11f4158090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:39.822 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.820+0000 7f11f8b8a640 1 --2- 172.21.15.157:0/2419298148 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11f40c3e20 0x7f11f4158090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:39.822 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.820+0000 7f11f8b8a640 1 -- 172.21.15.157:0/2419298148 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11f40ac580 con 0x7f11f41562d0 2024-04-24T23:34:39.822 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.820+0000 7f11f3fff640 1 --2- 172.21.15.157:0/2419298148 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11f406b6a0 0x7f11f40aa890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:34:39.823 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.820+0000 7f11f8b8a640 1 --2- 172.21.15.157:0/2419298148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11f41562d0 0x7f11f41e1b50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f11ec00efc0 tx=0x7f11ec00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:39.823 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.820+0000 7f11f0ff9640 1 -- 172.21.15.157:0/2419298148 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ec018020 con 0x7f11f41562d0 2024-04-24T23:34:39.823 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.820+0000 7f11f0ff9640 1 -- 172.21.15.157:0/2419298148 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11ec014480 con 0x7f11f41562d0 2024-04-24T23:34:39.823 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.820+0000 7f11fa5ed640 1 -- 172.21.15.157:0/2419298148 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11f40ac860 con 0x7f11f41562d0 2024-04-24T23:34:39.823 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.820+0000 7f11f0ff9640 1 -- 172.21.15.157:0/2419298148 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ec014870 con 0x7f11f41562d0 2024-04-24T23:34:39.823 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.820+0000 7f11fa5ed640 1 -- 172.21.15.157:0/2419298148 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11f41585d0 con 0x7f11f41562d0 2024-04-24T23:34:39.823 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.820+0000 7f11f0ff9640 1 -- 172.21.15.157:0/2419298148 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f11ec01b020 con 0x7f11f41562d0 2024-04-24T23:34:39.823 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.820+0000 7f11f0ff9640 1 -- 172.21.15.157:0/2419298148 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f11ec01de00 con 0x7f11f41562d0 2024-04-24T23:34:39.823 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.821+0000 7f11fa5ed640 1 -- 172.21.15.157:0/2419298148 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11f40688d0 con 0x7f11f41562d0 2024-04-24T23:34:39.823 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.823+0000 7f11f0ff9640 1 -- 172.21.15.157:0/2419298148 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f11ec022030 con 0x7f11f41562d0 2024-04-24T23:34:39.884 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.884+0000 7f11fa5ed640 1 -- 172.21.15.157:0/2419298148 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f11f4074c70 con 0x7f11f41562d0 2024-04-24T23:34:39.884 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.885+0000 7f11f0ff9640 1 -- 172.21.15.157:0/2419298148 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f11ec010040 con 0x7f11f41562d0 2024-04-24T23:34:39.885 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:34:39.885 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":10,"fsid":"223685fb-5b64-46a5-9857-3791c9c07df2","created":"2024-04-24T23:34:37.504202+0000","modified":"2024-04-24T23:34:39.632954+0000","last_up_change":"0.000000","last_in_change":"2024-04-24T23:34:39.632954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"95d14ac8-fccf-4261-be8e-edafa746f629","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7be79da9-8eec-4990-9178-71d602e39d9b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"0127ed00-b7c3-466d-bfb0-01b0fd315f49","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"f2331acc-aa37-433e-868e-3561b6039562","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"1998854b-7398-43f5-ab7d-79fa0debcf97","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"c37153ff-9b2b-48b9-a684-2c82b6f39f15","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"8db79e1c-f36b-49cb-8d8a-a20c2aec57c8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"928586e9-296c-4b8b-9b27-42f9f155dbdc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:34:39.886 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.886+0000 7f11fa5ed640 1 -- 172.21.15.157:0/2419298148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11f41562d0 msgr2=0x7f11f41e1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:39.886 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.886+0000 7f11fa5ed640 1 --2- 172.21.15.157:0/2419298148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11f41562d0 0x7f11f41e1b50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f11ec00efc0 tx=0x7f11ec00c490 comp rx=0 tx=0).stop 2024-04-24T23:34:39.886 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.886+0000 7f11fa5ed640 1 -- 172.21.15.157:0/2419298148 shutdown_connections 2024-04-24T23:34:39.886 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.886+0000 7f11fa5ed640 1 --2- 172.21.15.157:0/2419298148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11f41562d0 0x7f11f41e1b50 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:39.886 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.886+0000 7f11fa5ed640 1 --2- 172.21.15.157:0/2419298148 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11f40c3e20 0x7f11f4158090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:39.886 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.886+0000 7f11fa5ed640 1 --2- 172.21.15.157:0/2419298148 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11f406b6a0 0x7f11f40aa890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:39.886 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.886+0000 7f11fa5ed640 1 -- 172.21.15.157:0/2419298148 >> 172.21.15.157:0/2419298148 conn(0x7f11f40a97a0 msgr2=0x7f11f4072410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:39.886 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.886+0000 7f11fa5ed640 1 -- 172.21.15.157:0/2419298148 shutdown_connections 2024-04-24T23:34:39.886 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:39.886+0000 7f11fa5ed640 1 -- 172.21.15.157:0/2419298148 wait complete. 2024-04-24T23:34:39.896 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-24T23:34:39.896 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-24T23:34:39.919 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-24T23:34:39.919+0000 7f590581a040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T23:34:39.933 INFO:tasks.ceph.mgr.y.smithi157.stderr:2024-04-24T23:34:39.933+0000 7fc502f8b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T23:34:39.988 INFO:tasks.ceph.mgr.y.smithi157.stderr:2024-04-24T23:34:39.988+0000 7fc502f8b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T23:34:40.042 INFO:tasks.ceph.mgr.y.smithi157.stderr:2024-04-24T23:34:40.042+0000 7fc502f8b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T23:34:40.153 INFO:tasks.ceph.mgr.y.smithi157.stderr:2024-04-24T23:34:40.153+0000 7fc502f8b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:34:40.182 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-24T23:34:40.182+0000 7f590581a040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T23:34:40.235 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-24T23:34:40.235+0000 7f590581a040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T23:34:40.265 INFO:tasks.ceph.osd.3.smithi157.stderr:2024-04-24T23:34:40.266+0000 7fc5ec4e1640 -1 Falling back to public interface 2024-04-24T23:34:40.273 INFO:tasks.ceph.osd.0.smithi157.stderr:2024-04-24T23:34:40.273+0000 7f9af7114640 -1 Falling back to public interface 2024-04-24T23:34:40.277 INFO:tasks.ceph.osd.1.smithi157.stderr:2024-04-24T23:34:40.277+0000 7f33b881d640 -1 Falling back to public interface 2024-04-24T23:34:40.285 INFO:tasks.ceph.osd.2.smithi157.stderr:2024-04-24T23:34:40.285+0000 7fbb55e0e640 -1 Falling back to public interface 2024-04-24T23:34:40.305 INFO:tasks.ceph.osd.4.smithi169.stderr:2024-04-24T23:34:40.305+0000 7f664ce1e640 -1 Falling back to public interface 2024-04-24T23:34:40.312 INFO:tasks.ceph.osd.6.smithi169.stderr:2024-04-24T23:34:40.312+0000 7f3ed0231640 -1 Falling back to public interface 2024-04-24T23:34:40.315 INFO:tasks.ceph.osd.7.smithi169.stderr:2024-04-24T23:34:40.315+0000 7fdc5beff640 -1 Falling back to public interface 2024-04-24T23:34:40.323 INFO:tasks.ceph.osd.5.smithi169.stderr:2024-04-24T23:34:40.323+0000 7f96c1a1a640 -1 Falling back to public interface 2024-04-24T23:34:40.355 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-24T23:34:40.355+0000 7f590581a040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T23:34:40.533 INFO:tasks.ceph.mgr.y.smithi157.stderr:2024-04-24T23:34:40.534+0000 7fc502f8b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T23:34:40.638 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-24T23:34:40.638+0000 7f590581a040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T23:34:40.686 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-24T23:34:40.686+0000 7f590581a040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T23:34:40.702 DEBUG:teuthology.orchestra.run.smithi157:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-24T23:34:40.793 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.793+0000 7f6362676640 1 Processor -- start 2024-04-24T23:34:40.793 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.793+0000 7f6362676640 1 -- start start 2024-04-24T23:34:40.794 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.794+0000 7f6362676640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f635c14fb90 0x7f635c0a7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:40.794 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.794+0000 7f6362676640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f635c0c7330 0x7f635c1556e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:40.794 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.794+0000 7f6362676640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f635c0c65e0 con 0x7f635c14f7b0 2024-04-24T23:34:40.794 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.794+0000 7f6362676640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f635c0c77b0 con 0x7f635c0c7330 2024-04-24T23:34:40.794 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.794+0000 7f6362676640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f635c0c6910 con 0x7f635c14fb90 2024-04-24T23:34:40.794 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.794+0000 7f6360c13640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f635c0c7330 0x7f635c1556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:40.795 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.794+0000 7f6360c13640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f635c0c7330 0x7f635c1556e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58214/0 (socket says 172.21.15.157:58214) 2024-04-24T23:34:40.795 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.794+0000 7f6360c13640 1 -- 172.21.15.157:0/3991582340 learned_addr learned my addr 172.21.15.157:0/3991582340 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:34:40.795 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.794+0000 7f635b7fe640 1 --2- 172.21.15.157:0/3991582340 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f635c14fb90 0x7f635c0a7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:40.795 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.794+0000 7f6360c13640 1 -- 172.21.15.157:0/3991582340 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f635c14fb90 msgr2=0x7f635c0a7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:40.795 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.794+0000 7f6360c13640 1 --2- 172.21.15.157:0/3991582340 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f635c14fb90 0x7f635c0a7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:40.795 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.794+0000 7f6360c13640 1 -- 172.21.15.157:0/3991582340 >> v1:172.21.15.169:6789/0 conn(0x7f635c14f7b0 legacy=0x7f635c0c5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:40.795 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.794+0000 7f6360c13640 1 -- 172.21.15.157:0/3991582340 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f635c0c6260 con 0x7f635c0c7330 2024-04-24T23:34:40.796 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.795+0000 7f6360c13640 1 --2- 172.21.15.157:0/3991582340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f635c0c7330 0x7f635c1556e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6354009e50 tx=0x7f635402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29ab6d7344902a48 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:40.796 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.795+0000 7f635affd640 1 -- 172.21.15.157:0/3991582340 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6354036070 con 0x7f635c0c7330 2024-04-24T23:34:40.797 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.795+0000 7f635affd640 1 -- 172.21.15.157:0/3991582340 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6354031600 con 0x7f635c0c7330 2024-04-24T23:34:40.797 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.795+0000 7f635affd640 1 -- 172.21.15.157:0/3991582340 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63540319d0 con 0x7f635c0c7330 2024-04-24T23:34:40.797 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.796+0000 7f6362676640 1 -- 172.21.15.157:0/3991582340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f635c0c7330 msgr2=0x7f635c1556e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:40.797 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.796+0000 7f6362676640 1 --2- 172.21.15.157:0/3991582340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f635c0c7330 0x7f635c1556e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6354009e50 tx=0x7f635402a680 comp rx=0 tx=0).stop 2024-04-24T23:34:40.797 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.796+0000 7f6362676640 1 -- 172.21.15.157:0/3991582340 shutdown_connections 2024-04-24T23:34:40.797 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.796+0000 7f6362676640 1 --2- 172.21.15.157:0/3991582340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f635c0c7330 0x7f635c1556e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:40.797 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.796+0000 7f6362676640 1 --2- 172.21.15.157:0/3991582340 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f635c14fb90 0x7f635c0a7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:40.797 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.796+0000 7f6362676640 1 -- 172.21.15.157:0/3991582340 >> 172.21.15.157:0/3991582340 conn(0x7f635c0a9780 msgr2=0x7f635c0a9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:40.797 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.796+0000 7f6362676640 1 -- 172.21.15.157:0/3991582340 shutdown_connections 2024-04-24T23:34:40.797 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.796+0000 7f6362676640 1 -- 172.21.15.157:0/3991582340 wait complete. 2024-04-24T23:34:40.797 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.797+0000 7f6362676640 1 Processor -- start 2024-04-24T23:34:40.797 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.797+0000 7f6362676640 1 -- start start 2024-04-24T23:34:40.799 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.798+0000 7f6362676640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f635c0c7330 0x7f635c0bcc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:40.799 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.798+0000 7f6362676640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f635c14f7b0 0x7f635c14bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:40.799 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.798+0000 7f6362676640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f635c14fb90 0x7f635c14c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:40.799 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.798+0000 7f6362676640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f635c0c27e0 con 0x7f635c14f7b0 2024-04-24T23:34:40.799 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.798+0000 7f6362676640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f635c156570 con 0x7f635c14fb90 2024-04-24T23:34:40.799 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.798+0000 7f6362676640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f635c0c2960 con 0x7f635c0c7330 2024-04-24T23:34:40.799 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.798+0000 7f635b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f635c14f7b0 0x7f635c14bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:40.799 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.798+0000 7f635b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f635c14f7b0 0x7f635c14bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58222/0 (socket says 172.21.15.157:58222) 2024-04-24T23:34:40.800 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.798+0000 7f635bfff640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f635c0c7330 0x7f635c0bcc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:40.800 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.798+0000 7f635b7fe640 1 -- 172.21.15.157:0/1706105786 learned_addr learned my addr 172.21.15.157:0/1706105786 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:34:40.800 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.798+0000 7f6360c13640 1 --2- 172.21.15.157:0/1706105786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f635c14fb90 0x7f635c14c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:40.800 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.798+0000 7f635bfff640 1 -- 172.21.15.157:0/1706105786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f635c14fb90 msgr2=0x7f635c14c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:40.800 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.798+0000 7f635bfff640 1 --2- 172.21.15.157:0/1706105786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f635c14fb90 0x7f635c14c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:40.800 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.798+0000 7f635bfff640 1 -- 172.21.15.157:0/1706105786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f635c14f7b0 msgr2=0x7f635c14bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:40.800 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.798+0000 7f635bfff640 1 --2- 172.21.15.157:0/1706105786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f635c14f7b0 0x7f635c14bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:40.800 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.798+0000 7f635bfff640 1 -- 172.21.15.157:0/1706105786 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f635c0ada60 con 0x7f635c0c7330 2024-04-24T23:34:40.800 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.798+0000 7f635bfff640 1 --2- 172.21.15.157:0/1706105786 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f635c0c7330 0x7f635c0bcc60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f635000c370 tx=0x7f635000c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:40.800 INFO:tasks.ceph.mgr.y.smithi157.stderr:2024-04-24T23:34:40.798+0000 7fc502f8b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T23:34:40.801 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.798+0000 7f6358ff9640 1 -- 172.21.15.157:0/1706105786 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63500052d0 con 0x7f635c0c7330 2024-04-24T23:34:40.802 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.798+0000 7f6358ff9640 1 -- 172.21.15.157:0/1706105786 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6350005490 con 0x7f635c0c7330 2024-04-24T23:34:40.802 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.798+0000 7f6358ff9640 1 -- 172.21.15.157:0/1706105786 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6350007260 con 0x7f635c0c7330 2024-04-24T23:34:40.802 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.798+0000 7f6362676640 1 -- 172.21.15.157:0/1706105786 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f635c152370 con 0x7f635c0c7330 2024-04-24T23:34:40.802 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.799+0000 7f6362676640 1 -- 172.21.15.157:0/1706105786 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f635c152930 con 0x7f635c0c7330 2024-04-24T23:34:40.802 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.799+0000 7f6358ff9640 1 -- 172.21.15.157:0/1706105786 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6350007400 con 0x7f635c0c7330 2024-04-24T23:34:40.802 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.799+0000 7f6358ff9640 1 -- 172.21.15.157:0/1706105786 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f6350003a00 con 0x7f635c0c7330 2024-04-24T23:34:40.802 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.800+0000 7f6362676640 1 -- 172.21.15.157:0/1706105786 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f635c075530 con 0x7f635c0c7330 2024-04-24T23:34:40.802 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.802+0000 7f6358ff9640 1 -- 172.21.15.157:0/1706105786 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f635c0ada60 con 0x7f635c0c7330 2024-04-24T23:34:40.844 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-24T23:34:40.844+0000 7f590581a040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T23:34:40.853 INFO:tasks.ceph.mgr.y.smithi157.stderr:2024-04-24T23:34:40.853+0000 7fc502f8b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T23:34:40.863 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.864+0000 7f6362676640 1 -- 172.21.15.157:0/1706105786 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f635c0bd1a0 con 0x7f635c0c7330 2024-04-24T23:34:40.864 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.865+0000 7f6358ff9640 1 -- 172.21.15.157:0/1706105786 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f63500035f0 con 0x7f635c0c7330 2024-04-24T23:34:40.864 INFO:teuthology.misc.health.smithi157.stdout: 2024-04-24T23:34:40.864 INFO:teuthology.misc.health.smithi157.stdout:{"epoch":10,"fsid":"223685fb-5b64-46a5-9857-3791c9c07df2","created":"2024-04-24T23:34:37.504202+0000","modified":"2024-04-24T23:34:39.632954+0000","last_up_change":"0.000000","last_in_change":"2024-04-24T23:34:39.632954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"95d14ac8-fccf-4261-be8e-edafa746f629","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7be79da9-8eec-4990-9178-71d602e39d9b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"0127ed00-b7c3-466d-bfb0-01b0fd315f49","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"f2331acc-aa37-433e-868e-3561b6039562","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"1998854b-7398-43f5-ab7d-79fa0debcf97","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"c37153ff-9b2b-48b9-a684-2c82b6f39f15","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"8db79e1c-f36b-49cb-8d8a-a20c2aec57c8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"928586e9-296c-4b8b-9b27-42f9f155dbdc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:34:40.865 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.865+0000 7f6362676640 1 -- 172.21.15.157:0/1706105786 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f635c0c7330 msgr2=0x7f635c0bcc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:40.865 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.865+0000 7f6362676640 1 --2- 172.21.15.157:0/1706105786 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f635c0c7330 0x7f635c0bcc60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f635000c370 tx=0x7f635000c830 comp rx=0 tx=0).stop 2024-04-24T23:34:40.866 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.866+0000 7f6362676640 1 -- 172.21.15.157:0/1706105786 shutdown_connections 2024-04-24T23:34:40.866 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.866+0000 7f6362676640 1 --2- 172.21.15.157:0/1706105786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f635c14fb90 0x7f635c14c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:40.866 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.866+0000 7f6362676640 1 --2- 172.21.15.157:0/1706105786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f635c14f7b0 0x7f635c14bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:40.866 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.866+0000 7f6362676640 1 --2- 172.21.15.157:0/1706105786 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f635c0c7330 0x7f635c0bcc60 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:40.866 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.866+0000 7f6362676640 1 -- 172.21.15.157:0/1706105786 >> 172.21.15.157:0/1706105786 conn(0x7f635c0a9780 msgr2=0x7f635c0c55c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:40.866 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.866+0000 7f6362676640 1 -- 172.21.15.157:0/1706105786 shutdown_connections 2024-04-24T23:34:40.866 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:40.866+0000 7f6362676640 1 -- 172.21.15.157:0/1706105786 wait complete. 2024-04-24T23:34:40.875 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-24T23:34:40.971 INFO:tasks.ceph.mgr.y.smithi157.stderr:2024-04-24T23:34:40.971+0000 7fc502f8b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T23:34:40.983 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-24T23:34:40.983+0000 7f590581a040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T23:34:41.033 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-24T23:34:41.033+0000 7f590581a040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T23:34:41.103 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-24T23:34:41.103+0000 7f590581a040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T23:34:41.151 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-24T23:34:41.151+0000 7f590581a040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T23:34:41.248 INFO:tasks.ceph.mgr.y.smithi157.stderr:2024-04-24T23:34:41.248+0000 7fc502f8b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T23:34:41.294 INFO:tasks.ceph.mgr.y.smithi157.stderr:2024-04-24T23:34:41.295+0000 7fc502f8b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T23:34:41.452 INFO:tasks.ceph.mgr.y.smithi157.stderr:2024-04-24T23:34:41.453+0000 7fc502f8b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T23:34:41.586 INFO:tasks.ceph.mgr.y.smithi157.stderr:2024-04-24T23:34:41.587+0000 7fc502f8b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T23:34:41.635 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-24T23:34:41.635+0000 7f590581a040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T23:34:41.636 INFO:tasks.ceph.mgr.y.smithi157.stderr:2024-04-24T23:34:41.637+0000 7fc502f8b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T23:34:41.708 INFO:tasks.ceph.mgr.y.smithi157.stderr:2024-04-24T23:34:41.708+0000 7fc502f8b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T23:34:41.752 INFO:tasks.ceph.mgr.y.smithi157.stderr:2024-04-24T23:34:41.753+0000 7fc502f8b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T23:34:41.754 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-24T23:34:41.754+0000 7f590581a040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T23:34:42.232 INFO:tasks.ceph.mgr.y.smithi157.stderr:2024-04-24T23:34:42.233+0000 7fc502f8b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T23:34:42.347 INFO:tasks.ceph.mgr.y.smithi157.stderr:2024-04-24T23:34:42.347+0000 7fc502f8b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T23:34:42.769 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-24T23:34:42.769+0000 7f58a5f77640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-24T23:34:42.769 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-24T23:34:42.769+0000 7f58a5f77640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-24T23:34:43.733 INFO:tasks.ceph.osd.3.smithi157.stderr:2024-04-24T23:34:43.734+0000 7fc5ec4e1640 -1 osd.3 0 log_to_monitors true 2024-04-24T23:34:43.775 INFO:tasks.ceph.osd.1.smithi157.stderr:2024-04-24T23:34:43.775+0000 7f33b881d640 -1 osd.1 0 log_to_monitors true 2024-04-24T23:34:43.785 INFO:tasks.ceph.osd.4.smithi169.stderr:2024-04-24T23:34:43.785+0000 7f664ce1e640 -1 osd.4 0 log_to_monitors true 2024-04-24T23:34:43.795 INFO:tasks.ceph.osd.7.smithi169.stderr:2024-04-24T23:34:43.795+0000 7fdc5beff640 -1 osd.7 0 log_to_monitors true 2024-04-24T23:34:43.817 INFO:tasks.ceph.osd.2.smithi157.stderr:2024-04-24T23:34:43.817+0000 7fbb55e0e640 -1 osd.2 0 log_to_monitors true 2024-04-24T23:34:43.821 INFO:tasks.ceph.osd.0.smithi157.stderr:2024-04-24T23:34:43.821+0000 7f9af7114640 -1 osd.0 0 log_to_monitors true 2024-04-24T23:34:43.833 INFO:tasks.ceph.osd.5.smithi169.stderr:2024-04-24T23:34:43.834+0000 7f96c1a1a640 -1 osd.5 0 log_to_monitors true 2024-04-24T23:34:43.840 INFO:tasks.ceph.osd.6.smithi169.stderr:2024-04-24T23:34:43.840+0000 7f3ed0231640 -1 osd.6 0 log_to_monitors true 2024-04-24T23:34:45.345 INFO:tasks.ceph.osd.3.smithi157.stderr:2024-04-24T23:34:45.345+0000 7fc5e8477640 -1 osd.3 0 waiting for initial osdmap 2024-04-24T23:34:45.359 INFO:tasks.ceph.osd.3.smithi157.stderr:2024-04-24T23:34:45.360+0000 7fc5e0297640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:34:46.582 INFO:tasks.ceph.osd.2.smithi157.stderr:2024-04-24T23:34:46.582+0000 7fbb517df640 -1 osd.2 0 waiting for initial osdmap 2024-04-24T23:34:46.597 INFO:tasks.ceph.osd.2.smithi157.stderr:2024-04-24T23:34:46.598+0000 7fbb495ff640 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:34:46.618 INFO:tasks.ceph.osd.0.smithi157.stderr:2024-04-24T23:34:46.618+0000 7f9af30aa640 -1 osd.0 0 waiting for initial osdmap 2024-04-24T23:34:46.622 INFO:tasks.ceph.osd.1.smithi157.stderr:2024-04-24T23:34:46.623+0000 7f33b47b3640 -1 osd.1 0 waiting for initial osdmap 2024-04-24T23:34:46.628 INFO:tasks.ceph.osd.0.smithi157.stderr:2024-04-24T23:34:46.629+0000 7f9aeaeca640 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:34:46.636 INFO:tasks.ceph.osd.1.smithi157.stderr:2024-04-24T23:34:46.637+0000 7f33ac5d3640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:34:46.787 INFO:tasks.ceph.osd.4.smithi169.stderr:2024-04-24T23:34:46.787+0000 7f6648859640 -1 osd.4 0 waiting for initial osdmap 2024-04-24T23:34:46.796 INFO:tasks.ceph.osd.5.smithi169.stderr:2024-04-24T23:34:46.796+0000 7f96bd3a0640 -1 osd.5 0 waiting for initial osdmap 2024-04-24T23:34:46.799 INFO:tasks.ceph.osd.6.smithi169.stderr:2024-04-24T23:34:46.799+0000 7f3ecb780640 -1 osd.6 0 waiting for initial osdmap 2024-04-24T23:34:46.811 INFO:tasks.ceph.osd.4.smithi169.stderr:2024-04-24T23:34:46.811+0000 7f664068a640 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:34:46.814 INFO:tasks.ceph.osd.5.smithi169.stderr:2024-04-24T23:34:46.814+0000 7f96b51c0640 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:34:46.818 INFO:tasks.ceph.osd.6.smithi169.stderr:2024-04-24T23:34:46.819+0000 7f3ec35b1640 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:34:46.836 INFO:tasks.ceph.osd.7.smithi169.stderr:2024-04-24T23:34:46.836+0000 7fdc57e95640 -1 osd.7 0 waiting for initial osdmap 2024-04-24T23:34:46.847 INFO:tasks.ceph.osd.7.smithi169.stderr:2024-04-24T23:34:46.847+0000 7fdc4fcb5640 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:34:47.681 DEBUG:teuthology.orchestra.run.smithi157:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-24T23:34:47.782 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.782+0000 7f11bc64d640 1 Processor -- start 2024-04-24T23:34:47.782 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.782+0000 7f11bc64d640 1 -- start start 2024-04-24T23:34:47.784 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.783+0000 7f11bc64d640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11b4150920 0x7f11b4153d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:47.784 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.783+0000 7f11bc64d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f11b40c6280 0x7f11b4070e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:47.784 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.783+0000 7f11bc64d640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11b406ebb0 con 0x7f11b40c3f50 2024-04-24T23:34:47.784 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.783+0000 7f11bc64d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f11b40c3d40 con 0x7f11b40c6280 2024-04-24T23:34:47.784 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.783+0000 7f11bc64d640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f11b406e960 con 0x7f11b4150920 2024-04-24T23:34:47.784 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.784+0000 7f11ba3e9640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11b4150920 0x7f11b4153d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:47.784 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.784+0000 7f11babea640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f11b40c3f50 0x7f11b40a7d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.157:51834/0 (socket says 172.21.15.157:51834) 2024-04-24T23:34:47.784 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.784+0000 7f11babea640 1 -- 172.21.15.157:0/3529455391 learned_addr learned my addr 172.21.15.157:0/3529455391 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-24T23:34:47.784 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.784+0000 7f11b9be8640 1 --2- 172.21.15.157:0/3529455391 >> v2:172.21.15.157:3300/0 conn(0x7f11b40c6280 0x7f11b4070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:47.784 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.784+0000 7f11b93e7640 1 -- 172.21.15.157:0/3529455391 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1629523139 0 0) 0x7f11b406ebb0 con 0x7f11b40c3f50 2024-04-24T23:34:47.784 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.784+0000 7f11ba3e9640 1 -- 172.21.15.157:0/3529455391 >> v2:172.21.15.157:3300/0 conn(0x7f11b40c6280 msgr2=0x7f11b4070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:47.784 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.784+0000 7f11ba3e9640 1 --2- 172.21.15.157:0/3529455391 >> v2:172.21.15.157:3300/0 conn(0x7f11b40c6280 0x7f11b4070e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:47.785 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.784+0000 7f11ba3e9640 1 -- 172.21.15.157:0/3529455391 >> v1:172.21.15.169:6789/0 conn(0x7f11b40c3f50 legacy=0x7f11b40a7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:47.785 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.784+0000 7f11ba3e9640 1 -- 172.21.15.157:0/3529455391 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11b4153270 con 0x7f11b4150920 2024-04-24T23:34:47.786 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.785+0000 7f11ba3e9640 1 --2- 172.21.15.157:0/3529455391 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11b4150920 0x7f11b4153d50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f11b00097a0 tx=0x7f11b002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d49996e60caa79b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:47.786 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.785+0000 7f11b93e7640 1 -- 172.21.15.157:0/3529455391 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b0037070 con 0x7f11b4150920 2024-04-24T23:34:47.786 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.785+0000 7f11b93e7640 1 -- 172.21.15.157:0/3529455391 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11b0032440 con 0x7f11b4150920 2024-04-24T23:34:47.786 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.785+0000 7f11b93e7640 1 -- 172.21.15.157:0/3529455391 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b0032810 con 0x7f11b4150920 2024-04-24T23:34:47.786 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.786+0000 7f11bc64d640 1 -- 172.21.15.157:0/3529455391 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11b4150920 msgr2=0x7f11b4153d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:47.786 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.786+0000 7f11bc64d640 1 --2- 172.21.15.157:0/3529455391 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11b4150920 0x7f11b4153d50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f11b00097a0 tx=0x7f11b002a210 comp rx=0 tx=0).stop 2024-04-24T23:34:47.786 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.786+0000 7f11bc64d640 1 -- 172.21.15.157:0/3529455391 shutdown_connections 2024-04-24T23:34:47.786 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.786+0000 7f11bc64d640 1 --2- 172.21.15.157:0/3529455391 >> v2:172.21.15.157:3300/0 conn(0x7f11b40c6280 0x7f11b4070e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:47.787 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.786+0000 7f11bc64d640 1 --2- 172.21.15.157:0/3529455391 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11b4150920 0x7f11b4153d50 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:47.787 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.786+0000 7f11bc64d640 1 -- 172.21.15.157:0/3529455391 >> 172.21.15.157:0/3529455391 conn(0x7f11b40c55c0 msgr2=0x7f11b40c59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:47.787 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.786+0000 7f11bc64d640 1 -- 172.21.15.157:0/3529455391 shutdown_connections 2024-04-24T23:34:47.787 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.786+0000 7f11bc64d640 1 -- 172.21.15.157:0/3529455391 wait complete. 2024-04-24T23:34:47.789 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.787+0000 7f11bc64d640 1 Processor -- start 2024-04-24T23:34:47.789 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.787+0000 7f11bc64d640 1 -- start start 2024-04-24T23:34:47.789 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.788+0000 7f11bc64d640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11b40c3f50 0x7f11b41ebe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:47.789 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.788+0000 7f11bc64d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11b40c6280 0x7f11b41ec390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:47.789 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.788+0000 7f11bc64d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11b4150920 0x7f11b414e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:47.790 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.788+0000 7f11bc64d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f11b406ee90 con 0x7f11b40c6280 2024-04-24T23:34:47.790 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.788+0000 7f11bc64d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f11b4150d00 con 0x7f11b4150920 2024-04-24T23:34:47.790 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.788+0000 7f11bc64d640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f11b40a8440 con 0x7f11b40c3f50 2024-04-24T23:34:47.790 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.788+0000 7f11b9be8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11b40c6280 0x7f11b41ec390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:47.790 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.788+0000 7f11b9be8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11b40c6280 0x7f11b41ec390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:41236/0 (socket says 172.21.15.157:41236) 2024-04-24T23:34:47.790 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.788+0000 7f11b9be8640 1 -- 172.21.15.157:0/3388500356 learned_addr learned my addr 172.21.15.157:0/3388500356 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:34:47.790 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.788+0000 7f11ba3e9640 1 --2- 172.21.15.157:0/3388500356 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11b40c3f50 0x7f11b41ebe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:47.790 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.788+0000 7f11babea640 1 --2- 172.21.15.157:0/3388500356 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11b4150920 0x7f11b414e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:47.790 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.788+0000 7f11babea640 1 -- 172.21.15.157:0/3388500356 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11b40c3f50 msgr2=0x7f11b41ebe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:47.790 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.788+0000 7f11babea640 1 --2- 172.21.15.157:0/3388500356 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11b40c3f50 0x7f11b41ebe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:47.790 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.788+0000 7f11babea640 1 -- 172.21.15.157:0/3388500356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11b40c6280 msgr2=0x7f11b41ec390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:47.790 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.788+0000 7f11babea640 1 --2- 172.21.15.157:0/3388500356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11b40c6280 0x7f11b41ec390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:47.790 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.788+0000 7f11babea640 1 -- 172.21.15.157:0/3388500356 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11b41d95b0 con 0x7f11b4150920 2024-04-24T23:34:47.791 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.788+0000 7f11ba3e9640 1 --2- 172.21.15.157:0/3388500356 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11b40c3f50 0x7f11b41ebe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:34:47.791 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.788+0000 7f11b9be8640 1 --2- 172.21.15.157:0/3388500356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11b40c6280 0x7f11b41ec390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:34:47.791 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.788+0000 7f11babea640 1 --2- 172.21.15.157:0/3388500356 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11b4150920 0x7f11b414e900 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f11ac00c780 tx=0x7f11ac00cc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:47.791 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.789+0000 7f11aaffd640 1 -- 172.21.15.157:0/3388500356 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ac018070 con 0x7f11b4150920 2024-04-24T23:34:47.791 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.789+0000 7f11aaffd640 1 -- 172.21.15.157:0/3388500356 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11ac004d60 con 0x7f11b4150920 2024-04-24T23:34:47.791 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.789+0000 7f11bc64d640 1 -- 172.21.15.157:0/3388500356 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11b41eca00 con 0x7f11b4150920 2024-04-24T23:34:47.791 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.789+0000 7f11aaffd640 1 -- 172.21.15.157:0/3388500356 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ac00a880 con 0x7f11b4150920 2024-04-24T23:34:47.791 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.789+0000 7f11bc64d640 1 -- 172.21.15.157:0/3388500356 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11b40adbd0 con 0x7f11b4150920 2024-04-24T23:34:47.791 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.790+0000 7f11bc64d640 1 -- 172.21.15.157:0/3388500356 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11b41d9780 con 0x7f11b4150920 2024-04-24T23:34:47.792 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.790+0000 7f11aaffd640 1 -- 172.21.15.157:0/3388500356 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7f11ac00aa20 con 0x7f11b4150920 2024-04-24T23:34:47.792 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.791+0000 7f11aaffd640 1 --2- 172.21.15.157:0/3388500356 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f119806bda0 0x7f119806e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:47.792 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.791+0000 7f11ba3e9640 1 --2- 172.21.15.157:0/3388500356 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f119806bda0 0x7f119806e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:47.792 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.791+0000 7f11aaffd640 1 -- 172.21.15.157:0/3388500356 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 4471+0+0 (secure 0 0 0) 0x7f11ac089d20 con 0x7f11b4150920 2024-04-24T23:34:47.792 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.791+0000 7f11ba3e9640 1 --2- 172.21.15.157:0/3388500356 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f119806bda0 0x7f119806e260 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f11b0034000 tx=0x7f11b002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:47.795 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.795+0000 7f11aaffd640 1 -- 172.21.15.157:0/3388500356 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f11ac0585b0 con 0x7f11b4150920 2024-04-24T23:34:47.973 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.974+0000 7f11bc64d640 1 -- 172.21.15.157:0/3388500356 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f11b4075530 con 0x7f11b4150920 2024-04-24T23:34:47.974 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.975+0000 7f11aaffd640 1 -- 172.21.15.157:0/3388500356 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) ==== 74+0+10956 (secure 0 0 0) 0x7f11ac05c8b0 con 0x7f11b4150920 2024-04-24T23:34:47.975 INFO:teuthology.misc.health.smithi157.stdout: 2024-04-24T23:34:47.975 INFO:teuthology.misc.health.smithi157.stdout:{"epoch":14,"fsid":"223685fb-5b64-46a5-9857-3791c9c07df2","created":"2024-04-24T23:34:37.504202+0000","modified":"2024-04-24T23:34:47.584963+0000","last_up_change":"2024-04-24T23:34:47.584963+0000","last_in_change":"2024-04-24T23:34:39.632954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"95d14ac8-fccf-4261-be8e-edafa746f629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6809","nonce":4003822160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6811","nonce":4003822160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6815","nonce":4003822160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6813","nonce":4003822160}]},"public_addr":"172.21.15.157:6809/4003822160","cluster_addr":"172.21.15.157:6811/4003822160","heartbeat_back_addr":"172.21.15.157:6815/4003822160","heartbeat_front_addr":"172.21.15.157:6813/4003822160","state":["exists","up"]},{"osd":1,"uuid":"7be79da9-8eec-4990-9178-71d602e39d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6817","nonce":1557575576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6819","nonce":1557575576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6823","nonce":1557575576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6821","nonce":1557575576}]},"public_addr":"172.21.15.157:6817/1557575576","cluster_addr":"172.21.15.157:6819/1557575576","heartbeat_back_addr":"172.21.15.157:6823/1557575576","heartbeat_front_addr":"172.21.15.157:6821/1557575576","state":["exists","up"]},{"osd":2,"uuid":"0127ed00-b7c3-466d-bfb0-01b0fd315f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6825","nonce":2793948092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6827","nonce":2793948092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6831","nonce":2793948092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6829","nonce":2793948092}]},"public_addr":"172.21.15.157:6825/2793948092","cluster_addr":"172.21.15.157:6827/2793948092","heartbeat_back_addr":"172.21.15.157:6831/2793948092","heartbeat_front_addr":"172.21.15.157:6829/2793948092","state":["exists","up"]},{"osd":3,"uuid":"f2331acc-aa37-433e-868e-3561b6039562","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6801","nonce":422758549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6803","nonce":422758549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6807","nonce":422758549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6805","nonce":422758549}]},"public_addr":"172.21.15.157:6801/422758549","cluster_addr":"172.21.15.157:6803/422758549","heartbeat_back_addr":"172.21.15.157:6807/422758549","heartbeat_front_addr":"172.21.15.157:6805/422758549","state":["exists","up"]},{"osd":4,"uuid":"1998854b-7398-43f5-ab7d-79fa0debcf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6801","nonce":321910099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6803","nonce":321910099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6807","nonce":321910099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6805","nonce":321910099}]},"public_addr":"172.21.15.169:6801/321910099","cluster_addr":"172.21.15.169:6803/321910099","heartbeat_back_addr":"172.21.15.169:6807/321910099","heartbeat_front_addr":"172.21.15.169:6805/321910099","state":["exists","up"]},{"osd":5,"uuid":"c37153ff-9b2b-48b9-a684-2c82b6f39f15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6825","nonce":243321711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6827","nonce":243321711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6831","nonce":243321711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6829","nonce":243321711}]},"public_addr":"172.21.15.169:6825/243321711","cluster_addr":"172.21.15.169:6827/243321711","heartbeat_back_addr":"172.21.15.169:6831/243321711","heartbeat_front_addr":"172.21.15.169:6829/243321711","state":["exists","up"]},{"osd":6,"uuid":"8db79e1c-f36b-49cb-8d8a-a20c2aec57c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6809","nonce":3441420785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6811","nonce":3441420785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6815","nonce":3441420785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6813","nonce":3441420785}]},"public_addr":"172.21.15.169:6809/3441420785","cluster_addr":"172.21.15.169:6811/3441420785","heartbeat_back_addr":"172.21.15.169:6815/3441420785","heartbeat_front_addr":"172.21.15.169:6813/3441420785","state":["exists","up"]},{"osd":7,"uuid":"928586e9-296c-4b8b-9b27-42f9f155dbdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6817","nonce":1663179020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6819","nonce":1663179020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6823","nonce":1663179020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6821","nonce":1663179020}]},"public_addr":"172.21.15.169:6817/1663179020","cluster_addr":"172.21.15.169:6819/1663179020","heartbeat_back_addr":"172.21.15.169:6823/1663179020","heartbeat_front_addr":"172.21.15.169:6821/1663179020","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.727706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:34:47.977 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.977+0000 7f11bc64d640 1 -- 172.21.15.157:0/3388500356 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f119806bda0 msgr2=0x7f119806e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:47.977 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.977+0000 7f11bc64d640 1 --2- 172.21.15.157:0/3388500356 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f119806bda0 0x7f119806e260 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f11b0034000 tx=0x7f11b002e000 comp rx=0 tx=0).stop 2024-04-24T23:34:47.977 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.977+0000 7f11bc64d640 1 -- 172.21.15.157:0/3388500356 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11b4150920 msgr2=0x7f11b414e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:47.977 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.977+0000 7f11bc64d640 1 --2- 172.21.15.157:0/3388500356 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11b4150920 0x7f11b414e900 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f11ac00c780 tx=0x7f11ac00cc40 comp rx=0 tx=0).stop 2024-04-24T23:34:47.977 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.977+0000 7f11bc64d640 1 -- 172.21.15.157:0/3388500356 shutdown_connections 2024-04-24T23:34:47.977 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.977+0000 7f11bc64d640 1 --2- 172.21.15.157:0/3388500356 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f119806bda0 0x7f119806e260 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:47.977 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.977+0000 7f11bc64d640 1 --2- 172.21.15.157:0/3388500356 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11b4150920 0x7f11b414e900 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:47.977 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.977+0000 7f11bc64d640 1 --2- 172.21.15.157:0/3388500356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11b40c6280 0x7f11b41ec390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:47.977 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.977+0000 7f11bc64d640 1 --2- 172.21.15.157:0/3388500356 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11b40c3f50 0x7f11b41ebe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:47.977 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.977+0000 7f11bc64d640 1 -- 172.21.15.157:0/3388500356 >> 172.21.15.157:0/3388500356 conn(0x7f11b40c55c0 msgr2=0x7f11b40a9820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:47.977 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.977+0000 7f11bc64d640 1 -- 172.21.15.157:0/3388500356 shutdown_connections 2024-04-24T23:34:47.978 INFO:teuthology.misc.health.smithi157.stderr:2024-04-24T23:34:47.977+0000 7f11bc64d640 1 -- 172.21.15.157:0/3388500356 wait complete. 2024-04-24T23:34:47.986 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-24T23:34:47.986 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-24T23:34:47.987 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-24T23:34:47.987 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mds -f --cluster ceph -i a 2024-04-24T23:34:47.989 INFO:tasks.ceph.mds.a:Started 2024-04-24T23:34:47.989 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-24T23:34:47.989 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mds -f --cluster ceph -i c 2024-04-24T23:34:47.991 INFO:tasks.ceph.mds.c:Started 2024-04-24T23:34:47.991 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-24T23:34:47.991 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mds -f --cluster ceph -i b 2024-04-24T23:34:47.994 INFO:tasks.ceph.mds.b:Started 2024-04-24T23:34:47.994 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-24T23:34:47.994 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mds -f --cluster ceph -i d 2024-04-24T23:34:47.996 INFO:tasks.ceph.mds.d:Started 2024-04-24T23:34:47.996 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-24T23:34:47.997 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:34:48.063 INFO:tasks.ceph.mds.c.smithi157.stdout:starting mds.c at 2024-04-24T23:34:48.063 INFO:tasks.ceph.mds.a.smithi157.stdout:starting mds.a at 2024-04-24T23:34:48.073 INFO:tasks.ceph.mds.d.smithi169.stdout:starting mds.d at 2024-04-24T23:34:48.073 INFO:tasks.ceph.mds.b.smithi169.stdout:starting mds.b at 2024-04-24T23:34:48.098 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.098+0000 7fe0e756a640 1 Processor -- start 2024-04-24T23:34:48.098 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.098+0000 7fe0e756a640 1 -- start start 2024-04-24T23:34:48.101 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.099+0000 7fe0e756a640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0e00be360 0x7fe0e0155350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:48.101 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.099+0000 7fe0e756a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0e00c8bb0 con 0x7fe0e0155890 2024-04-24T23:34:48.101 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.099+0000 7fe0e756a640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0e0159480 con 0x7fe0e0152b40 2024-04-24T23:34:48.101 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.099+0000 7fe0e756a640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe0e00c8e90 con 0x7fe0e00be360 2024-04-24T23:34:48.101 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.099+0000 7fe0e5b07640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fe0e0155890 0x7fe0e015fcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:60600/0 (socket says 172.21.15.157:60600) 2024-04-24T23:34:48.101 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.099+0000 7fe0e5b07640 1 -- 172.21.15.157:0/2463367854 learned_addr learned my addr 172.21.15.157:0/2463367854 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-24T23:34:48.101 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.099+0000 7fe0e4b05640 1 --2- 172.21.15.157:0/2463367854 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0e00be360 0x7fe0e0155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:48.101 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.100+0000 7fe0d7fff640 1 -- 172.21.15.157:0/2463367854 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2793990656 0 0) 0x7fe0e00c8bb0 con 0x7fe0e0155890 2024-04-24T23:34:48.101 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.100+0000 7fe0d7fff640 1 -- 172.21.15.157:0/2463367854 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe0cc003520 con 0x7fe0e0155890 2024-04-24T23:34:48.101 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.100+0000 7fe0d7fff640 1 -- 172.21.15.157:0/2463367854 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1949558054 0 0) 0x7fe0e0159480 con 0x7fe0e0152b40 2024-04-24T23:34:48.101 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.100+0000 7fe0d7fff640 1 -- 172.21.15.157:0/2463367854 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe0e00c8bb0 con 0x7fe0e0152b40 2024-04-24T23:34:48.101 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.100+0000 7fe0e4b05640 1 -- 172.21.15.157:0/2463367854 >> v1:172.21.15.169:6789/0 conn(0x7fe0e0152b40 legacy=0x7fe0e00c5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:48.102 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.100+0000 7fe0e4b05640 1 -- 172.21.15.157:0/2463367854 >> v1:172.21.15.157:6789/0 conn(0x7fe0e0155890 legacy=0x7fe0e015fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:48.102 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.100+0000 7fe0e4b05640 1 -- 172.21.15.157:0/2463367854 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0e0159760 con 0x7fe0e00be360 2024-04-24T23:34:48.102 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.100+0000 7fe0e4b05640 1 --2- 172.21.15.157:0/2463367854 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0e00be360 0x7fe0e0155350 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe0d00097d0 tx=0x7fe0d002a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4e104064e1cf20c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:48.102 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.100+0000 7fe0d7fff640 1 -- 172.21.15.157:0/2463367854 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d0037070 con 0x7fe0e00be360 2024-04-24T23:34:48.102 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.100+0000 7fe0d7fff640 1 -- 172.21.15.157:0/2463367854 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0d002ae50 con 0x7fe0e00be360 2024-04-24T23:34:48.102 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.100+0000 7fe0d7fff640 1 -- 172.21.15.157:0/2463367854 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d0032660 con 0x7fe0e00be360 2024-04-24T23:34:48.102 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.101+0000 7fe0e756a640 1 -- 172.21.15.157:0/2463367854 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0e00be360 msgr2=0x7fe0e0155350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:48.102 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.101+0000 7fe0e756a640 1 --2- 172.21.15.157:0/2463367854 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0e00be360 0x7fe0e0155350 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe0d00097d0 tx=0x7fe0d002a100 comp rx=0 tx=0).stop 2024-04-24T23:34:48.102 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.101+0000 7fe0e756a640 1 -- 172.21.15.157:0/2463367854 shutdown_connections 2024-04-24T23:34:48.102 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.101+0000 7fe0e756a640 1 --2- 172.21.15.157:0/2463367854 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0e00be360 0x7fe0e0155350 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:48.102 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.101+0000 7fe0e756a640 1 -- 172.21.15.157:0/2463367854 >> 172.21.15.157:0/2463367854 conn(0x7fe0e00672c0 msgr2=0x7fe0e00676d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:48.102 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.101+0000 7fe0e756a640 1 -- 172.21.15.157:0/2463367854 shutdown_connections 2024-04-24T23:34:48.102 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.101+0000 7fe0e756a640 1 -- 172.21.15.157:0/2463367854 wait complete. 2024-04-24T23:34:48.105 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.102+0000 7fe0e756a640 1 Processor -- start 2024-04-24T23:34:48.105 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.102+0000 7fe0e756a640 1 -- start start 2024-04-24T23:34:48.105 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.103+0000 7fe0e756a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0e00be360 0x7fe0e00ac470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:48.105 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.103+0000 7fe0e756a640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0e0152b40 0x7fe0e014ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:48.105 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.103+0000 7fe0e756a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0e0155890 0x7fe0e014bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:48.105 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.103+0000 7fe0e756a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e0155d40 con 0x7fe0e00be360 2024-04-24T23:34:48.105 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.103+0000 7fe0e756a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e00cb210 con 0x7fe0e0155890 2024-04-24T23:34:48.105 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.103+0000 7fe0e756a640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe0e00caa00 con 0x7fe0e0152b40 2024-04-24T23:34:48.106 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.103+0000 7fe0e5306640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0e00be360 0x7fe0e00ac470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:48.106 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.103+0000 7fe0e5306640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0e00be360 0x7fe0e00ac470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:41292/0 (socket says 172.21.15.157:41292) 2024-04-24T23:34:48.106 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.103+0000 7fe0e5306640 1 -- 172.21.15.157:0/967450734 learned_addr learned my addr 172.21.15.157:0/967450734 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:34:48.106 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.103+0000 7fe0e4b05640 1 --2- 172.21.15.157:0/967450734 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0e0152b40 0x7fe0e014ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:48.106 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.103+0000 7fe0e5b07640 1 --2- 172.21.15.157:0/967450734 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0e0155890 0x7fe0e014bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:48.106 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.103+0000 7fe0e5306640 1 -- 172.21.15.157:0/967450734 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0e0152b40 msgr2=0x7fe0e014ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:48.106 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.103+0000 7fe0e5306640 1 --2- 172.21.15.157:0/967450734 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0e0152b40 0x7fe0e014ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:48.106 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.103+0000 7fe0e5306640 1 -- 172.21.15.157:0/967450734 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0e0155890 msgr2=0x7fe0e014bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:48.106 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.103+0000 7fe0e5306640 1 --2- 172.21.15.157:0/967450734 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0e0155890 0x7fe0e014bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:48.106 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.103+0000 7fe0e5306640 1 -- 172.21.15.157:0/967450734 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0e0158ad0 con 0x7fe0e00be360 2024-04-24T23:34:48.106 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.103+0000 7fe0e5306640 1 --2- 172.21.15.157:0/967450734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0e00be360 0x7fe0e00ac470 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc00c7a0 tx=0x7fe0dc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:48.106 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.103+0000 7fe0d5ffb640 1 -- 172.21.15.157:0/967450734 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc003b70 con 0x7fe0e00be360 2024-04-24T23:34:48.106 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.104+0000 7fe0d5ffb640 1 -- 172.21.15.157:0/967450734 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0dc003d10 con 0x7fe0e00be360 2024-04-24T23:34:48.107 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.104+0000 7fe0e756a640 1 -- 172.21.15.157:0/967450734 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0e00aa9a0 con 0x7fe0e00be360 2024-04-24T23:34:48.107 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.104+0000 7fe0d5ffb640 1 -- 172.21.15.157:0/967450734 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc00aa30 con 0x7fe0e00be360 2024-04-24T23:34:48.107 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.104+0000 7fe0e756a640 1 -- 172.21.15.157:0/967450734 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0e00aa6c0 con 0x7fe0e00be360 2024-04-24T23:34:48.107 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.105+0000 7fe0e756a640 1 -- 172.21.15.157:0/967450734 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0e00681c0 con 0x7fe0e00be360 2024-04-24T23:34:48.107 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.105+0000 7fe0d5ffb640 1 -- 172.21.15.157:0/967450734 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7fe0dc00abd0 con 0x7fe0e00be360 2024-04-24T23:34:48.107 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.105+0000 7fe0d5ffb640 1 --2- 172.21.15.157:0/967450734 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fe0c406bda0 0x7fe0c406e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:48.107 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.105+0000 7fe0d5ffb640 1 -- 172.21.15.157:0/967450734 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 4471+0+0 (secure 0 0 0) 0x7fe0dc08a720 con 0x7fe0e00be360 2024-04-24T23:34:48.107 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.106+0000 7fe0e4b05640 1 --2- 172.21.15.157:0/967450734 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fe0c406bda0 0x7fe0c406e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:48.108 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.106+0000 7fe0e4b05640 1 --2- 172.21.15.157:0/967450734 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fe0c406bda0 0x7fe0c406e260 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe0d0034000 tx=0x7fe0d002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:48.109 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.110+0000 7fe0d5ffb640 1 -- 172.21.15.157:0/967450734 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe0dc059710 con 0x7fe0e00be360 2024-04-24T23:34:48.285 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.286+0000 7fe0e756a640 1 -- 172.21.15.157:0/967450734 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe0e006d3f0 con 0x7fe0e00be360 2024-04-24T23:34:48.286 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.287+0000 7fe0d5ffb640 1 -- 172.21.15.157:0/967450734 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) ==== 74+0+10956 (secure 0 0 0) 0x7fe0dc059010 con 0x7fe0e00be360 2024-04-24T23:34:48.286 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:34:48.287 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":14,"fsid":"223685fb-5b64-46a5-9857-3791c9c07df2","created":"2024-04-24T23:34:37.504202+0000","modified":"2024-04-24T23:34:47.584963+0000","last_up_change":"2024-04-24T23:34:47.584963+0000","last_in_change":"2024-04-24T23:34:39.632954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"95d14ac8-fccf-4261-be8e-edafa746f629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6809","nonce":4003822160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6811","nonce":4003822160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6815","nonce":4003822160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6813","nonce":4003822160}]},"public_addr":"172.21.15.157:6809/4003822160","cluster_addr":"172.21.15.157:6811/4003822160","heartbeat_back_addr":"172.21.15.157:6815/4003822160","heartbeat_front_addr":"172.21.15.157:6813/4003822160","state":["exists","up"]},{"osd":1,"uuid":"7be79da9-8eec-4990-9178-71d602e39d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6817","nonce":1557575576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6819","nonce":1557575576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6823","nonce":1557575576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6821","nonce":1557575576}]},"public_addr":"172.21.15.157:6817/1557575576","cluster_addr":"172.21.15.157:6819/1557575576","heartbeat_back_addr":"172.21.15.157:6823/1557575576","heartbeat_front_addr":"172.21.15.157:6821/1557575576","state":["exists","up"]},{"osd":2,"uuid":"0127ed00-b7c3-466d-bfb0-01b0fd315f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6825","nonce":2793948092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6827","nonce":2793948092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6831","nonce":2793948092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6829","nonce":2793948092}]},"public_addr":"172.21.15.157:6825/2793948092","cluster_addr":"172.21.15.157:6827/2793948092","heartbeat_back_addr":"172.21.15.157:6831/2793948092","heartbeat_front_addr":"172.21.15.157:6829/2793948092","state":["exists","up"]},{"osd":3,"uuid":"f2331acc-aa37-433e-868e-3561b6039562","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6801","nonce":422758549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6803","nonce":422758549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6807","nonce":422758549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6805","nonce":422758549}]},"public_addr":"172.21.15.157:6801/422758549","cluster_addr":"172.21.15.157:6803/422758549","heartbeat_back_addr":"172.21.15.157:6807/422758549","heartbeat_front_addr":"172.21.15.157:6805/422758549","state":["exists","up"]},{"osd":4,"uuid":"1998854b-7398-43f5-ab7d-79fa0debcf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6801","nonce":321910099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6803","nonce":321910099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6807","nonce":321910099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6805","nonce":321910099}]},"public_addr":"172.21.15.169:6801/321910099","cluster_addr":"172.21.15.169:6803/321910099","heartbeat_back_addr":"172.21.15.169:6807/321910099","heartbeat_front_addr":"172.21.15.169:6805/321910099","state":["exists","up"]},{"osd":5,"uuid":"c37153ff-9b2b-48b9-a684-2c82b6f39f15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6825","nonce":243321711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6827","nonce":243321711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6831","nonce":243321711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6829","nonce":243321711}]},"public_addr":"172.21.15.169:6825/243321711","cluster_addr":"172.21.15.169:6827/243321711","heartbeat_back_addr":"172.21.15.169:6831/243321711","heartbeat_front_addr":"172.21.15.169:6829/243321711","state":["exists","up"]},{"osd":6,"uuid":"8db79e1c-f36b-49cb-8d8a-a20c2aec57c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6809","nonce":3441420785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6811","nonce":3441420785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6815","nonce":3441420785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6813","nonce":3441420785}]},"public_addr":"172.21.15.169:6809/3441420785","cluster_addr":"172.21.15.169:6811/3441420785","heartbeat_back_addr":"172.21.15.169:6815/3441420785","heartbeat_front_addr":"172.21.15.169:6813/3441420785","state":["exists","up"]},{"osd":7,"uuid":"928586e9-296c-4b8b-9b27-42f9f155dbdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6817","nonce":1663179020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6819","nonce":1663179020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6823","nonce":1663179020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6821","nonce":1663179020}]},"public_addr":"172.21.15.169:6817/1663179020","cluster_addr":"172.21.15.169:6819/1663179020","heartbeat_back_addr":"172.21.15.169:6823/1663179020","heartbeat_front_addr":"172.21.15.169:6821/1663179020","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.727706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:34:48.288 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.289+0000 7fe0e756a640 1 -- 172.21.15.157:0/967450734 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fe0c406bda0 msgr2=0x7fe0c406e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:48.289 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.289+0000 7fe0e756a640 1 --2- 172.21.15.157:0/967450734 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fe0c406bda0 0x7fe0c406e260 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe0d0034000 tx=0x7fe0d002e000 comp rx=0 tx=0).stop 2024-04-24T23:34:48.289 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.289+0000 7fe0e756a640 1 -- 172.21.15.157:0/967450734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0e00be360 msgr2=0x7fe0e00ac470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:48.289 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.289+0000 7fe0e756a640 1 --2- 172.21.15.157:0/967450734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0e00be360 0x7fe0e00ac470 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc00c7a0 tx=0x7fe0dc00cc60 comp rx=0 tx=0).stop 2024-04-24T23:34:48.289 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.289+0000 7fe0e756a640 1 -- 172.21.15.157:0/967450734 shutdown_connections 2024-04-24T23:34:48.289 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.289+0000 7fe0e756a640 1 --2- 172.21.15.157:0/967450734 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fe0c406bda0 0x7fe0c406e260 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:48.289 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.289+0000 7fe0e756a640 1 --2- 172.21.15.157:0/967450734 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0e0155890 0x7fe0e014bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:48.289 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.289+0000 7fe0e756a640 1 --2- 172.21.15.157:0/967450734 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0e0152b40 0x7fe0e014ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:48.289 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.289+0000 7fe0e756a640 1 --2- 172.21.15.157:0/967450734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0e00be360 0x7fe0e00ac470 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:48.289 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.289+0000 7fe0e756a640 1 -- 172.21.15.157:0/967450734 >> 172.21.15.157:0/967450734 conn(0x7fe0e00672c0 msgr2=0x7fe0e01531c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:48.289 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.289+0000 7fe0e756a640 1 -- 172.21.15.157:0/967450734 shutdown_connections 2024-04-24T23:34:48.289 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.289+0000 7fe0e756a640 1 -- 172.21.15.157:0/967450734 wait complete. 2024-04-24T23:34:48.299 INFO:tasks.cephfs.filesystem.ceph_manager:[] 2024-04-24T23:34:48.299 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:34:48.397 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.397+0000 7f0f6f012640 1 Processor -- start 2024-04-24T23:34:48.397 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.397+0000 7f0f6f012640 1 -- start start 2024-04-24T23:34:48.399 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.398+0000 7f0f6f012640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0f681529a0 0x7f0f680c36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:48.399 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.398+0000 7f0f6f012640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0f68158fe0 0x7f0f680c5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:48.399 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.398+0000 7f0f6f012640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f680c3c30 con 0x7f0f680c5630 2024-04-24T23:34:48.399 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.398+0000 7f0f6f012640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f0f680c6240 con 0x7f0f68158fe0 2024-04-24T23:34:48.399 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.398+0000 7f0f6f012640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0f68152d80 con 0x7f0f681529a0 2024-04-24T23:34:48.399 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.398+0000 7f0f6d5af640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0f68158fe0 0x7f0f680c5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:48.399 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.398+0000 7f0f67fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f0f680c5630 0x7f0f681588d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:60614/0 (socket says 172.21.15.157:60614) 2024-04-24T23:34:48.399 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.398+0000 7f0f6d5af640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0f68158fe0 0x7f0f680c5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:41302/0 (socket says 172.21.15.157:41302) 2024-04-24T23:34:48.399 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.398+0000 7f0f67fff640 1 -- 172.21.15.157:0/1416443165 learned_addr learned my addr 172.21.15.157:0/1416443165 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-24T23:34:48.399 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.398+0000 7f0f6cdae640 1 --2- 172.21.15.157:0/1416443165 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0f681529a0 0x7f0f680c36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:48.400 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.398+0000 7f0f6d5af640 1 -- 172.21.15.157:0/1416443165 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0f681529a0 msgr2=0x7f0f680c36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:48.400 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.398+0000 7f0f6d5af640 1 --2- 172.21.15.157:0/1416443165 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0f681529a0 0x7f0f680c36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:48.400 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.398+0000 7f0f6d5af640 1 -- 172.21.15.157:0/1416443165 >> v1:172.21.15.157:6789/0 conn(0x7f0f680c5630 legacy=0x7f0f681588d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:48.400 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.398+0000 7f0f677fe640 1 -- 172.21.15.157:0/1416443165 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1650645034 0 0) 0x7f0f680c3c30 con 0x7f0f680c5630 2024-04-24T23:34:48.400 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.398+0000 7f0f6d5af640 1 -- 172.21.15.157:0/1416443165 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f680c3f00 con 0x7f0f68158fe0 2024-04-24T23:34:48.400 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.399+0000 7f0f6d5af640 1 --2- 172.21.15.157:0/1416443165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f68158fe0 0x7f0f680c5a90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0f60009d90 tx=0x7f0f6002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af83d8a633fc5335 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:48.400 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.399+0000 7f0f677fe640 1 -- 172.21.15.157:0/1416443165 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f60031600 con 0x7f0f68158fe0 2024-04-24T23:34:48.400 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.399+0000 7f0f677fe640 1 -- 172.21.15.157:0/1416443165 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f600317a0 con 0x7f0f68158fe0 2024-04-24T23:34:48.400 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.399+0000 7f0f677fe640 1 -- 172.21.15.157:0/1416443165 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f60031b70 con 0x7f0f68158fe0 2024-04-24T23:34:48.403 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.400+0000 7f0f6f012640 1 -- 172.21.15.157:0/1416443165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f68158fe0 msgr2=0x7f0f680c5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:48.403 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.400+0000 7f0f6f012640 1 --2- 172.21.15.157:0/1416443165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f68158fe0 0x7f0f680c5a90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0f60009d90 tx=0x7f0f6002a5d0 comp rx=0 tx=0).stop 2024-04-24T23:34:48.403 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.400+0000 7f0f6f012640 1 -- 172.21.15.157:0/1416443165 shutdown_connections 2024-04-24T23:34:48.403 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.400+0000 7f0f6f012640 1 --2- 172.21.15.157:0/1416443165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f68158fe0 0x7f0f680c5a90 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:48.403 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.400+0000 7f0f6f012640 1 --2- 172.21.15.157:0/1416443165 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0f681529a0 0x7f0f680c36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:48.403 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.400+0000 7f0f6f012640 1 -- 172.21.15.157:0/1416443165 >> 172.21.15.157:0/1416443165 conn(0x7f0f680c8ba0 msgr2=0x7f0f680c8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:48.404 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.400+0000 7f0f6f012640 1 -- 172.21.15.157:0/1416443165 shutdown_connections 2024-04-24T23:34:48.404 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.400+0000 7f0f6f012640 1 -- 172.21.15.157:0/1416443165 wait complete. 2024-04-24T23:34:48.404 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.401+0000 7f0f6f012640 1 Processor -- start 2024-04-24T23:34:48.404 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.401+0000 7f0f6f012640 1 -- start start 2024-04-24T23:34:48.404 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.402+0000 7f0f6f012640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0f680c5630 0x7f0f680a88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:48.404 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.402+0000 7f0f6f012640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f681529a0 0x7f0f680ad6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:48.404 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.402+0000 7f0f6f012640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f68158fe0 0x7f0f681512b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:48.404 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.402+0000 7f0f6f012640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0f68155c70 con 0x7f0f68158fe0 2024-04-24T23:34:48.404 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.402+0000 7f0f6f012640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0f68155990 con 0x7f0f681529a0 2024-04-24T23:34:48.404 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.402+0000 7f0f6f012640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0f68155360 con 0x7f0f680c5630 2024-04-24T23:34:48.404 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.402+0000 7f0f6d5af640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f68158fe0 0x7f0f681512b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:48.404 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.402+0000 7f0f6d5af640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f68158fe0 0x7f0f681512b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:41314/0 (socket says 172.21.15.157:41314) 2024-04-24T23:34:48.404 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.402+0000 7f0f6d5af640 1 -- 172.21.15.157:0/3279520374 learned_addr learned my addr 172.21.15.157:0/3279520374 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:34:48.404 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.402+0000 7f0f6cdae640 1 --2- 172.21.15.157:0/3279520374 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0f680c5630 0x7f0f680a88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:48.405 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.402+0000 7f0f67fff640 1 --2- 172.21.15.157:0/3279520374 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f681529a0 0x7f0f680ad6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:48.405 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.402+0000 7f0f6d5af640 1 -- 172.21.15.157:0/3279520374 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0f680c5630 msgr2=0x7f0f680a88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:48.405 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.402+0000 7f0f6d5af640 1 --2- 172.21.15.157:0/3279520374 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0f680c5630 0x7f0f680a88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:48.405 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.402+0000 7f0f6d5af640 1 -- 172.21.15.157:0/3279520374 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f681529a0 msgr2=0x7f0f680ad6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:48.405 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.402+0000 7f0f6d5af640 1 --2- 172.21.15.157:0/3279520374 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f681529a0 0x7f0f680ad6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:48.405 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.402+0000 7f0f6d5af640 1 -- 172.21.15.157:0/3279520374 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f680ac820 con 0x7f0f68158fe0 2024-04-24T23:34:48.405 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.402+0000 7f0f6d5af640 1 --2- 172.21.15.157:0/3279520374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f68158fe0 0x7f0f681512b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0f6002abc0 tx=0x7f0f60004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:48.405 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.403+0000 7f0f657fa640 1 -- 172.21.15.157:0/3279520374 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f60040070 con 0x7f0f68158fe0 2024-04-24T23:34:48.405 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.403+0000 7f0f657fa640 1 -- 172.21.15.157:0/3279520374 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f6003c830 con 0x7f0f68158fe0 2024-04-24T23:34:48.405 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.403+0000 7f0f6f012640 1 -- 172.21.15.157:0/3279520374 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f681eaac0 con 0x7f0f68158fe0 2024-04-24T23:34:48.405 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.403+0000 7f0f657fa640 1 -- 172.21.15.157:0/3279520374 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6003cbe0 con 0x7f0f68158fe0 2024-04-24T23:34:48.405 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.403+0000 7f0f6f012640 1 -- 172.21.15.157:0/3279520374 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f681eb080 con 0x7f0f68158fe0 2024-04-24T23:34:48.405 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.404+0000 7f0f6f012640 1 -- 172.21.15.157:0/3279520374 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f680681c0 con 0x7f0f68158fe0 2024-04-24T23:34:48.406 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.404+0000 7f0f657fa640 1 -- 172.21.15.157:0/3279520374 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7f0f60039040 con 0x7f0f68158fe0 2024-04-24T23:34:48.406 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.404+0000 7f0f657fa640 1 --2- 172.21.15.157:0/3279520374 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f0f5406bda0 0x7f0f5406e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:48.406 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.405+0000 7f0f657fa640 1 -- 172.21.15.157:0/3279520374 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 4471+0+0 (secure 0 0 0) 0x7f0f600abcc0 con 0x7f0f68158fe0 2024-04-24T23:34:48.407 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.405+0000 7f0f6cdae640 1 --2- 172.21.15.157:0/3279520374 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f0f5406bda0 0x7f0f5406e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:48.407 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.405+0000 7f0f6cdae640 1 --2- 172.21.15.157:0/3279520374 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f0f5406bda0 0x7f0f5406e260 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0f58009b40 tx=0x7f0f58009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:48.408 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.409+0000 7f0f657fa640 1 -- 172.21.15.157:0/3279520374 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0f6007a700 con 0x7f0f68158fe0 2024-04-24T23:34:48.622 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.622+0000 7f0f6f012640 1 -- 172.21.15.157:0/3279520374 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0f680c63c0 con 0x7f0f68158fe0 2024-04-24T23:34:48.637 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.623+0000 7f0f657fa640 1 -- 172.21.15.157:0/3279520374 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1395 (secure 0 0 0) 0x7f0f6007ea00 con 0x7f0f68158fe0 2024-04-24T23:34:48.638 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:34:48.638 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1398599086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":1398599086},{"type":"v1","addr":"172.21.15.157:6835","nonce":1398599086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-24T23:34:48.639 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 2 2024-04-24T23:34:48.639 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.625+0000 7f0f6f012640 1 -- 172.21.15.157:0/3279520374 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f0f5406bda0 msgr2=0x7f0f5406e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:48.639 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.625+0000 7f0f6f012640 1 --2- 172.21.15.157:0/3279520374 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f0f5406bda0 0x7f0f5406e260 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0f58009b40 tx=0x7f0f58009190 comp rx=0 tx=0).stop 2024-04-24T23:34:48.639 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.625+0000 7f0f6f012640 1 -- 172.21.15.157:0/3279520374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f68158fe0 msgr2=0x7f0f681512b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:48.640 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.625+0000 7f0f6f012640 1 --2- 172.21.15.157:0/3279520374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f68158fe0 0x7f0f681512b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0f6002abc0 tx=0x7f0f60004a40 comp rx=0 tx=0).stop 2024-04-24T23:34:48.640 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.625+0000 7f0f6f012640 1 -- 172.21.15.157:0/3279520374 shutdown_connections 2024-04-24T23:34:48.640 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.625+0000 7f0f6f012640 1 --2- 172.21.15.157:0/3279520374 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f0f5406bda0 0x7f0f5406e260 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:48.640 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.625+0000 7f0f6f012640 1 --2- 172.21.15.157:0/3279520374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f68158fe0 0x7f0f681512b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:48.640 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.625+0000 7f0f6f012640 1 --2- 172.21.15.157:0/3279520374 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f681529a0 0x7f0f680ad6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:48.640 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.625+0000 7f0f6f012640 1 --2- 172.21.15.157:0/3279520374 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0f680c5630 0x7f0f680a88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:48.640 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.625+0000 7f0f6f012640 1 -- 172.21.15.157:0/3279520374 >> 172.21.15.157:0/3279520374 conn(0x7f0f680c8ba0 msgr2=0x7f0f68154b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:48.640 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.625+0000 7f0f6f012640 1 -- 172.21.15.157:0/3279520374 shutdown_connections 2024-04-24T23:34:48.640 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:48.625+0000 7f0f6f012640 1 -- 172.21.15.157:0/3279520374 wait complete. 2024-04-24T23:34:50.641 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:34:50.740 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.740+0000 7fb72a4d6640 1 Processor -- start 2024-04-24T23:34:50.740 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.740+0000 7fb72a4d6640 1 -- start start 2024-04-24T23:34:50.741 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.741+0000 7fb72a4d6640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb724152b40 0x7fb7240c5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:50.741 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.741+0000 7fb72a4d6640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb7240c5b70 0x7fb724155a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:50.741 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.741+0000 7fb72a4d6640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7240c8bb0 con 0x7fb7240be360 2024-04-24T23:34:50.742 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.741+0000 7fb72a4d6640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fb7240c8e90 con 0x7fb7240c5b70 2024-04-24T23:34:50.742 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.741+0000 7fb72a4d6640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb724159480 con 0x7fb724152b40 2024-04-24T23:34:50.742 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.741+0000 7fb728a73640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb7240c5b70 0x7fb724155a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:50.742 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.741+0000 7fb728a73640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb7240c5b70 0x7fb724155a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.157:33360/0 (socket says 172.21.15.157:33360) 2024-04-24T23:34:50.742 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.741+0000 7fb728a73640 1 -- 172.21.15.157:0/2551471069 learned_addr learned my addr 172.21.15.157:0/2551471069 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:34:50.742 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.741+0000 7fb7237fe640 1 --1- 172.21.15.157:0/2551471069 >> v1:172.21.15.169:6789/0 conn(0x7fb7240be360 0x7fb724155350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.157:51862/0 (socket says 172.21.15.157:51862) 2024-04-24T23:34:50.742 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.741+0000 7fb723fff640 1 --2- 172.21.15.157:0/2551471069 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb724152b40 0x7fb7240c5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:50.742 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.742+0000 7fb728a73640 1 -- 172.21.15.157:0/2551471069 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb724152b40 msgr2=0x7fb7240c5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:50.742 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.742+0000 7fb728a73640 1 --2- 172.21.15.157:0/2551471069 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb724152b40 0x7fb7240c5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:50.742 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.742+0000 7fb728a73640 1 -- 172.21.15.157:0/2551471069 >> v1:172.21.15.169:6789/0 conn(0x7fb7240be360 legacy=0x7fb724155350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:50.742 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.742+0000 7fb728a73640 1 -- 172.21.15.157:0/2551471069 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb724159760 con 0x7fb7240c5b70 2024-04-24T23:34:50.744 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.742+0000 7fb728a73640 1 --2- 172.21.15.157:0/2551471069 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7240c5b70 0x7fb724155a60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb71c009e50 tx=0x7fb71c02a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40147e7a82a6873d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:50.744 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.742+0000 7fb722ffd640 1 -- 172.21.15.157:0/2551471069 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71c036070 con 0x7fb7240c5b70 2024-04-24T23:34:50.744 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.742+0000 7fb722ffd640 1 -- 172.21.15.157:0/2551471069 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb71c031600 con 0x7fb7240c5b70 2024-04-24T23:34:50.744 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.743+0000 7fb722ffd640 1 -- 172.21.15.157:0/2551471069 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71c0319d0 con 0x7fb7240c5b70 2024-04-24T23:34:50.744 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.743+0000 7fb72a4d6640 1 -- 172.21.15.157:0/2551471069 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7240c5b70 msgr2=0x7fb724155a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:50.744 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.743+0000 7fb72a4d6640 1 --2- 172.21.15.157:0/2551471069 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7240c5b70 0x7fb724155a60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb71c009e50 tx=0x7fb71c02a5f0 comp rx=0 tx=0).stop 2024-04-24T23:34:50.744 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.743+0000 7fb72a4d6640 1 -- 172.21.15.157:0/2551471069 shutdown_connections 2024-04-24T23:34:50.744 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.743+0000 7fb72a4d6640 1 --2- 172.21.15.157:0/2551471069 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7240c5b70 0x7fb724155a60 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:50.744 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.743+0000 7fb72a4d6640 1 --2- 172.21.15.157:0/2551471069 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb724152b40 0x7fb7240c5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:50.744 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.743+0000 7fb72a4d6640 1 -- 172.21.15.157:0/2551471069 >> 172.21.15.157:0/2551471069 conn(0x7fb7240672c0 msgr2=0x7fb7240676d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:50.744 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.743+0000 7fb72a4d6640 1 -- 172.21.15.157:0/2551471069 shutdown_connections 2024-04-24T23:34:50.744 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.743+0000 7fb72a4d6640 1 -- 172.21.15.157:0/2551471069 wait complete. 2024-04-24T23:34:50.745 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.744+0000 7fb72a4d6640 1 Processor -- start 2024-04-24T23:34:50.745 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.744+0000 7fb72a4d6640 1 -- start start 2024-04-24T23:34:50.746 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.745+0000 7fb72a4d6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7240be360 0x7fb7240bed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:50.746 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.745+0000 7fb72a4d6640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7240c5b70 0x7fb7240bf2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:50.746 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.745+0000 7fb72a4d6640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb724152b40 0x7fb7240ac290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:50.747 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.745+0000 7fb72a4d6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb7240caa00 con 0x7fb7240be360 2024-04-24T23:34:50.747 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.745+0000 7fb72a4d6640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb7240cb210 con 0x7fb7240c5b70 2024-04-24T23:34:50.747 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.745+0000 7fb72a4d6640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb7240cba40 con 0x7fb724152b40 2024-04-24T23:34:50.747 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.745+0000 7fb723fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7240be360 0x7fb7240bed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:50.747 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.745+0000 7fb7237fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7240c5b70 0x7fb7240bf2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:50.747 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.745+0000 7fb728a73640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb724152b40 0x7fb7240ac290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:50.747 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.745+0000 7fb723fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7240be360 0x7fb7240bed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:41328/0 (socket says 172.21.15.157:41328) 2024-04-24T23:34:50.747 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.745+0000 7fb723fff640 1 -- 172.21.15.157:0/1303517501 learned_addr learned my addr 172.21.15.157:0/1303517501 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:34:50.747 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.745+0000 7fb723fff640 1 -- 172.21.15.157:0/1303517501 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb724152b40 msgr2=0x7fb7240ac290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:50.747 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.745+0000 7fb723fff640 1 --2- 172.21.15.157:0/1303517501 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb724152b40 0x7fb7240ac290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:50.747 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.745+0000 7fb723fff640 1 -- 172.21.15.157:0/1303517501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7240c5b70 msgr2=0x7fb7240bf2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:50.747 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.745+0000 7fb723fff640 1 --2- 172.21.15.157:0/1303517501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7240c5b70 0x7fb7240bf2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:50.747 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.745+0000 7fb723fff640 1 -- 172.21.15.157:0/1303517501 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7241eb310 con 0x7fb7240be360 2024-04-24T23:34:50.747 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.746+0000 7fb723fff640 1 --2- 172.21.15.157:0/1303517501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7240be360 0x7fb7240bed90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb71400e9b0 tx=0x7fb71400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:50.748 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.746+0000 7fb720ff9640 1 -- 172.21.15.157:0/1303517501 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb714017070 con 0x7fb7240be360 2024-04-24T23:34:50.748 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.746+0000 7fb720ff9640 1 -- 172.21.15.157:0/1303517501 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb71400c9a0 con 0x7fb7240be360 2024-04-24T23:34:50.748 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.746+0000 7fb72a4d6640 1 -- 172.21.15.157:0/1303517501 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7240ab7b0 con 0x7fb7240be360 2024-04-24T23:34:50.748 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.746+0000 7fb720ff9640 1 -- 172.21.15.157:0/1303517501 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71400cd90 con 0x7fb7240be360 2024-04-24T23:34:50.748 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.746+0000 7fb72a4d6640 1 -- 172.21.15.157:0/1303517501 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7241eb000 con 0x7fb7240be360 2024-04-24T23:34:50.748 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.747+0000 7fb72a4d6640 1 -- 172.21.15.157:0/1303517501 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7240681c0 con 0x7fb7240be360 2024-04-24T23:34:50.749 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.747+0000 7fb720ff9640 1 -- 172.21.15.157:0/1303517501 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7fb71401a070 con 0x7fb7240be360 2024-04-24T23:34:50.749 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.748+0000 7fb720ff9640 1 --2- 172.21.15.157:0/1303517501 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fb71006bda0 0x7fb71006e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:50.749 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.748+0000 7fb720ff9640 1 -- 172.21.15.157:0/1303517501 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fb71408acb0 con 0x7fb7240be360 2024-04-24T23:34:50.749 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.748+0000 7fb7237fe640 1 --2- 172.21.15.157:0/1303517501 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fb71006bda0 0x7fb71006e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:50.749 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.748+0000 7fb7237fe640 1 --2- 172.21.15.157:0/1303517501 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fb71006bda0 0x7fb71006e260 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb7241520c0 tx=0x7fb718007e80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:50.752 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.752+0000 7fb720ff9640 1 -- 172.21.15.157:0/1303517501 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb7140594d0 con 0x7fb7240be360 2024-04-24T23:34:50.964 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.965+0000 7fb72a4d6640 1 -- 172.21.15.157:0/1303517501 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb72406d3f0 con 0x7fb7240be360 2024-04-24T23:34:50.965 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.965+0000 7fb720ff9640 1 -- 172.21.15.157:0/1303517501 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3876 (secure 0 0 0) 0x7fb71405d7d0 con 0x7fb7240be360 2024-04-24T23:34:50.965 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:34:50.966 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1276438765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1276438765},{"type":"v1","addr":"172.21.15.169:6837","nonce":1276438765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1398599086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":1398599086},{"type":"v1","addr":"172.21.15.157:6835","nonce":1398599086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4258,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3795840339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3795840339},{"type":"v1","addr":"172.21.15.157:6834","nonce":3795840339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4270,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6836/3782746080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6835","nonce":3782746080},{"type":"v1","addr":"172.21.15.169:6836","nonce":3782746080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-24T23:34:50.966 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 3 2024-04-24T23:34:50.967 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.967+0000 7fb72a4d6640 1 -- 172.21.15.157:0/1303517501 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fb71006bda0 msgr2=0x7fb71006e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:50.967 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.967+0000 7fb72a4d6640 1 --2- 172.21.15.157:0/1303517501 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fb71006bda0 0x7fb71006e260 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb7241520c0 tx=0x7fb718007e80 comp rx=0 tx=0).stop 2024-04-24T23:34:50.968 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.967+0000 7fb72a4d6640 1 -- 172.21.15.157:0/1303517501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7240be360 msgr2=0x7fb7240bed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:50.968 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.967+0000 7fb72a4d6640 1 --2- 172.21.15.157:0/1303517501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7240be360 0x7fb7240bed90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb71400e9b0 tx=0x7fb71400ee70 comp rx=0 tx=0).stop 2024-04-24T23:34:50.968 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.967+0000 7fb72a4d6640 1 -- 172.21.15.157:0/1303517501 shutdown_connections 2024-04-24T23:34:50.968 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.967+0000 7fb72a4d6640 1 --2- 172.21.15.157:0/1303517501 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fb71006bda0 0x7fb71006e260 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:50.968 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.967+0000 7fb72a4d6640 1 --2- 172.21.15.157:0/1303517501 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb724152b40 0x7fb7240ac290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:50.968 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.967+0000 7fb72a4d6640 1 --2- 172.21.15.157:0/1303517501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7240c5b70 0x7fb7240bf2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:50.968 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.967+0000 7fb72a4d6640 1 --2- 172.21.15.157:0/1303517501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7240be360 0x7fb7240bed90 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:50.968 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.967+0000 7fb72a4d6640 1 -- 172.21.15.157:0/1303517501 >> 172.21.15.157:0/1303517501 conn(0x7fb7240672c0 msgr2=0x7fb72414ef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:50.968 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.968+0000 7fb72a4d6640 1 -- 172.21.15.157:0/1303517501 shutdown_connections 2024-04-24T23:34:50.968 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:50.968+0000 7fb72a4d6640 1 -- 172.21.15.157:0/1303517501 wait complete. 2024-04-24T23:34:50.978 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:34:51.075 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.075+0000 7f265fb09640 1 Processor -- start 2024-04-24T23:34:51.075 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.075+0000 7f265fb09640 1 -- start start 2024-04-24T23:34:51.077 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.076+0000 7f265fb09640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2658073130 0x7f26580c2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:51.077 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.076+0000 7f265fb09640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f265814b720 0x7f265814bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:51.077 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.076+0000 7f265fb09640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f265806d3c0 con 0x7f26580c4b20 2024-04-24T23:34:51.077 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.076+0000 7f265fb09640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f265806c400 con 0x7f2658073130 2024-04-24T23:34:51.077 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.076+0000 7f265fb09640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2658073510 con 0x7f265814b720 2024-04-24T23:34:51.077 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.076+0000 7f265d0a4640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f26580c4b20 0x7f265806ccb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:60630/0 (socket says 172.21.15.157:60630) 2024-04-24T23:34:51.077 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.076+0000 7f265d0a4640 1 -- 172.21.15.157:0/1131812750 learned_addr learned my addr 172.21.15.157:0/1131812750 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-24T23:34:51.078 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.076+0000 7f265d8a5640 1 --2- 172.21.15.157:0/1131812750 >> v2:172.21.15.169:3300/0 conn(0x7f2658073130 0x7f26580c2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:51.078 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.076+0000 7f265e0a6640 1 --2- 172.21.15.157:0/1131812750 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f265814b720 0x7f265814bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:51.078 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.077+0000 7f265c8a3640 1 -- 172.21.15.157:0/1131812750 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1515493985 0 0) 0x7f265806d3c0 con 0x7f26580c4b20 2024-04-24T23:34:51.078 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.077+0000 7f265c8a3640 1 -- 172.21.15.157:0/1131812750 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2644003520 con 0x7f26580c4b20 2024-04-24T23:34:51.078 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.077+0000 7f265d8a5640 1 -- 172.21.15.157:0/1131812750 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f265814b720 msgr2=0x7f265814bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:51.078 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.077+0000 7f265d8a5640 1 --2- 172.21.15.157:0/1131812750 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f265814b720 0x7f265814bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:51.078 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.077+0000 7f265d8a5640 1 -- 172.21.15.157:0/1131812750 >> v1:172.21.15.157:6789/0 conn(0x7f26580c4b20 legacy=0x7f265806ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:51.078 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.077+0000 7f265d8a5640 1 -- 172.21.15.157:0/1131812750 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f265806c6a0 con 0x7f2658073130 2024-04-24T23:34:51.078 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.077+0000 7f265d8a5640 1 --2- 172.21.15.157:0/1131812750 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2658073130 0x7f26580c2990 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f26480097a0 tx=0x7f264802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f05ab0e862c1e563 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:51.078 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.078+0000 7f265c8a3640 1 -- 172.21.15.157:0/1131812750 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2648037070 con 0x7f2658073130 2024-04-24T23:34:51.078 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.078+0000 7f265c8a3640 1 -- 172.21.15.157:0/1131812750 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f264802ae50 con 0x7f2658073130 2024-04-24T23:34:51.078 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.078+0000 7f265c8a3640 1 -- 172.21.15.157:0/1131812750 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2648032640 con 0x7f2658073130 2024-04-24T23:34:51.081 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.078+0000 7f265fb09640 1 -- 172.21.15.157:0/1131812750 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2658073130 msgr2=0x7f26580c2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:51.081 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.078+0000 7f265fb09640 1 --2- 172.21.15.157:0/1131812750 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2658073130 0x7f26580c2990 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f26480097a0 tx=0x7f264802a100 comp rx=0 tx=0).stop 2024-04-24T23:34:51.081 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.078+0000 7f265fb09640 1 -- 172.21.15.157:0/1131812750 shutdown_connections 2024-04-24T23:34:51.081 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.078+0000 7f265fb09640 1 --2- 172.21.15.157:0/1131812750 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f265814b720 0x7f265814bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:51.081 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.078+0000 7f265fb09640 1 --2- 172.21.15.157:0/1131812750 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2658073130 0x7f26580c2990 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:51.082 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.078+0000 7f265fb09640 1 -- 172.21.15.157:0/1131812750 >> 172.21.15.157:0/1131812750 conn(0x7f26580a8720 msgr2=0x7f26580a8b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:51.082 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.078+0000 7f265fb09640 1 -- 172.21.15.157:0/1131812750 shutdown_connections 2024-04-24T23:34:51.082 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.078+0000 7f265fb09640 1 -- 172.21.15.157:0/1131812750 wait complete. 2024-04-24T23:34:51.082 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.079+0000 7f265fb09640 1 Processor -- start 2024-04-24T23:34:51.082 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.079+0000 7f265fb09640 1 -- start start 2024-04-24T23:34:51.082 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.080+0000 7f265fb09640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2658073130 0x7f2658152030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:51.082 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.080+0000 7f265fb09640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26580c4b20 0x7f265814a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:51.082 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.080+0000 7f265fb09640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f265814b720 0x7f265814ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:51.082 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.080+0000 7f265fb09640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f26580c2470 con 0x7f26580c4b20 2024-04-24T23:34:51.082 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.080+0000 7f265fb09640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f26580c2190 con 0x7f2658073130 2024-04-24T23:34:51.082 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.080+0000 7f265fb09640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f265814c0d0 con 0x7f265814b720 2024-04-24T23:34:51.082 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.080+0000 7f265d0a4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26580c4b20 0x7f265814a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:51.082 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.080+0000 7f265d0a4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26580c4b20 0x7f265814a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:41340/0 (socket says 172.21.15.157:41340) 2024-04-24T23:34:51.082 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.080+0000 7f265d0a4640 1 -- 172.21.15.157:0/154926712 learned_addr learned my addr 172.21.15.157:0/154926712 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:34:51.083 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.080+0000 7f265d8a5640 1 --2- 172.21.15.157:0/154926712 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2658073130 0x7f2658152030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:51.083 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.080+0000 7f265e0a6640 1 --2- 172.21.15.157:0/154926712 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f265814b720 0x7f265814ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:51.083 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.080+0000 7f265d0a4640 1 -- 172.21.15.157:0/154926712 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f265814b720 msgr2=0x7f265814ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:51.083 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.080+0000 7f265d0a4640 1 --2- 172.21.15.157:0/154926712 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f265814b720 0x7f265814ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:51.083 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.080+0000 7f265d0a4640 1 -- 172.21.15.157:0/154926712 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2658073130 msgr2=0x7f2658152030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:51.083 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.080+0000 7f265d0a4640 1 --2- 172.21.15.157:0/154926712 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2658073130 0x7f2658152030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:51.083 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.080+0000 7f265d0a4640 1 -- 172.21.15.157:0/154926712 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26581e9370 con 0x7f26580c4b20 2024-04-24T23:34:51.083 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.081+0000 7f265d0a4640 1 --2- 172.21.15.157:0/154926712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26580c4b20 0x7f265814a710 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f265400c500 tx=0x7f265400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:51.083 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.081+0000 7f264e7fc640 1 -- 172.21.15.157:0/154926712 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2654017070 con 0x7f26580c4b20 2024-04-24T23:34:51.083 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.081+0000 7f264e7fc640 1 -- 172.21.15.157:0/154926712 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2654005ce0 con 0x7f26580c4b20 2024-04-24T23:34:51.083 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.081+0000 7f264e7fc640 1 -- 172.21.15.157:0/154926712 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26540072f0 con 0x7f26580c4b20 2024-04-24T23:34:51.083 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.081+0000 7f265fb09640 1 -- 172.21.15.157:0/154926712 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26581e9650 con 0x7f26580c4b20 2024-04-24T23:34:51.083 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.081+0000 7f265fb09640 1 -- 172.21.15.157:0/154926712 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26580c57c0 con 0x7f26580c4b20 2024-04-24T23:34:51.083 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.082+0000 7f265fb09640 1 -- 172.21.15.157:0/154926712 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26580681c0 con 0x7f26580c4b20 2024-04-24T23:34:51.084 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.082+0000 7f264e7fc640 1 -- 172.21.15.157:0/154926712 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7f265401a070 con 0x7f26580c4b20 2024-04-24T23:34:51.084 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.082+0000 7f264e7fc640 1 --2- 172.21.15.157:0/154926712 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f263c06bda0 0x7f263c06e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:51.084 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.083+0000 7f264e7fc640 1 -- 172.21.15.157:0/154926712 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f2654089e00 con 0x7f26580c4b20 2024-04-24T23:34:51.084 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.083+0000 7f265d8a5640 1 --2- 172.21.15.157:0/154926712 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f263c06bda0 0x7f263c06e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:51.084 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.083+0000 7f265d8a5640 1 --2- 172.21.15.157:0/154926712 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f263c06bda0 0x7f263c06e260 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2648034000 tx=0x7f264802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:51.086 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.087+0000 7f264e7fc640 1 -- 172.21.15.157:0/154926712 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2654058500 con 0x7f26580c4b20 2024-04-24T23:34:51.262 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.262+0000 7f265fb09640 1 -- 172.21.15.157:0/154926712 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f26580c4870 con 0x7f26580c4b20 2024-04-24T23:34:51.262 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.263+0000 7f264e7fc640 1 -- 172.21.15.157:0/154926712 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12908 (secure 0 0 0) 0x7f265405c800 con 0x7f26580c4b20 2024-04-24T23:34:51.263 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:34:51.263 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":17,"fsid":"223685fb-5b64-46a5-9857-3791c9c07df2","created":"2024-04-24T23:34:37.504202+0000","modified":"2024-04-24T23:34:50.614258+0000","last_up_change":"2024-04-24T23:34:47.584963+0000","last_in_change":"2024-04-24T23:34:39.632954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:34:47.796655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"95d14ac8-fccf-4261-be8e-edafa746f629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6809","nonce":4003822160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6811","nonce":4003822160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6815","nonce":4003822160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6813","nonce":4003822160}]},"public_addr":"172.21.15.157:6809/4003822160","cluster_addr":"172.21.15.157:6811/4003822160","heartbeat_back_addr":"172.21.15.157:6815/4003822160","heartbeat_front_addr":"172.21.15.157:6813/4003822160","state":["exists","up"]},{"osd":1,"uuid":"7be79da9-8eec-4990-9178-71d602e39d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6817","nonce":1557575576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6819","nonce":1557575576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6823","nonce":1557575576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6821","nonce":1557575576}]},"public_addr":"172.21.15.157:6817/1557575576","cluster_addr":"172.21.15.157:6819/1557575576","heartbeat_back_addr":"172.21.15.157:6823/1557575576","heartbeat_front_addr":"172.21.15.157:6821/1557575576","state":["exists","up"]},{"osd":2,"uuid":"0127ed00-b7c3-466d-bfb0-01b0fd315f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6825","nonce":2793948092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6827","nonce":2793948092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6831","nonce":2793948092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6829","nonce":2793948092}]},"public_addr":"172.21.15.157:6825/2793948092","cluster_addr":"172.21.15.157:6827/2793948092","heartbeat_back_addr":"172.21.15.157:6831/2793948092","heartbeat_front_addr":"172.21.15.157:6829/2793948092","state":["exists","up"]},{"osd":3,"uuid":"f2331acc-aa37-433e-868e-3561b6039562","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6801","nonce":422758549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6803","nonce":422758549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6807","nonce":422758549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6805","nonce":422758549}]},"public_addr":"172.21.15.157:6801/422758549","cluster_addr":"172.21.15.157:6803/422758549","heartbeat_back_addr":"172.21.15.157:6807/422758549","heartbeat_front_addr":"172.21.15.157:6805/422758549","state":["exists","up"]},{"osd":4,"uuid":"1998854b-7398-43f5-ab7d-79fa0debcf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6801","nonce":321910099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6803","nonce":321910099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6807","nonce":321910099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6805","nonce":321910099}]},"public_addr":"172.21.15.169:6801/321910099","cluster_addr":"172.21.15.169:6803/321910099","heartbeat_back_addr":"172.21.15.169:6807/321910099","heartbeat_front_addr":"172.21.15.169:6805/321910099","state":["exists","up"]},{"osd":5,"uuid":"c37153ff-9b2b-48b9-a684-2c82b6f39f15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6825","nonce":243321711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6827","nonce":243321711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6831","nonce":243321711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6829","nonce":243321711}]},"public_addr":"172.21.15.169:6825/243321711","cluster_addr":"172.21.15.169:6827/243321711","heartbeat_back_addr":"172.21.15.169:6831/243321711","heartbeat_front_addr":"172.21.15.169:6829/243321711","state":["exists","up"]},{"osd":6,"uuid":"8db79e1c-f36b-49cb-8d8a-a20c2aec57c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6809","nonce":3441420785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6811","nonce":3441420785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6815","nonce":3441420785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6813","nonce":3441420785}]},"public_addr":"172.21.15.169:6809/3441420785","cluster_addr":"172.21.15.169:6811/3441420785","heartbeat_back_addr":"172.21.15.169:6815/3441420785","heartbeat_front_addr":"172.21.15.169:6813/3441420785","state":["exists","up"]},{"osd":7,"uuid":"928586e9-296c-4b8b-9b27-42f9f155dbdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6817","nonce":1663179020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6819","nonce":1663179020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6823","nonce":1663179020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6821","nonce":1663179020}]},"public_addr":"172.21.15.169:6817/1663179020","cluster_addr":"172.21.15.169:6819/1663179020","heartbeat_back_addr":"172.21.15.169:6823/1663179020","heartbeat_front_addr":"172.21.15.169:6821/1663179020","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.848039+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.826489+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.801120+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.727706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.824343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.791915+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.878073+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.810831+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:34:51.265 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.265+0000 7f265fb09640 1 -- 172.21.15.157:0/154926712 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f263c06bda0 msgr2=0x7f263c06e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:51.265 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.265+0000 7f265fb09640 1 --2- 172.21.15.157:0/154926712 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f263c06bda0 0x7f263c06e260 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2648034000 tx=0x7f264802e000 comp rx=0 tx=0).stop 2024-04-24T23:34:51.265 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.265+0000 7f265fb09640 1 -- 172.21.15.157:0/154926712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26580c4b20 msgr2=0x7f265814a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:51.265 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.265+0000 7f265fb09640 1 --2- 172.21.15.157:0/154926712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26580c4b20 0x7f265814a710 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f265400c500 tx=0x7f265400c9c0 comp rx=0 tx=0).stop 2024-04-24T23:34:51.265 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.265+0000 7f265fb09640 1 -- 172.21.15.157:0/154926712 shutdown_connections 2024-04-24T23:34:51.265 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.265+0000 7f265fb09640 1 --2- 172.21.15.157:0/154926712 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f263c06bda0 0x7f263c06e260 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:51.266 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.265+0000 7f265fb09640 1 --2- 172.21.15.157:0/154926712 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f265814b720 0x7f265814ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:51.266 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.265+0000 7f265fb09640 1 --2- 172.21.15.157:0/154926712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26580c4b20 0x7f265814a710 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:51.266 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.265+0000 7f265fb09640 1 --2- 172.21.15.157:0/154926712 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2658073130 0x7f2658152030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:51.266 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.265+0000 7f265fb09640 1 -- 172.21.15.157:0/154926712 >> 172.21.15.157:0/154926712 conn(0x7f26580a8720 msgr2=0x7f26580a7930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:51.266 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.265+0000 7f265fb09640 1 -- 172.21.15.157:0/154926712 shutdown_connections 2024-04-24T23:34:51.266 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.265+0000 7f265fb09640 1 -- 172.21.15.157:0/154926712 wait complete. 2024-04-24T23:34:51.275 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:34:47.796655+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:34:51.276 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:34:51.381 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.378+0000 7f922adb0640 1 Processor -- start 2024-04-24T23:34:51.381 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.378+0000 7f922adb0640 1 -- start start 2024-04-24T23:34:51.381 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.379+0000 7f922adb0640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9224156390 0x7f9224152cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:51.381 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.379+0000 7f922adb0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92240736b0 con 0x7f92240c3370 2024-04-24T23:34:51.381 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.379+0000 7f922adb0640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92240a60e0 con 0x7f92240a5c50 2024-04-24T23:34:51.382 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.379+0000 7f922adb0640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f9224155f00 con 0x7f9224156390 2024-04-24T23:34:51.382 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.379+0000 7f9228b4c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f92240c3370 0x7f92240c12e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:60642/0 (socket says 172.21.15.157:60642) 2024-04-24T23:34:51.382 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.379+0000 7f9228b4c640 1 -- 172.21.15.157:0/3274066787 learned_addr learned my addr 172.21.15.157:0/3274066787 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-24T23:34:51.382 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.379+0000 7f922934d640 1 --2- 172.21.15.157:0/3274066787 >> v2:172.21.15.169:3300/0 conn(0x7f9224156390 0x7f9224152cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:51.382 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.379+0000 7f92237fe640 1 -- 172.21.15.157:0/3274066787 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1440303757 0 0) 0x7f92240736b0 con 0x7f92240c3370 2024-04-24T23:34:51.382 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.379+0000 7f92237fe640 1 -- 172.21.15.157:0/3274066787 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f920c003520 con 0x7f92240c3370 2024-04-24T23:34:51.382 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.379+0000 7f922934d640 1 -- 172.21.15.157:0/3274066787 >> v1:172.21.15.169:6789/0 conn(0x7f92240a5c50 legacy=0x7f922415ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:51.382 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.380+0000 7f92237fe640 1 -- 172.21.15.157:0/3274066787 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 769786392 0 0) 0x7f92240a60e0 con 0x7f92240a5c50 2024-04-24T23:34:51.382 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.380+0000 7f922934d640 1 -- 172.21.15.157:0/3274066787 >> v1:172.21.15.157:6789/0 conn(0x7f92240c3370 legacy=0x7f92240c12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:51.382 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.380+0000 7f922934d640 1 -- 172.21.15.157:0/3274066787 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92240a63c0 con 0x7f9224156390 2024-04-24T23:34:51.382 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.380+0000 7f922934d640 1 --2- 172.21.15.157:0/3274066787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9224156390 0x7f9224152cc0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f921c009d90 tx=0x7f921c02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f38ba758dd0a9ca server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:51.382 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.380+0000 7f92237fe640 1 -- 172.21.15.157:0/3274066787 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f921c035020 con 0x7f9224156390 2024-04-24T23:34:51.382 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.380+0000 7f92237fe640 1 -- 172.21.15.157:0/3274066787 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f921c007540 con 0x7f9224156390 2024-04-24T23:34:51.383 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.380+0000 7f92237fe640 1 -- 172.21.15.157:0/3274066787 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9214001670 con 0x7f9224156390 2024-04-24T23:34:51.383 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.381+0000 7f922adb0640 1 -- 172.21.15.157:0/3274066787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9224156390 msgr2=0x7f9224152cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:51.383 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.381+0000 7f922adb0640 1 --2- 172.21.15.157:0/3274066787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9224156390 0x7f9224152cc0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f921c009d90 tx=0x7f921c02a660 comp rx=0 tx=0).stop 2024-04-24T23:34:51.383 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.381+0000 7f922adb0640 1 -- 172.21.15.157:0/3274066787 shutdown_connections 2024-04-24T23:34:51.383 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.381+0000 7f922adb0640 1 --2- 172.21.15.157:0/3274066787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9224156390 0x7f9224152cc0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:51.383 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.381+0000 7f922adb0640 1 -- 172.21.15.157:0/3274066787 >> 172.21.15.157:0/3274066787 conn(0x7f92240c68e0 msgr2=0x7f92240c6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:51.383 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.381+0000 7f922adb0640 1 -- 172.21.15.157:0/3274066787 shutdown_connections 2024-04-24T23:34:51.383 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.381+0000 7f922adb0640 1 -- 172.21.15.157:0/3274066787 wait complete. 2024-04-24T23:34:51.386 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.382+0000 7f922adb0640 1 Processor -- start 2024-04-24T23:34:51.386 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.382+0000 7f922adb0640 1 -- start start 2024-04-24T23:34:51.386 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.383+0000 7f922adb0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92240c3370 0x7f9224145a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:51.386 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.383+0000 7f922adb0640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9224156390 0x7f9224149140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:51.386 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.383+0000 7f922adb0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9224148790 0x7f922414dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:51.386 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.383+0000 7f922adb0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f92240c19f0 con 0x7f92240c3370 2024-04-24T23:34:51.386 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.383+0000 7f922adb0640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9224155b90 con 0x7f9224148790 2024-04-24T23:34:51.386 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.383+0000 7f922adb0640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f92240c5bc0 con 0x7f9224156390 2024-04-24T23:34:51.386 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.383+0000 7f9228b4c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92240c3370 0x7f9224145a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:51.386 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.383+0000 7f9228b4c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92240c3370 0x7f9224145a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:41342/0 (socket says 172.21.15.157:41342) 2024-04-24T23:34:51.387 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.383+0000 7f9228b4c640 1 -- 172.21.15.157:0/2603558193 learned_addr learned my addr 172.21.15.157:0/2603558193 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:34:51.387 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.383+0000 7f9228b4c640 1 -- 172.21.15.157:0/2603558193 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9224156390 msgr2=0x7f9224149140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:51.387 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.383+0000 7f9228b4c640 1 --2- 172.21.15.157:0/2603558193 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9224156390 0x7f9224149140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:51.387 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.383+0000 7f9228b4c640 1 -- 172.21.15.157:0/2603558193 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9224148790 msgr2=0x7f922414dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:51.387 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.383+0000 7f9228b4c640 1 --2- 172.21.15.157:0/2603558193 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9224148790 0x7f922414dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:51.387 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.383+0000 7f9228b4c640 1 -- 172.21.15.157:0/2603558193 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92241eafe0 con 0x7f92240c3370 2024-04-24T23:34:51.387 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.383+0000 7f9228b4c640 1 --2- 172.21.15.157:0/2603558193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92240c3370 0x7f9224145a30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f921400c760 tx=0x7f921400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:51.387 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.384+0000 7f92217fa640 1 -- 172.21.15.157:0/2603558193 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9214018070 con 0x7f92240c3370 2024-04-24T23:34:51.387 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.384+0000 7f92217fa640 1 -- 172.21.15.157:0/2603558193 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92140074e0 con 0x7f92240c3370 2024-04-24T23:34:51.387 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.384+0000 7f922adb0640 1 -- 172.21.15.157:0/2603558193 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92241eb2c0 con 0x7f92240c3370 2024-04-24T23:34:51.387 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.384+0000 7f92217fa640 1 -- 172.21.15.157:0/2603558193 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9214004a00 con 0x7f92240c3370 2024-04-24T23:34:51.387 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.384+0000 7f922adb0640 1 -- 172.21.15.157:0/2603558193 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92241eb880 con 0x7f92240c3370 2024-04-24T23:34:51.387 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.385+0000 7f922adb0640 1 -- 172.21.15.157:0/2603558193 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92240681c0 con 0x7f92240c3370 2024-04-24T23:34:51.388 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.385+0000 7f92217fa640 1 -- 172.21.15.157:0/2603558193 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7f921401b070 con 0x7f92240c3370 2024-04-24T23:34:51.388 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.385+0000 7f92217fa640 1 --2- 172.21.15.157:0/2603558193 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f921006bda0 0x7f921006e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:51.388 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.385+0000 7f92217fa640 1 -- 172.21.15.157:0/2603558193 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f9214089f40 con 0x7f92240c3370 2024-04-24T23:34:51.388 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.386+0000 7f9223fff640 1 --2- 172.21.15.157:0/2603558193 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f921006bda0 0x7f921006e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:51.388 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.386+0000 7f9223fff640 1 --2- 172.21.15.157:0/2603558193 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f921006bda0 0x7f921006e260 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f92240ac7a0 tx=0x7f9218007910 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:51.389 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.390+0000 7f92217fa640 1 -- 172.21.15.157:0/2603558193 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9214058db0 con 0x7f92240c3370 2024-04-24T23:34:51.564 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.564+0000 7f922adb0640 1 -- 172.21.15.157:0/2603558193 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f92241460f0 con 0x7f92240c3370 2024-04-24T23:34:51.564 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.564+0000 7f92217fa640 1 -- 172.21.15.157:0/2603558193 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) ==== 93+0+10 (secure 0 0 0) 0x7f92140586b0 con 0x7f92240c3370 2024-04-24T23:34:51.564 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2024-04-24T23:34:51.566 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.566+0000 7f922adb0640 1 -- 172.21.15.157:0/2603558193 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f921006bda0 msgr2=0x7f921006e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:51.567 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.566+0000 7f922adb0640 1 --2- 172.21.15.157:0/2603558193 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f921006bda0 0x7f921006e260 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f92240ac7a0 tx=0x7f9218007910 comp rx=0 tx=0).stop 2024-04-24T23:34:51.567 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.566+0000 7f922adb0640 1 -- 172.21.15.157:0/2603558193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92240c3370 msgr2=0x7f9224145a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:51.567 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.566+0000 7f922adb0640 1 --2- 172.21.15.157:0/2603558193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92240c3370 0x7f9224145a30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f921400c760 tx=0x7f921400cc20 comp rx=0 tx=0).stop 2024-04-24T23:34:51.567 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.567+0000 7f922adb0640 1 -- 172.21.15.157:0/2603558193 shutdown_connections 2024-04-24T23:34:51.567 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.567+0000 7f922adb0640 1 --2- 172.21.15.157:0/2603558193 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f921006bda0 0x7f921006e260 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:51.567 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.567+0000 7f922adb0640 1 --2- 172.21.15.157:0/2603558193 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9224148790 0x7f922414dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:51.567 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.567+0000 7f922adb0640 1 --2- 172.21.15.157:0/2603558193 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9224156390 0x7f9224149140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:51.567 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.567+0000 7f922adb0640 1 --2- 172.21.15.157:0/2603558193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92240c3370 0x7f9224145a30 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:51.567 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.567+0000 7f922adb0640 1 -- 172.21.15.157:0/2603558193 >> 172.21.15.157:0/2603558193 conn(0x7f92240c68e0 msgr2=0x7f922406d830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:51.567 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.567+0000 7f922adb0640 1 -- 172.21.15.157:0/2603558193 shutdown_connections 2024-04-24T23:34:51.567 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.567+0000 7f922adb0640 1 -- 172.21.15.157:0/2603558193 wait complete. 2024-04-24T23:34:51.577 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-24T23:34:51.673 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.674+0000 7f1354276640 1 Processor -- start 2024-04-24T23:34:51.673 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.674+0000 7f1354276640 1 -- start start 2024-04-24T23:34:51.675 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.674+0000 7f1354276640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f134c0c2110 0x7f134c0ab5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:51.675 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.674+0000 7f1354276640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f134c0c3820 0x7f134c0c0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:51.675 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.674+0000 7f1354276640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f134c155d60 con 0x7f134c06b3d0 2024-04-24T23:34:51.675 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.674+0000 7f1354276640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f134c06e030 con 0x7f134c0c3820 2024-04-24T23:34:51.675 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.674+0000 7f1354276640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f134c14f7f0 con 0x7f134c0c2110 2024-04-24T23:34:51.675 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.675+0000 7f1352813640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f134c0c3820 0x7f134c0c0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:51.675 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.675+0000 7f1352813640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f134c0c3820 0x7f134c0c0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:41350/0 (socket says 172.21.15.157:41350) 2024-04-24T23:34:51.675 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.675+0000 7f1352813640 1 -- 172.21.15.157:0/1507302136 learned_addr learned my addr 172.21.15.157:0/1507302136 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:34:51.675 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.675+0000 7f1351811640 1 --2- 172.21.15.157:0/1507302136 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f134c0c2110 0x7f134c0ab5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:51.675 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.675+0000 7f1352813640 1 -- 172.21.15.157:0/1507302136 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f134c0c2110 msgr2=0x7f134c0ab5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:51.675 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.675+0000 7f1352813640 1 --2- 172.21.15.157:0/1507302136 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f134c0c2110 0x7f134c0ab5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:51.675 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.675+0000 7f1352813640 1 -- 172.21.15.157:0/1507302136 >> v1:172.21.15.169:6789/0 conn(0x7f134c06b3d0 legacy=0x7f134c06b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:51.676 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.675+0000 7f1352813640 1 -- 172.21.15.157:0/1507302136 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f134c156040 con 0x7f134c0c3820 2024-04-24T23:34:51.676 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.675+0000 7f1351811640 1 --2- 172.21.15.157:0/1507302136 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f134c0c2110 0x7f134c0ab5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:34:51.677 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.676+0000 7f1352813640 1 --2- 172.21.15.157:0/1507302136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f134c0c3820 0x7f134c0c0620 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1344009e50 tx=0x7f134402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29aabb0ae9116038 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:51.677 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.676+0000 7f1351010640 1 -- 172.21.15.157:0/1507302136 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1344036070 con 0x7f134c0c3820 2024-04-24T23:34:51.677 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.676+0000 7f1351010640 1 -- 172.21.15.157:0/1507302136 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1344031600 con 0x7f134c0c3820 2024-04-24T23:34:51.677 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.676+0000 7f1351010640 1 -- 172.21.15.157:0/1507302136 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13440319d0 con 0x7f134c0c3820 2024-04-24T23:34:51.677 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.676+0000 7f1354276640 1 -- 172.21.15.157:0/1507302136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f134c0c3820 msgr2=0x7f134c0c0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:51.677 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.676+0000 7f1354276640 1 --2- 172.21.15.157:0/1507302136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f134c0c3820 0x7f134c0c0620 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1344009e50 tx=0x7f134402a710 comp rx=0 tx=0).stop 2024-04-24T23:34:51.678 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.676+0000 7f1354276640 1 -- 172.21.15.157:0/1507302136 shutdown_connections 2024-04-24T23:34:51.678 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.676+0000 7f1354276640 1 --2- 172.21.15.157:0/1507302136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f134c0c3820 0x7f134c0c0620 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:51.678 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.676+0000 7f1354276640 1 --2- 172.21.15.157:0/1507302136 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f134c0c2110 0x7f134c0ab5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:51.678 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.676+0000 7f1354276640 1 -- 172.21.15.157:0/1507302136 >> 172.21.15.157:0/1507302136 conn(0x7f134c0c71f0 msgr2=0x7f134c0c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:51.678 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.676+0000 7f1354276640 1 -- 172.21.15.157:0/1507302136 shutdown_connections 2024-04-24T23:34:51.678 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.676+0000 7f1354276640 1 -- 172.21.15.157:0/1507302136 wait complete. 2024-04-24T23:34:51.678 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.677+0000 7f1354276640 1 Processor -- start 2024-04-24T23:34:51.678 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.678+0000 7f1354276640 1 -- start start 2024-04-24T23:34:51.680 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.678+0000 7f1354276640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f134c06b3d0 0x7f134c14a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:51.680 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.678+0000 7f1354276640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f134c0c2110 0x7f134c1eaae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:51.680 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.678+0000 7f1354276640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f134c0c3820 0x7f134c1eb020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:51.680 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.678+0000 7f1354276640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f134c0ac6b0 con 0x7f134c0c3820 2024-04-24T23:34:51.681 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.678+0000 7f1354276640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f134c06bec0 con 0x7f134c06b3d0 2024-04-24T23:34:51.681 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.678+0000 7f1354276640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f134c0c1bf0 con 0x7f134c0c2110 2024-04-24T23:34:51.681 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.678+0000 7f1352813640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f134c0c3820 0x7f134c1eb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:51.681 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.678+0000 7f1352813640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f134c0c3820 0x7f134c1eb020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:41366/0 (socket says 172.21.15.157:41366) 2024-04-24T23:34:51.681 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.678+0000 7f1352813640 1 -- 172.21.15.157:0/2604492055 learned_addr learned my addr 172.21.15.157:0/2604492055 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:34:51.681 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.678+0000 7f1351811640 1 --2- 172.21.15.157:0/2604492055 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f134c0c2110 0x7f134c1eaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:51.681 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.678+0000 7f1352012640 1 --2- 172.21.15.157:0/2604492055 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f134c06b3d0 0x7f134c14a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:51.681 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.679+0000 7f1352813640 1 -- 172.21.15.157:0/2604492055 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f134c0c2110 msgr2=0x7f134c1eaae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:51.681 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.679+0000 7f1352813640 1 --2- 172.21.15.157:0/2604492055 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f134c0c2110 0x7f134c1eaae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:51.681 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.679+0000 7f1352813640 1 -- 172.21.15.157:0/2604492055 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f134c06b3d0 msgr2=0x7f134c14a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:51.681 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.679+0000 7f1352813640 1 --2- 172.21.15.157:0/2604492055 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f134c06b3d0 0x7f134c14a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:51.681 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.679+0000 7f1352813640 1 -- 172.21.15.157:0/2604492055 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f134c14ac00 con 0x7f134c0c3820 2024-04-24T23:34:51.681 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.679+0000 7f1352813640 1 --2- 172.21.15.157:0/2604492055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f134c0c3820 0x7f134c1eb020 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1344034040 tx=0x7f1344004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:51.682 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.679+0000 7f1342ffd640 1 -- 172.21.15.157:0/2604492055 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1344036050 con 0x7f134c0c3820 2024-04-24T23:34:51.682 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.679+0000 7f1342ffd640 1 -- 172.21.15.157:0/2604492055 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1344042070 con 0x7f134c0c3820 2024-04-24T23:34:51.682 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.679+0000 7f1342ffd640 1 -- 172.21.15.157:0/2604492055 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1344009510 con 0x7f134c0c3820 2024-04-24T23:34:51.682 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.679+0000 7f1354276640 1 -- 172.21.15.157:0/2604492055 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f134c150ec0 con 0x7f134c0c3820 2024-04-24T23:34:51.682 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.679+0000 7f1354276640 1 -- 172.21.15.157:0/2604492055 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f134c151480 con 0x7f134c0c3820 2024-04-24T23:34:51.682 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.680+0000 7f1354276640 1 -- 172.21.15.157:0/2604492055 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f134c0681c0 con 0x7f134c0c3820 2024-04-24T23:34:51.682 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.680+0000 7f1342ffd640 1 -- 172.21.15.157:0/2604492055 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7f1344007bd0 con 0x7f134c0c3820 2024-04-24T23:34:51.682 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.681+0000 7f1342ffd640 1 --2- 172.21.15.157:0/2604492055 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f133006bda0 0x7f133006e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:51.682 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.681+0000 7f1342ffd640 1 -- 172.21.15.157:0/2604492055 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f13440ab6d0 con 0x7f134c0c3820 2024-04-24T23:34:51.682 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.681+0000 7f1352012640 1 --2- 172.21.15.157:0/2604492055 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f133006bda0 0x7f133006e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:51.683 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.681+0000 7f1352012640 1 --2- 172.21.15.157:0/2604492055 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f133006bda0 0x7f133006e260 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f134c153ea0 tx=0x7f1348009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:51.684 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.685+0000 7f1342ffd640 1 -- 172.21.15.157:0/2604492055 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f134c14ac00 con 0x7f134c0c3820 2024-04-24T23:34:51.860 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.860+0000 7f1354276640 1 -- 172.21.15.157:0/2604492055 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f134c0ab8f0 con 0x7f134c0c3820 2024-04-24T23:34:51.860 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.860+0000 7f1342ffd640 1 -- 172.21.15.157:0/2604492055 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) ==== 84+0+65 (secure 0 0 0) 0x7f1344079ef0 con 0x7f134c0c3820 2024-04-24T23:34:51.860 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:34:51.860 INFO:teuthology.orchestra.run.smithi157.stdout:[ 2024-04-24T23:34:51.860 INFO:teuthology.orchestra.run.smithi157.stdout: { 2024-04-24T23:34:51.860 INFO:teuthology.orchestra.run.smithi157.stdout: "poolnum": 1, 2024-04-24T23:34:51.860 INFO:teuthology.orchestra.run.smithi157.stdout: "poolname": ".mgr" 2024-04-24T23:34:51.861 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-24T23:34:51.861 INFO:teuthology.orchestra.run.smithi157.stdout:] 2024-04-24T23:34:51.862 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.862+0000 7f1354276640 1 -- 172.21.15.157:0/2604492055 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f133006bda0 msgr2=0x7f133006e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:51.862 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.862+0000 7f1354276640 1 --2- 172.21.15.157:0/2604492055 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f133006bda0 0x7f133006e260 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f134c153ea0 tx=0x7f1348009040 comp rx=0 tx=0).stop 2024-04-24T23:34:51.862 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.862+0000 7f1354276640 1 -- 172.21.15.157:0/2604492055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f134c0c3820 msgr2=0x7f134c1eb020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:51.863 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.862+0000 7f1354276640 1 --2- 172.21.15.157:0/2604492055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f134c0c3820 0x7f134c1eb020 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1344034040 tx=0x7f1344004a40 comp rx=0 tx=0).stop 2024-04-24T23:34:51.863 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.862+0000 7f1354276640 1 -- 172.21.15.157:0/2604492055 shutdown_connections 2024-04-24T23:34:51.863 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.862+0000 7f1354276640 1 --2- 172.21.15.157:0/2604492055 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f133006bda0 0x7f133006e260 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:51.863 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.862+0000 7f1354276640 1 --2- 172.21.15.157:0/2604492055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f134c0c3820 0x7f134c1eb020 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:51.863 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.862+0000 7f1354276640 1 --2- 172.21.15.157:0/2604492055 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f134c0c2110 0x7f134c1eaae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:51.863 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.862+0000 7f1354276640 1 --2- 172.21.15.157:0/2604492055 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f134c06b3d0 0x7f134c14a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:51.863 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.862+0000 7f1354276640 1 -- 172.21.15.157:0/2604492055 >> 172.21.15.157:0/2604492055 conn(0x7f134c0c71f0 msgr2=0x7f134c06c040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:51.863 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.862+0000 7f1354276640 1 -- 172.21.15.157:0/2604492055 shutdown_connections 2024-04-24T23:34:51.863 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.862+0000 7f1354276640 1 -- 172.21.15.157:0/2604492055 wait complete. 2024-04-24T23:34:51.872 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-24T23:34:51.873 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-24T23:34:51.974 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.974+0000 7f3a6756a640 1 Processor -- start 2024-04-24T23:34:51.974 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.974+0000 7f3a6756a640 1 -- start start 2024-04-24T23:34:51.975 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.975+0000 7f3a6756a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3a6014f3a0 0x7f3a6006d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:51.975 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.975+0000 7f3a6756a640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a600c7860 0x7f3a600c7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:51.975 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.975+0000 7f3a6756a640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a600c75f0 con 0x7f3a600afd00 2024-04-24T23:34:51.976 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.975+0000 7f3a6756a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f3a60155ef0 con 0x7f3a6014f3a0 2024-04-24T23:34:51.976 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.975+0000 7f3a6756a640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3a601561d0 con 0x7f3a600c7860 2024-04-24T23:34:51.976 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.975+0000 7f3a65306640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3a6014f3a0 0x7f3a6006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:51.976 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.975+0000 7f3a65306640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3a6014f3a0 0x7f3a6006d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:41370/0 (socket says 172.21.15.157:41370) 2024-04-24T23:34:51.976 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.975+0000 7f3a65306640 1 -- 172.21.15.157:0/391191397 learned_addr learned my addr 172.21.15.157:0/391191397 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:34:51.976 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.975+0000 7f3a65b07640 1 --2- 172.21.15.157:0/391191397 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a600c7860 0x7f3a600c7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:51.976 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.976+0000 7f3a65306640 1 -- 172.21.15.157:0/391191397 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a600c7860 msgr2=0x7f3a600c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:51.976 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.976+0000 7f3a65306640 1 --2- 172.21.15.157:0/391191397 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a600c7860 0x7f3a600c7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:51.976 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.976+0000 7f3a65306640 1 -- 172.21.15.157:0/391191397 >> v1:172.21.15.169:6789/0 conn(0x7f3a600afd00 legacy=0x7f3a600b0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:51.976 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.976+0000 7f3a65306640 1 -- 172.21.15.157:0/391191397 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a600c57d0 con 0x7f3a6014f3a0 2024-04-24T23:34:51.978 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.976+0000 7f3a65306640 1 --2- 172.21.15.157:0/391191397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a6014f3a0 0x7f3a6006d530 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c009e50 tx=0x7f3a5c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b96f26137a9fc789 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:51.978 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.976+0000 7f3a57fff640 1 -- 172.21.15.157:0/391191397 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c036070 con 0x7f3a6014f3a0 2024-04-24T23:34:51.978 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.976+0000 7f3a57fff640 1 -- 172.21.15.157:0/391191397 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a5c031600 con 0x7f3a6014f3a0 2024-04-24T23:34:51.978 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.976+0000 7f3a57fff640 1 -- 172.21.15.157:0/391191397 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c0319d0 con 0x7f3a6014f3a0 2024-04-24T23:34:51.978 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.977+0000 7f3a6756a640 1 -- 172.21.15.157:0/391191397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a6014f3a0 msgr2=0x7f3a6006d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:51.978 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.977+0000 7f3a6756a640 1 --2- 172.21.15.157:0/391191397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a6014f3a0 0x7f3a6006d530 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c009e50 tx=0x7f3a5c02a710 comp rx=0 tx=0).stop 2024-04-24T23:34:51.978 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.977+0000 7f3a6756a640 1 -- 172.21.15.157:0/391191397 shutdown_connections 2024-04-24T23:34:51.978 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.977+0000 7f3a6756a640 1 --2- 172.21.15.157:0/391191397 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a600c7860 0x7f3a600c7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:51.978 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.977+0000 7f3a6756a640 1 --2- 172.21.15.157:0/391191397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a6014f3a0 0x7f3a6006d530 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:51.978 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.977+0000 7f3a6756a640 1 -- 172.21.15.157:0/391191397 >> 172.21.15.157:0/391191397 conn(0x7f3a60066ea0 msgr2=0x7f3a600672b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:51.978 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.977+0000 7f3a6756a640 1 -- 172.21.15.157:0/391191397 shutdown_connections 2024-04-24T23:34:51.978 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.977+0000 7f3a6756a640 1 -- 172.21.15.157:0/391191397 wait complete. 2024-04-24T23:34:51.979 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.978+0000 7f3a6756a640 1 Processor -- start 2024-04-24T23:34:51.979 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.978+0000 7f3a6756a640 1 -- start start 2024-04-24T23:34:51.981 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.979+0000 7f3a6756a640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a600afd00 0x7f3a6014df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:51.981 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.979+0000 7f3a6756a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a600c7860 0x7f3a60147150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:51.981 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.979+0000 7f3a6756a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a6014f3a0 0x7f3a60147690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:51.981 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.979+0000 7f3a6756a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3a6014f780 con 0x7f3a600c7860 2024-04-24T23:34:51.981 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.979+0000 7f3a6756a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3a600c5d90 con 0x7f3a6014f3a0 2024-04-24T23:34:51.981 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.979+0000 7f3a6756a640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3a600b08a0 con 0x7f3a600afd00 2024-04-24T23:34:51.981 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.979+0000 7f3a64b05640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a600c7860 0x7f3a60147150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:51.981 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.979+0000 7f3a64b05640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a600c7860 0x7f3a60147150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:41378/0 (socket says 172.21.15.157:41378) 2024-04-24T23:34:51.981 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.979+0000 7f3a64b05640 1 -- 172.21.15.157:0/1652837112 learned_addr learned my addr 172.21.15.157:0/1652837112 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:34:51.981 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.979+0000 7f3a65306640 1 --2- 172.21.15.157:0/1652837112 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a600afd00 0x7f3a6014df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:51.981 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.979+0000 7f3a65b07640 1 --2- 172.21.15.157:0/1652837112 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a6014f3a0 0x7f3a60147690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:51.982 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.979+0000 7f3a64b05640 1 -- 172.21.15.157:0/1652837112 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a600afd00 msgr2=0x7f3a6014df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:51.982 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.979+0000 7f3a64b05640 1 --2- 172.21.15.157:0/1652837112 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a600afd00 0x7f3a6014df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:51.982 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.979+0000 7f3a64b05640 1 -- 172.21.15.157:0/1652837112 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a6014f3a0 msgr2=0x7f3a60147690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:51.982 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.979+0000 7f3a64b05640 1 --2- 172.21.15.157:0/1652837112 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a6014f3a0 0x7f3a60147690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:51.982 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.979+0000 7f3a64b05640 1 -- 172.21.15.157:0/1652837112 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a600bf3a0 con 0x7f3a600c7860 2024-04-24T23:34:51.982 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.979+0000 7f3a65b07640 1 --2- 172.21.15.157:0/1652837112 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a6014f3a0 0x7f3a60147690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:34:51.982 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.979+0000 7f3a64b05640 1 --2- 172.21.15.157:0/1652837112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a600c7860 0x7f3a60147150 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3a5000c480 tx=0x7f3a5000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:51.982 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.979+0000 7f3a55ffb640 1 -- 172.21.15.157:0/1652837112 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a500052d0 con 0x7f3a600c7860 2024-04-24T23:34:51.982 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.979+0000 7f3a55ffb640 1 -- 172.21.15.157:0/1652837112 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a50005470 con 0x7f3a600c7860 2024-04-24T23:34:51.982 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.980+0000 7f3a6756a640 1 -- 172.21.15.157:0/1652837112 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a601e1610 con 0x7f3a600c7860 2024-04-24T23:34:51.982 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.980+0000 7f3a55ffb640 1 -- 172.21.15.157:0/1652837112 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a50012870 con 0x7f3a600c7860 2024-04-24T23:34:51.982 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.980+0000 7f3a6756a640 1 -- 172.21.15.157:0/1652837112 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a6014e470 con 0x7f3a600c7860 2024-04-24T23:34:51.983 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.981+0000 7f3a6756a640 1 -- 172.21.15.157:0/1652837112 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a600681c0 con 0x7f3a600c7860 2024-04-24T23:34:51.983 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.981+0000 7f3a55ffb640 1 -- 172.21.15.157:0/1652837112 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7f3a50012a10 con 0x7f3a600c7860 2024-04-24T23:34:51.983 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.981+0000 7f3a55ffb640 1 --2- 172.21.15.157:0/1652837112 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f3a4406bda0 0x7f3a4406e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:51.983 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.982+0000 7f3a55ffb640 1 -- 172.21.15.157:0/1652837112 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f3a5008a550 con 0x7f3a600c7860 2024-04-24T23:34:51.983 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.982+0000 7f3a65306640 1 --2- 172.21.15.157:0/1652837112 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f3a4406bda0 0x7f3a4406e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:51.983 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.982+0000 7f3a65306640 1 --2- 172.21.15.157:0/1652837112 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f3a4406bda0 0x7f3a4406e260 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c033000 tx=0x7f3a5c009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:51.985 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:51.986+0000 7f3a55ffb640 1 -- 172.21.15.157:0/1652837112 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3a500593c0 con 0x7f3a600c7860 2024-04-24T23:34:52.159 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.159+0000 7f3a6756a640 1 -- 172.21.15.157:0/1652837112 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f3a601dfa80 con 0x7f3a600c7860 2024-04-24T23:34:52.338 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.338+0000 7f3a55ffb640 1 -- 172.21.15.157:0/1652837112 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) ==== 138+0+0 (secure 0 0 0) 0x7f3a50058cc0 con 0x7f3a600c7860 2024-04-24T23:34:52.338 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_metadata' created 2024-04-24T23:34:52.342 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.342+0000 7f3a6756a640 1 -- 172.21.15.157:0/1652837112 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f3a4406bda0 msgr2=0x7f3a4406e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:52.343 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.342+0000 7f3a6756a640 1 --2- 172.21.15.157:0/1652837112 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f3a4406bda0 0x7f3a4406e260 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c033000 tx=0x7f3a5c009000 comp rx=0 tx=0).stop 2024-04-24T23:34:52.343 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.342+0000 7f3a6756a640 1 -- 172.21.15.157:0/1652837112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a600c7860 msgr2=0x7f3a60147150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:52.343 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.342+0000 7f3a6756a640 1 --2- 172.21.15.157:0/1652837112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a600c7860 0x7f3a60147150 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3a5000c480 tx=0x7f3a5000c940 comp rx=0 tx=0).stop 2024-04-24T23:34:52.344 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.343+0000 7f3a6756a640 1 -- 172.21.15.157:0/1652837112 shutdown_connections 2024-04-24T23:34:52.344 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.343+0000 7f3a6756a640 1 --2- 172.21.15.157:0/1652837112 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f3a4406bda0 0x7f3a4406e260 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:52.344 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.343+0000 7f3a6756a640 1 --2- 172.21.15.157:0/1652837112 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a6014f3a0 0x7f3a60147690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:52.344 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.343+0000 7f3a6756a640 1 --2- 172.21.15.157:0/1652837112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a600c7860 0x7f3a60147150 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:52.344 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.343+0000 7f3a6756a640 1 --2- 172.21.15.157:0/1652837112 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a600afd00 0x7f3a6014df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:52.344 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.343+0000 7f3a6756a640 1 -- 172.21.15.157:0/1652837112 >> 172.21.15.157:0/1652837112 conn(0x7f3a60066ea0 msgr2=0x7f3a601e1ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:52.344 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.344+0000 7f3a6756a640 1 -- 172.21.15.157:0/1652837112 shutdown_connections 2024-04-24T23:34:52.344 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.344+0000 7f3a6756a640 1 -- 172.21.15.157:0/1652837112 wait complete. 2024-04-24T23:34:52.357 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-24T23:34:52.461 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.462+0000 7f584fda5640 1 Processor -- start 2024-04-24T23:34:52.462 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.462+0000 7f584fda5640 1 -- start start 2024-04-24T23:34:52.464 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.463+0000 7f584fda5640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5848073100 0x7f58480c2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:52.464 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.463+0000 7f584fda5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5848067270 0x7f584815fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:52.464 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.463+0000 7f584fda5640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58481553a0 con 0x7f58480c14f0 2024-04-24T23:34:52.464 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.463+0000 7f584fda5640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f5848155660 con 0x7f5848067270 2024-04-24T23:34:52.464 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.463+0000 7f584fda5640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f58480c0730 con 0x7f5848073100 2024-04-24T23:34:52.464 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.463+0000 7f584e342640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5848067270 0x7f584815fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:52.464 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.463+0000 7f584e342640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5848067270 0x7f584815fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:41392/0 (socket says 172.21.15.157:41392) 2024-04-24T23:34:52.464 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.463+0000 7f584e342640 1 -- 172.21.15.157:0/714020335 learned_addr learned my addr 172.21.15.157:0/714020335 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:34:52.464 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.463+0000 7f584d340640 1 --1- 172.21.15.157:0/714020335 >> v1:172.21.15.157:6789/0 conn(0x7f58480c14f0 0x7f584806b160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:60644/0 (socket says 172.21.15.157:60644) 2024-04-24T23:34:52.464 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.463+0000 7f584db41640 1 --2- 172.21.15.157:0/714020335 >> v2:172.21.15.169:3300/0 conn(0x7f5848073100 0x7f58480c2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:52.464 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.463+0000 7f584e342640 1 -- 172.21.15.157:0/714020335 >> v2:172.21.15.169:3300/0 conn(0x7f5848073100 msgr2=0x7f58480c2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:52.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.463+0000 7f584e342640 1 --2- 172.21.15.157:0/714020335 >> v2:172.21.15.169:3300/0 conn(0x7f5848073100 0x7f58480c2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:52.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.463+0000 7f584e342640 1 -- 172.21.15.157:0/714020335 >> v1:172.21.15.157:6789/0 conn(0x7f58480c14f0 legacy=0x7f584806b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:52.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.463+0000 7f584cb3f640 1 -- 172.21.15.157:0/714020335 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 548862727 0 0) 0x7f58481553a0 con 0x7f58480c14f0 2024-04-24T23:34:52.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.463+0000 7f584e342640 1 -- 172.21.15.157:0/714020335 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58480c0a10 con 0x7f5848067270 2024-04-24T23:34:52.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.463+0000 7f584db41640 1 --2- 172.21.15.157:0/714020335 >> v2:172.21.15.169:3300/0 conn(0x7f5848073100 0x7f58480c2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:34:52.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.464+0000 7f584e342640 1 --2- 172.21.15.157:0/714020335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5848067270 0x7f584815fd20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f58480c82b0 tx=0x7f584002a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcd6b5dcf5247e5e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:52.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.464+0000 7f584cb3f640 1 -- 172.21.15.157:0/714020335 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5840037070 con 0x7f5848067270 2024-04-24T23:34:52.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.464+0000 7f584cb3f640 1 -- 172.21.15.157:0/714020335 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5840032440 con 0x7f5848067270 2024-04-24T23:34:52.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.464+0000 7f584cb3f640 1 -- 172.21.15.157:0/714020335 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5840032810 con 0x7f5848067270 2024-04-24T23:34:52.468 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.465+0000 7f584fda5640 1 -- 172.21.15.157:0/714020335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5848067270 msgr2=0x7f584815fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:52.469 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.465+0000 7f584fda5640 1 --2- 172.21.15.157:0/714020335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5848067270 0x7f584815fd20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f58480c82b0 tx=0x7f584002a150 comp rx=0 tx=0).stop 2024-04-24T23:34:52.469 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.465+0000 7f584fda5640 1 -- 172.21.15.157:0/714020335 shutdown_connections 2024-04-24T23:34:52.469 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.465+0000 7f584fda5640 1 --2- 172.21.15.157:0/714020335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5848067270 0x7f584815fd20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:52.469 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.465+0000 7f584fda5640 1 --2- 172.21.15.157:0/714020335 >> v2:172.21.15.169:3300/0 conn(0x7f5848073100 0x7f58480c2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:52.469 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.465+0000 7f584fda5640 1 -- 172.21.15.157:0/714020335 >> 172.21.15.157:0/714020335 conn(0x7f58480c3d60 msgr2=0x7f58480c4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:52.469 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.465+0000 7f584fda5640 1 -- 172.21.15.157:0/714020335 shutdown_connections 2024-04-24T23:34:52.469 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.465+0000 7f584fda5640 1 -- 172.21.15.157:0/714020335 wait complete. 2024-04-24T23:34:52.469 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.466+0000 7f584fda5640 1 Processor -- start 2024-04-24T23:34:52.469 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.466+0000 7f584fda5640 1 -- start start 2024-04-24T23:34:52.469 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.466+0000 7f584fda5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5848073100 0x7f58481eeb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:52.469 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.466+0000 7f584fda5640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58480c14f0 0x7f584806bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:52.469 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.466+0000 7f584fda5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5848143840 0x7f5848143cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:52.469 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.466+0000 7f584fda5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f58480711a0 con 0x7f5848143840 2024-04-24T23:34:52.469 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.466+0000 7f584fda5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f58480c0fd0 con 0x7f5848073100 2024-04-24T23:34:52.470 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.466+0000 7f584fda5640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5848071320 con 0x7f58480c14f0 2024-04-24T23:34:52.470 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.467+0000 7f584e342640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5848143840 0x7f5848143cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:52.470 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.467+0000 7f584e342640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5848143840 0x7f5848143cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:41394/0 (socket says 172.21.15.157:41394) 2024-04-24T23:34:52.470 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.467+0000 7f584e342640 1 -- 172.21.15.157:0/1033009774 learned_addr learned my addr 172.21.15.157:0/1033009774 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:34:52.470 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.467+0000 7f584e342640 1 -- 172.21.15.157:0/1033009774 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58480c14f0 msgr2=0x7f584806bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:52.470 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.467+0000 7f584e342640 1 --2- 172.21.15.157:0/1033009774 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58480c14f0 0x7f584806bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:52.470 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.467+0000 7f584db41640 1 --2- 172.21.15.157:0/1033009774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5848073100 0x7f58481eeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:52.470 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.467+0000 7f584e342640 1 -- 172.21.15.157:0/1033009774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5848073100 msgr2=0x7f58481eeb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:52.470 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.467+0000 7f584e342640 1 --2- 172.21.15.157:0/1033009774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5848073100 0x7f58481eeb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:52.470 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.467+0000 7f584e342640 1 -- 172.21.15.157:0/1033009774 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58481ef0d0 con 0x7f5848143840 2024-04-24T23:34:52.470 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.467+0000 7f584e342640 1 --2- 172.21.15.157:0/1033009774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5848143840 0x7f5848143cf0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f58400090b0 tx=0x7f58400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:52.470 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.467+0000 7f583e7fc640 1 -- 172.21.15.157:0/1033009774 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5840037040 con 0x7f5848143840 2024-04-24T23:34:52.470 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.467+0000 7f583e7fc640 1 -- 172.21.15.157:0/1033009774 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5840042070 con 0x7f5848143840 2024-04-24T23:34:52.471 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.467+0000 7f584fda5640 1 -- 172.21.15.157:0/1033009774 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5848157bb0 con 0x7f5848143840 2024-04-24T23:34:52.471 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.467+0000 7f583e7fc640 1 -- 172.21.15.157:0/1033009774 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5840030bd0 con 0x7f5848143840 2024-04-24T23:34:52.471 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.467+0000 7f584fda5640 1 -- 172.21.15.157:0/1033009774 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5848158ac0 con 0x7f5848143840 2024-04-24T23:34:52.471 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.468+0000 7f584fda5640 1 -- 172.21.15.157:0/1033009774 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58480681c0 con 0x7f5848143840 2024-04-24T23:34:52.471 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.468+0000 7f583e7fc640 1 -- 172.21.15.157:0/1033009774 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5840047460 con 0x7f5848143840 2024-04-24T23:34:52.471 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.469+0000 7f583e7fc640 1 --2- 172.21.15.157:0/1033009774 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f582c06be30 0x7f582c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:52.471 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.469+0000 7f583e7fc640 1 -- 172.21.15.157:0/1033009774 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5227+0+0 (secure 0 0 0) 0x7f58400ab8d0 con 0x7f5848143840 2024-04-24T23:34:52.471 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.469+0000 7f584db41640 1 --2- 172.21.15.157:0/1033009774 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f582c06be30 0x7f582c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:52.471 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.469+0000 7f584db41640 1 --2- 172.21.15.157:0/1033009774 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f582c06be30 0x7f582c06e2f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5844008f80 tx=0x7f58440084d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:52.473 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.474+0000 7f583e7fc640 1 -- 172.21.15.157:0/1033009774 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f584007a4f0 con 0x7f5848143840 2024-04-24T23:34:52.648 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:52.649+0000 7f584fda5640 1 -- 172.21.15.157:0/1033009774 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f584806f740 con 0x7f5848143840 2024-04-24T23:34:53.340 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.341+0000 7f583e7fc640 1 -- 172.21.15.157:0/1033009774 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v19) ==== 171+0+0 (secure 0 0 0) 0x7f5840079df0 con 0x7f5848143840 2024-04-24T23:34:53.341 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_data' created 2024-04-24T23:34:53.346 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.345+0000 7f584fda5640 1 -- 172.21.15.157:0/1033009774 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f582c06be30 msgr2=0x7f582c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:53.346 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.345+0000 7f584fda5640 1 --2- 172.21.15.157:0/1033009774 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f582c06be30 0x7f582c06e2f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5844008f80 tx=0x7f58440084d0 comp rx=0 tx=0).stop 2024-04-24T23:34:53.346 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.345+0000 7f584fda5640 1 -- 172.21.15.157:0/1033009774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5848143840 msgr2=0x7f5848143cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:53.346 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.345+0000 7f584fda5640 1 --2- 172.21.15.157:0/1033009774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5848143840 0x7f5848143cf0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f58400090b0 tx=0x7f58400046c0 comp rx=0 tx=0).stop 2024-04-24T23:34:53.346 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.346+0000 7f584fda5640 1 -- 172.21.15.157:0/1033009774 shutdown_connections 2024-04-24T23:34:53.346 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.346+0000 7f584fda5640 1 --2- 172.21.15.157:0/1033009774 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f582c06be30 0x7f582c06e2f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:53.346 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.346+0000 7f584fda5640 1 --2- 172.21.15.157:0/1033009774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5848143840 0x7f5848143cf0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:53.346 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.346+0000 7f584fda5640 1 --2- 172.21.15.157:0/1033009774 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58480c14f0 0x7f584806bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:53.346 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.346+0000 7f584fda5640 1 --2- 172.21.15.157:0/1033009774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5848073100 0x7f58481eeb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:53.346 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.346+0000 7f584fda5640 1 -- 172.21.15.157:0/1033009774 >> 172.21.15.157:0/1033009774 conn(0x7f58480c3d60 msgr2=0x7f58480c2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:53.348 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.349+0000 7f584fda5640 1 -- 172.21.15.157:0/1033009774 shutdown_connections 2024-04-24T23:34:53.349 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.350+0000 7f584fda5640 1 -- 172.21.15.157:0/1033009774 wait complete. 2024-04-24T23:34:53.374 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-24T23:34:53.478 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.478+0000 7f4bf0aca640 1 Processor -- start 2024-04-24T23:34:53.478 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.478+0000 7f4bf0aca640 1 -- start start 2024-04-24T23:34:53.479 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.479+0000 7f4bf0aca640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4bec0b03e0 0x7f4bec1567e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:53.479 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.479+0000 7f4bf0aca640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4bec0c55d0 0x7f4bec0c5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:53.480 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.479+0000 7f4bf0aca640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bec0cb330 con 0x7f4bec1592f0 2024-04-24T23:34:53.480 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.479+0000 7f4bf0aca640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f4bec0b07c0 con 0x7f4bec0b03e0 2024-04-24T23:34:53.480 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.479+0000 7f4bf0aca640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f4bec156310 con 0x7f4bec0c55d0 2024-04-24T23:34:53.480 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.479+0000 7f4bea59c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4bec0b03e0 0x7f4bec1567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:53.480 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.479+0000 7f4bea59c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4bec0b03e0 0x7f4bec1567e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:41410/0 (socket says 172.21.15.157:41410) 2024-04-24T23:34:53.480 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.479+0000 7f4bea59c640 1 -- 172.21.15.157:0/918550416 learned_addr learned my addr 172.21.15.157:0/918550416 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:34:53.480 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.479+0000 7f4be9d9b640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f4bec1592f0 0x7f4bec155c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:60658/0 (socket says 172.21.15.157:60658) 2024-04-24T23:34:53.480 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.480+0000 7f4bead9d640 1 --2- 172.21.15.157:0/918550416 >> v2:172.21.15.169:3300/0 conn(0x7f4bec0c55d0 0x7f4bec0c5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:53.480 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.480+0000 7f4bea59c640 1 -- 172.21.15.157:0/918550416 >> v2:172.21.15.169:3300/0 conn(0x7f4bec0c55d0 msgr2=0x7f4bec0c5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:53.480 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.480+0000 7f4bea59c640 1 --2- 172.21.15.157:0/918550416 >> v2:172.21.15.169:3300/0 conn(0x7f4bec0c55d0 0x7f4bec0c5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:53.480 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.480+0000 7f4bea59c640 1 -- 172.21.15.157:0/918550416 >> v1:172.21.15.157:6789/0 conn(0x7f4bec1592f0 legacy=0x7f4bec155c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:53.480 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.480+0000 7f4bea59c640 1 -- 172.21.15.157:0/918550416 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bec0cb610 con 0x7f4bec0b03e0 2024-04-24T23:34:53.482 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.480+0000 7f4bea59c640 1 --2- 172.21.15.157:0/918550416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bec0b03e0 0x7f4bec1567e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4bec0c6af0 tx=0x7f4bdc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8bf7c3aa3ef88e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:53.482 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.481+0000 7f4be959a640 1 -- 172.21.15.157:0/918550416 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc036070 con 0x7f4bec0b03e0 2024-04-24T23:34:53.482 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.481+0000 7f4be959a640 1 -- 172.21.15.157:0/918550416 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bdc031600 con 0x7f4bec0b03e0 2024-04-24T23:34:53.482 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.481+0000 7f4be959a640 1 -- 172.21.15.157:0/918550416 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc0319d0 con 0x7f4bec0b03e0 2024-04-24T23:34:53.482 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.481+0000 7f4bf0aca640 1 -- 172.21.15.157:0/918550416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bec0b03e0 msgr2=0x7f4bec1567e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:53.482 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.481+0000 7f4bf0aca640 1 --2- 172.21.15.157:0/918550416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bec0b03e0 0x7f4bec1567e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4bec0c6af0 tx=0x7f4bdc02a6a0 comp rx=0 tx=0).stop 2024-04-24T23:34:53.482 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.481+0000 7f4bf0aca640 1 -- 172.21.15.157:0/918550416 shutdown_connections 2024-04-24T23:34:53.483 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.481+0000 7f4bf0aca640 1 --2- 172.21.15.157:0/918550416 >> v2:172.21.15.169:3300/0 conn(0x7f4bec0c55d0 0x7f4bec0c5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:53.483 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.481+0000 7f4bf0aca640 1 --2- 172.21.15.157:0/918550416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bec0b03e0 0x7f4bec1567e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:53.483 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.481+0000 7f4bf0aca640 1 -- 172.21.15.157:0/918550416 >> 172.21.15.157:0/918550416 conn(0x7f4bec067080 msgr2=0x7f4bec067490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:53.483 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.481+0000 7f4bf0aca640 1 -- 172.21.15.157:0/918550416 shutdown_connections 2024-04-24T23:34:53.483 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.481+0000 7f4bf0aca640 1 -- 172.21.15.157:0/918550416 wait complete. 2024-04-24T23:34:53.483 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.482+0000 7f4bf0aca640 1 Processor -- start 2024-04-24T23:34:53.483 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.483+0000 7f4bf0aca640 1 -- start start 2024-04-24T23:34:53.485 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.483+0000 7f4bf0aca640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4bec0b03e0 0x7f4bec0af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:53.486 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.483+0000 7f4bf0aca640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4bec0c55d0 0x7f4bec147c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:53.486 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.483+0000 7f4bf0aca640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bec1592f0 0x7f4bec1eb160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:53.486 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.483+0000 7f4bf0aca640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4bec1556e0 con 0x7f4bec1592f0 2024-04-24T23:34:53.486 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.483+0000 7f4bf0aca640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4bec155400 con 0x7f4bec0b03e0 2024-04-24T23:34:53.486 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.483+0000 7f4bf0aca640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4bec159780 con 0x7f4bec0c55d0 2024-04-24T23:34:53.486 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.483+0000 7f4bead9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bec1592f0 0x7f4bec1eb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:53.486 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.483+0000 7f4bead9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bec1592f0 0x7f4bec1eb160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:41420/0 (socket says 172.21.15.157:41420) 2024-04-24T23:34:53.486 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.483+0000 7f4bead9d640 1 -- 172.21.15.157:0/978366268 learned_addr learned my addr 172.21.15.157:0/978366268 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:34:53.486 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.483+0000 7f4be9d9b640 1 --2- 172.21.15.157:0/978366268 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4bec0c55d0 0x7f4bec147c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:53.486 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.483+0000 7f4bea59c640 1 --2- 172.21.15.157:0/978366268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4bec0b03e0 0x7f4bec0af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:53.486 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.484+0000 7f4bead9d640 1 -- 172.21.15.157:0/978366268 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4bec0c55d0 msgr2=0x7f4bec147c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:53.486 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.484+0000 7f4bead9d640 1 --2- 172.21.15.157:0/978366268 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4bec0c55d0 0x7f4bec147c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:53.487 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.484+0000 7f4bead9d640 1 -- 172.21.15.157:0/978366268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4bec0b03e0 msgr2=0x7f4bec0af7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:53.487 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.484+0000 7f4bead9d640 1 --2- 172.21.15.157:0/978366268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4bec0b03e0 0x7f4bec0af7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:53.487 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.484+0000 7f4bead9d640 1 -- 172.21.15.157:0/978366268 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bec148190 con 0x7f4bec1592f0 2024-04-24T23:34:53.487 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.484+0000 7f4bead9d640 1 --2- 172.21.15.157:0/978366268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bec1592f0 0x7f4bec1eb160 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4be400d830 tx=0x7f4be400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:53.487 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.484+0000 7f4bdaffd640 1 -- 172.21.15.157:0/978366268 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be400b660 con 0x7f4bec1592f0 2024-04-24T23:34:53.487 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.484+0000 7f4bdaffd640 1 -- 172.21.15.157:0/978366268 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4be4019070 con 0x7f4bec1592f0 2024-04-24T23:34:53.487 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.484+0000 7f4bf0aca640 1 -- 172.21.15.157:0/978366268 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bec0c9220 con 0x7f4bec1592f0 2024-04-24T23:34:53.487 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.484+0000 7f4bdaffd640 1 -- 172.21.15.157:0/978366268 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be400ba60 con 0x7f4bec1592f0 2024-04-24T23:34:53.487 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.484+0000 7f4bf0aca640 1 -- 172.21.15.157:0/978366268 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bec0c8f40 con 0x7f4bec1592f0 2024-04-24T23:34:53.487 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.485+0000 7f4bf0aca640 1 -- 172.21.15.157:0/978366268 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bec0c1ad0 con 0x7f4bec1592f0 2024-04-24T23:34:53.487 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.485+0000 7f4bdaffd640 1 -- 172.21.15.157:0/978366268 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4be401e460 con 0x7f4bec1592f0 2024-04-24T23:34:53.487 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.486+0000 7f4bdaffd640 1 --2- 172.21.15.157:0/978366268 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f4bcc06be30 0x7f4bcc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:53.487 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.486+0000 7f4bdaffd640 1 -- 172.21.15.157:0/978366268 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5609+0+0 (secure 0 0 0) 0x7f4be408b220 con 0x7f4bec1592f0 2024-04-24T23:34:53.487 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.486+0000 7f4bea59c640 1 --2- 172.21.15.157:0/978366268 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f4bcc06be30 0x7f4bcc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:53.488 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.487+0000 7f4bea59c640 1 --2- 172.21.15.157:0/978366268 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f4bcc06be30 0x7f4bcc06e2f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc033000 tx=0x7f4bdc009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:53.490 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.490+0000 7f4bdaffd640 1 -- 172.21.15.157:0/978366268 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4bec0c1ad0 con 0x7f4bec1592f0 2024-04-24T23:34:53.699 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:53.700+0000 7f4bf0aca640 1 -- 172.21.15.157:0/978366268 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f4bec0c4d50 con 0x7f4bec1592f0 2024-04-24T23:34:54.359 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.358+0000 7f4bdaffd640 1 -- 172.21.15.157:0/978366268 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f4be4059670 con 0x7f4bec1592f0 2024-04-24T23:34:54.360 INFO:teuthology.orchestra.run.smithi157.stderr:new fs with metadata pool 2 and data pool 3 2024-04-24T23:34:54.363 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.363+0000 7f4bf0aca640 1 -- 172.21.15.157:0/978366268 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f4bcc06be30 msgr2=0x7f4bcc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:54.364 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.364+0000 7f4bf0aca640 1 --2- 172.21.15.157:0/978366268 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f4bcc06be30 0x7f4bcc06e2f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc033000 tx=0x7f4bdc009000 comp rx=0 tx=0).stop 2024-04-24T23:34:54.364 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.364+0000 7f4bf0aca640 1 -- 172.21.15.157:0/978366268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bec1592f0 msgr2=0x7f4bec1eb160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:54.364 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.364+0000 7f4bf0aca640 1 --2- 172.21.15.157:0/978366268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bec1592f0 0x7f4bec1eb160 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4be400d830 tx=0x7f4be400dcf0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.367 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.366+0000 7f4bf0aca640 1 -- 172.21.15.157:0/978366268 shutdown_connections 2024-04-24T23:34:54.367 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.366+0000 7f4bf0aca640 1 --2- 172.21.15.157:0/978366268 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f4bcc06be30 0x7f4bcc06e2f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.367 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.366+0000 7f4bf0aca640 1 --2- 172.21.15.157:0/978366268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bec1592f0 0x7f4bec1eb160 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.367 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.366+0000 7f4bf0aca640 1 --2- 172.21.15.157:0/978366268 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4bec0c55d0 0x7f4bec147c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.367 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.366+0000 7f4bf0aca640 1 --2- 172.21.15.157:0/978366268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4bec0b03e0 0x7f4bec0af7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.367 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.366+0000 7f4bf0aca640 1 -- 172.21.15.157:0/978366268 >> 172.21.15.157:0/978366268 conn(0x7f4bec067080 msgr2=0x7f4bec067460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:54.371 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.371+0000 7f4bf0aca640 1 -- 172.21.15.157:0/978366268 shutdown_connections 2024-04-24T23:34:54.372 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.373+0000 7f4bf0aca640 1 -- 172.21.15.157:0/978366268 wait complete. 2024-04-24T23:34:54.388 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-24T23:34:54.388 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-24T23:34:54.491 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.491+0000 7f7973c5d640 1 Processor -- start 2024-04-24T23:34:54.491 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.492+0000 7f7973c5d640 1 -- start start 2024-04-24T23:34:54.495 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.492+0000 7f7973c5d640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f796c0c6640 0x7f796c152dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.495 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.492+0000 7f7973c5d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f796c14fdc0 con 0x7f796c0c7790 2024-04-24T23:34:54.495 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.492+0000 7f7973c5d640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f796c0c74b0 con 0x7f796c14f9e0 2024-04-24T23:34:54.495 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.492+0000 7f7973c5d640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f796c06f030 con 0x7f796c0c6640 2024-04-24T23:34:54.495 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.492+0000 7f79721fa640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f796c0c7790 0x7f796c0c3400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:37620/0 (socket says 172.21.15.157:37620) 2024-04-24T23:34:54.496 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.492+0000 7f79721fa640 1 -- 172.21.15.157:0/3991084458 learned_addr learned my addr 172.21.15.157:0/3991084458 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-24T23:34:54.496 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.492+0000 7f79711f8640 1 --2- 172.21.15.157:0/3991084458 >> v2:172.21.15.169:3300/0 conn(0x7f796c0c6640 0x7f796c152dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.496 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.492+0000 7f79709f7640 1 -- 172.21.15.157:0/3991084458 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1260311334 0 0) 0x7f7964001490 con 0x7f796c0c7790 2024-04-24T23:34:54.496 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.492+0000 7f79709f7640 1 -- 172.21.15.157:0/3991084458 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7958003520 con 0x7f796c0c7790 2024-04-24T23:34:54.496 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.492+0000 7f79709f7640 1 -- 172.21.15.157:0/3991084458 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 821130578 0 0) 0x7f796c0c74b0 con 0x7f796c14f9e0 2024-04-24T23:34:54.496 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.492+0000 7f79709f7640 1 -- 172.21.15.157:0/3991084458 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7964001490 con 0x7f796c14f9e0 2024-04-24T23:34:54.496 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.492+0000 7f79709f7640 1 -- 172.21.15.157:0/3991084458 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3167323627 0 0) 0x7f7958003520 con 0x7f796c0c7790 2024-04-24T23:34:54.496 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.492+0000 7f79709f7640 1 -- 172.21.15.157:0/3991084458 >> v2:172.21.15.169:3300/0 conn(0x7f796c0c6640 msgr2=0x7f796c152dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:54.496 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.492+0000 7f79709f7640 1 --2- 172.21.15.157:0/3991084458 >> v2:172.21.15.169:3300/0 conn(0x7f796c0c6640 0x7f796c152dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.496 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.492+0000 7f79709f7640 1 -- 172.21.15.157:0/3991084458 >> v1:172.21.15.169:6789/0 conn(0x7f796c14f9e0 legacy=0x7f796c06e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:54.496 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.492+0000 7f79709f7640 1 -- 172.21.15.157:0/3991084458 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f796c0c5b60 con 0x7f796c0c7790 2024-04-24T23:34:54.496 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.492+0000 7f79709f7640 1 -- 172.21.15.157:0/3991084458 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1480525340 0 0) 0x7f7964001bc0 con 0x7f796c0c7790 2024-04-24T23:34:54.496 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.492+0000 7f79711f8640 1 --2- 172.21.15.157:0/3991084458 >> v2:172.21.15.169:3300/0 conn(0x7f796c0c6640 0x7f796c152dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:34:54.496 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.492+0000 7f79709f7640 1 -- 172.21.15.157:0/3991084458 >> v1:172.21.15.157:6789/0 conn(0x7f796c0c7790 legacy=0x7f796c0c3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:54.497 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.492+0000 7f79709f7640 1 --2- 172.21.15.157:0/3991084458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7958003b40 0x7f7958003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.497 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.492+0000 7f79709f7640 1 --2- 172.21.15.157:0/3991084458 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7958004800 0x7f7958004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.497 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.492+0000 7f79709f7640 1 --2- 172.21.15.157:0/3991084458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79580051c0 0x7f795800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.497 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.492+0000 7f79709f7640 1 -- 172.21.15.157:0/3991084458 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f796c06f030 con 0x7f79580051c0 2024-04-24T23:34:54.497 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.492+0000 7f79709f7640 1 -- 172.21.15.157:0/3991084458 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f795800c0f0 con 0x7f7958003b40 2024-04-24T23:34:54.497 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.492+0000 7f79709f7640 1 -- 172.21.15.157:0/3991084458 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f795800c2f0 con 0x7f7958004800 2024-04-24T23:34:54.497 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.492+0000 7f79721fa640 1 --2- 172.21.15.157:0/3991084458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79580051c0 0x7f795800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.497 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.492+0000 7f79719f9640 1 --2- 172.21.15.157:0/3991084458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7958003b40 0x7f7958003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.497 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.492+0000 7f79711f8640 1 --2- 172.21.15.157:0/3991084458 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7958004800 0x7f7958004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.497 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.493+0000 7f79721fa640 1 -- 172.21.15.157:0/3991084458 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7958004800 msgr2=0x7f7958004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:54.497 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.493+0000 7f79721fa640 1 --2- 172.21.15.157:0/3991084458 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7958004800 0x7f7958004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.497 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.493+0000 7f79721fa640 1 -- 172.21.15.157:0/3991084458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7958003b40 msgr2=0x7f7958003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:54.497 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.493+0000 7f79721fa640 1 --2- 172.21.15.157:0/3991084458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7958003b40 0x7f7958003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.498 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.493+0000 7f79721fa640 1 -- 172.21.15.157:0/3991084458 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7958003520 con 0x7f79580051c0 2024-04-24T23:34:54.498 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.493+0000 7f79721fa640 1 -- 172.21.15.157:0/3991084458 reap_dead start 2024-04-24T23:34:54.498 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.493+0000 7f79721fa640 1 --2- 172.21.15.157:0/3991084458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79580051c0 0x7f795800b9c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f796400a590 tx=0x7f7958005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af3b5f6608fd866b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:54.498 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.493+0000 7f79709f7640 1 -- 172.21.15.157:0/3991084458 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796400a5d0 con 0x7f79580051c0 2024-04-24T23:34:54.498 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.493+0000 7f79709f7640 1 -- 172.21.15.157:0/3991084458 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7964031070 con 0x7f79580051c0 2024-04-24T23:34:54.498 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.493+0000 7f79709f7640 1 -- 172.21.15.157:0/3991084458 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7964036470 con 0x7f79580051c0 2024-04-24T23:34:54.498 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.494+0000 7f7973c5d640 1 -- 172.21.15.157:0/3991084458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79580051c0 msgr2=0x7f795800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:54.498 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.494+0000 7f7973c5d640 1 --2- 172.21.15.157:0/3991084458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79580051c0 0x7f795800b9c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f796400a590 tx=0x7f7958005830 comp rx=0 tx=0).stop 2024-04-24T23:34:54.498 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.494+0000 7f7973c5d640 1 -- 172.21.15.157:0/3991084458 shutdown_connections 2024-04-24T23:34:54.498 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.494+0000 7f7973c5d640 1 --2- 172.21.15.157:0/3991084458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79580051c0 0x7f795800b9c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.498 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.494+0000 7f7973c5d640 1 -- 172.21.15.157:0/3991084458 >> 172.21.15.157:0/3991084458 conn(0x7f796c0a96a0 msgr2=0x7f796c0a9a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:54.498 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.494+0000 7f7973c5d640 1 -- 172.21.15.157:0/3991084458 shutdown_connections 2024-04-24T23:34:54.498 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.494+0000 7f7973c5d640 1 -- 172.21.15.157:0/3991084458 wait complete. 2024-04-24T23:34:54.501 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.495+0000 7f7973c5d640 1 Processor -- start 2024-04-24T23:34:54.501 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.495+0000 7f7973c5d640 1 -- start start 2024-04-24T23:34:54.501 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.496+0000 7f7973c5d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79580051c0 0x7f796c0acda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.501 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.496+0000 7f7973c5d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f796c0ab960 0x7f796c0c0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.501 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.496+0000 7f7973c5d640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f796c0ac710 0x7f796c0abdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.501 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.496+0000 7f7973c5d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f796c0c6120 con 0x7f796c0ab960 2024-04-24T23:34:54.501 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.496+0000 7f7973c5d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f796c0a7460 con 0x7f79580051c0 2024-04-24T23:34:54.502 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.496+0000 7f7973c5d640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f796c072020 con 0x7f796c0ac710 2024-04-24T23:34:54.502 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.496+0000 7f79711f8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f796c0ab960 0x7f796c0c0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.502 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.496+0000 7f79711f8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f796c0ab960 0x7f796c0c0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:59842/0 (socket says 172.21.15.157:59842) 2024-04-24T23:34:54.502 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.496+0000 7f79711f8640 1 -- 172.21.15.157:0/1085429085 learned_addr learned my addr 172.21.15.157:0/1085429085 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:34:54.502 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.496+0000 7f79721fa640 1 --2- 172.21.15.157:0/1085429085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f796c0ac710 0x7f796c0abdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.502 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.496+0000 7f79719f9640 1 --2- 172.21.15.157:0/1085429085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79580051c0 0x7f796c0acda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.502 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.496+0000 7f79711f8640 1 -- 172.21.15.157:0/1085429085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f796c0ac710 msgr2=0x7f796c0abdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:54.502 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.496+0000 7f79711f8640 1 --2- 172.21.15.157:0/1085429085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f796c0ac710 0x7f796c0abdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.502 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.496+0000 7f79711f8640 1 -- 172.21.15.157:0/1085429085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79580051c0 msgr2=0x7f796c0acda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:54.502 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.496+0000 7f79711f8640 1 --2- 172.21.15.157:0/1085429085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79580051c0 0x7f796c0acda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.502 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.496+0000 7f79711f8640 1 -- 172.21.15.157:0/1085429085 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f796c1ebeb0 con 0x7f796c0ab960 2024-04-24T23:34:54.502 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.496+0000 7f79711f8640 1 --2- 172.21.15.157:0/1085429085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f796c0ab960 0x7f796c0c0f90 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f795c011a90 tx=0x7f795c011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:54.503 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.496+0000 7f79627fc640 1 -- 172.21.15.157:0/1085429085 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795c00fb50 con 0x7f796c0ab960 2024-04-24T23:34:54.503 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.497+0000 7f79627fc640 1 -- 172.21.15.157:0/1085429085 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f795c00fcf0 con 0x7f796c0ab960 2024-04-24T23:34:54.503 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.497+0000 7f79627fc640 1 -- 172.21.15.157:0/1085429085 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795c00fea0 con 0x7f796c0ab960 2024-04-24T23:34:54.503 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.497+0000 7f7973c5d640 1 -- 172.21.15.157:0/1085429085 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f796c1537f0 con 0x7f796c0ab960 2024-04-24T23:34:54.503 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.497+0000 7f7973c5d640 1 -- 172.21.15.157:0/1085429085 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f796c153ad0 con 0x7f796c0ab960 2024-04-24T23:34:54.503 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.498+0000 7f7973c5d640 1 -- 172.21.15.157:0/1085429085 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f796c0681c0 con 0x7f796c0ab960 2024-04-24T23:34:54.503 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.498+0000 7f79627fc640 1 -- 172.21.15.157:0/1085429085 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f795c012040 con 0x7f796c0ab960 2024-04-24T23:34:54.503 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.499+0000 7f79627fc640 1 --2- 172.21.15.157:0/1085429085 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f795006be30 0x7f795006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.503 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.499+0000 7f79627fc640 1 -- 172.21.15.157:0/1085429085 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f795c08b140 con 0x7f796c0ab960 2024-04-24T23:34:54.503 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.499+0000 7f79719f9640 1 --2- 172.21.15.157:0/1085429085 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f795006be30 0x7f795006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.503 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.499+0000 7f79719f9640 1 --2- 172.21.15.157:0/1085429085 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f795006be30 0x7f795006e2f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f796c14d730 tx=0x7f7968009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:54.504 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.503+0000 7f79627fc640 1 -- 172.21.15.157:0/1085429085 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f795c019030 con 0x7f796c0ab960 2024-04-24T23:34:54.686 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:54.686+0000 7f7973c5d640 1 -- 172.21.15.157:0/1085429085 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f796c0c20d0 con 0x7f796c0ab960 2024-04-24T23:34:55.352 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.352+0000 7f79627fc640 1 -- 172.21.15.157:0/1085429085 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v21) ==== 157+0+0 (secure 0 0 0) 0x7f795c059530 con 0x7f796c0ab960 2024-04-24T23:34:55.356 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.355+0000 7f7973c5d640 1 -- 172.21.15.157:0/1085429085 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f795006be30 msgr2=0x7f795006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:55.356 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.355+0000 7f7973c5d640 1 --2- 172.21.15.157:0/1085429085 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f795006be30 0x7f795006e2f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f796c14d730 tx=0x7f7968009040 comp rx=0 tx=0).stop 2024-04-24T23:34:55.356 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.355+0000 7f7973c5d640 1 -- 172.21.15.157:0/1085429085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f796c0ab960 msgr2=0x7f796c0c0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:55.356 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.355+0000 7f7973c5d640 1 --2- 172.21.15.157:0/1085429085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f796c0ab960 0x7f796c0c0f90 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f795c011a90 tx=0x7f795c011e90 comp rx=0 tx=0).stop 2024-04-24T23:34:55.356 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.355+0000 7f7973c5d640 1 -- 172.21.15.157:0/1085429085 shutdown_connections 2024-04-24T23:34:55.356 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.355+0000 7f7973c5d640 1 --2- 172.21.15.157:0/1085429085 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f795006be30 0x7f795006e2f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:55.356 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.355+0000 7f7973c5d640 1 --2- 172.21.15.157:0/1085429085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f796c0ac710 0x7f796c0abdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:55.356 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.355+0000 7f7973c5d640 1 --2- 172.21.15.157:0/1085429085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f796c0ab960 0x7f796c0c0f90 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:55.357 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.355+0000 7f7973c5d640 1 --2- 172.21.15.157:0/1085429085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79580051c0 0x7f796c0acda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:55.357 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.355+0000 7f7973c5d640 1 -- 172.21.15.157:0/1085429085 >> 172.21.15.157:0/1085429085 conn(0x7f796c0a96a0 msgr2=0x7f796c06e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:55.357 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.355+0000 7f7973c5d640 1 -- 172.21.15.157:0/1085429085 shutdown_connections 2024-04-24T23:34:55.357 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.355+0000 7f7973c5d640 1 -- 172.21.15.157:0/1085429085 wait complete. 2024-04-24T23:34:55.369 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-24T23:34:55.480 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.479+0000 7f0c72b09640 1 Processor -- start 2024-04-24T23:34:55.480 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.479+0000 7f0c72b09640 1 -- start start 2024-04-24T23:34:55.482 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.480+0000 7f0c72b09640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c6c152a30 0x7f0c6c0c5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:55.482 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.480+0000 7f0c72b09640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0c6c0c8bc0 0x7f0c6c155460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:55.482 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.480+0000 7f0c72b09640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c6c0c93b0 con 0x7f0c6c0c5b80 2024-04-24T23:34:55.482 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.480+0000 7f0c72b09640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f0c6c0c6e90 con 0x7f0c6c0c8bc0 2024-04-24T23:34:55.482 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.480+0000 7f0c72b09640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0c6c0c90d0 con 0x7f0c6c152a30 2024-04-24T23:34:55.482 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.480+0000 7f0c6bfff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0c6c0c8bc0 0x7f0c6c155460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:55.482 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.480+0000 7f0c6bfff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0c6c0c8bc0 0x7f0c6c155460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:59858/0 (socket says 172.21.15.157:59858) 2024-04-24T23:34:55.482 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.480+0000 7f0c6bfff640 1 -- 172.21.15.157:0/2040765522 learned_addr learned my addr 172.21.15.157:0/2040765522 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:34:55.483 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.481+0000 7f0c708a5640 1 --2- 172.21.15.157:0/2040765522 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c6c152a30 0x7f0c6c0c5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:55.483 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.481+0000 7f0c6bfff640 1 -- 172.21.15.157:0/2040765522 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c6c152a30 msgr2=0x7f0c6c0c5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:55.483 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.481+0000 7f0c6bfff640 1 --2- 172.21.15.157:0/2040765522 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c6c152a30 0x7f0c6c0c5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:55.483 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.481+0000 7f0c6bfff640 1 -- 172.21.15.157:0/2040765522 >> v1:172.21.15.169:6789/0 conn(0x7f0c6c0c5b80 legacy=0x7f0c6c1559a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:55.483 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.481+0000 7f0c6bfff640 1 -- 172.21.15.157:0/2040765522 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c6c158af0 con 0x7f0c6c0c8bc0 2024-04-24T23:34:55.483 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.481+0000 7f0c6bfff640 1 --2- 172.21.15.157:0/2040765522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c6c0c8bc0 0x7f0c6c155460 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0c60009870 tx=0x7f0c6002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f710fefe1d697efc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:55.483 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.481+0000 7f0c6b7fe640 1 -- 172.21.15.157:0/2040765522 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c60037070 con 0x7f0c6c0c8bc0 2024-04-24T23:34:55.483 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.481+0000 7f0c6b7fe640 1 -- 172.21.15.157:0/2040765522 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c6002ae10 con 0x7f0c6c0c8bc0 2024-04-24T23:34:55.483 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.481+0000 7f0c6b7fe640 1 -- 172.21.15.157:0/2040765522 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c60032660 con 0x7f0c6c0c8bc0 2024-04-24T23:34:55.486 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.482+0000 7f0c72b09640 1 -- 172.21.15.157:0/2040765522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c6c0c8bc0 msgr2=0x7f0c6c155460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:55.486 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.482+0000 7f0c72b09640 1 --2- 172.21.15.157:0/2040765522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c6c0c8bc0 0x7f0c6c155460 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0c60009870 tx=0x7f0c6002a100 comp rx=0 tx=0).stop 2024-04-24T23:34:55.486 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.482+0000 7f0c72b09640 1 -- 172.21.15.157:0/2040765522 shutdown_connections 2024-04-24T23:34:55.486 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.482+0000 7f0c72b09640 1 --2- 172.21.15.157:0/2040765522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c6c0c8bc0 0x7f0c6c155460 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:55.486 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.482+0000 7f0c72b09640 1 --2- 172.21.15.157:0/2040765522 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c6c152a30 0x7f0c6c0c5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:55.486 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.482+0000 7f0c72b09640 1 -- 172.21.15.157:0/2040765522 >> 172.21.15.157:0/2040765522 conn(0x7f0c6c0acc90 msgr2=0x7f0c6c0ad0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:55.486 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.482+0000 7f0c72b09640 1 -- 172.21.15.157:0/2040765522 shutdown_connections 2024-04-24T23:34:55.486 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.482+0000 7f0c72b09640 1 -- 172.21.15.157:0/2040765522 wait complete. 2024-04-24T23:34:55.486 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.483+0000 7f0c72b09640 1 Processor -- start 2024-04-24T23:34:55.486 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.483+0000 7f0c72b09640 1 -- start start 2024-04-24T23:34:55.486 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.484+0000 7f0c72b09640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c6c0c5b80 0x7f0c6c1eeda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:55.487 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.484+0000 7f0c72b09640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c6c0c8bc0 0x7f0c6c157d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:55.487 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.484+0000 7f0c72b09640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c6c152a30 0x7f0c6c0bec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:55.487 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.484+0000 7f0c72b09640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6c159390 con 0x7f0c6c0c8bc0 2024-04-24T23:34:55.487 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.484+0000 7f0c72b09640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6c1590b0 con 0x7f0c6c152a30 2024-04-24T23:34:55.487 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.484+0000 7f0c72b09640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0c6c06d170 con 0x7f0c6c0c5b80 2024-04-24T23:34:55.487 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.484+0000 7f0c6bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c6c0c8bc0 0x7f0c6c157d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:55.487 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.484+0000 7f0c6bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c6c0c8bc0 0x7f0c6c157d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:59860/0 (socket says 172.21.15.157:59860) 2024-04-24T23:34:55.487 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.484+0000 7f0c6bfff640 1 -- 172.21.15.157:0/3191349501 learned_addr learned my addr 172.21.15.157:0/3191349501 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:34:55.487 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.484+0000 7f0c708a5640 1 --2- 172.21.15.157:0/3191349501 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c6c0c5b80 0x7f0c6c1eeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:55.487 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.484+0000 7f0c710a6640 1 --2- 172.21.15.157:0/3191349501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c6c152a30 0x7f0c6c0bec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:55.487 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.484+0000 7f0c6bfff640 1 -- 172.21.15.157:0/3191349501 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c6c0c5b80 msgr2=0x7f0c6c1eeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:55.487 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.484+0000 7f0c6bfff640 1 --2- 172.21.15.157:0/3191349501 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c6c0c5b80 0x7f0c6c1eeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:55.487 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.484+0000 7f0c6bfff640 1 -- 172.21.15.157:0/3191349501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c6c152a30 msgr2=0x7f0c6c0bec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:55.487 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.484+0000 7f0c6bfff640 1 --2- 172.21.15.157:0/3191349501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c6c152a30 0x7f0c6c0bec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:55.488 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.484+0000 7f0c6bfff640 1 -- 172.21.15.157:0/3191349501 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c6c0bdc50 con 0x7f0c6c0c8bc0 2024-04-24T23:34:55.488 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.484+0000 7f0c6bfff640 1 --2- 172.21.15.157:0/3191349501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c6c0c8bc0 0x7f0c6c157d50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0c60000c00 tx=0x7f0c600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:55.488 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.485+0000 7f0c697fa640 1 -- 172.21.15.157:0/3191349501 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c60041070 con 0x7f0c6c0c8bc0 2024-04-24T23:34:55.488 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.485+0000 7f0c697fa640 1 -- 172.21.15.157:0/3191349501 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c6003c680 con 0x7f0c6c0c8bc0 2024-04-24T23:34:55.488 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.485+0000 7f0c72b09640 1 -- 172.21.15.157:0/3191349501 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c6c1ee2c0 con 0x7f0c6c0c8bc0 2024-04-24T23:34:55.488 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.485+0000 7f0c697fa640 1 -- 172.21.15.157:0/3191349501 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c60037040 con 0x7f0c6c0c8bc0 2024-04-24T23:34:55.488 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.485+0000 7f0c72b09640 1 -- 172.21.15.157:0/3191349501 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c6c1ee880 con 0x7f0c6c0c8bc0 2024-04-24T23:34:55.489 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.486+0000 7f0c72b09640 1 -- 172.21.15.157:0/3191349501 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c6c068230 con 0x7f0c6c0c8bc0 2024-04-24T23:34:55.489 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.486+0000 7f0c697fa640 1 -- 172.21.15.157:0/3191349501 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0c60044070 con 0x7f0c6c0c8bc0 2024-04-24T23:34:55.489 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.486+0000 7f0c697fa640 1 --2- 172.21.15.157:0/3191349501 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f0c5806be30 0x7f0c5806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:55.489 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.487+0000 7f0c697fa640 1 -- 172.21.15.157:0/3191349501 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 6007+0+0 (secure 0 0 0) 0x7f0c600ab8d0 con 0x7f0c6c0c8bc0 2024-04-24T23:34:55.489 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.487+0000 7f0c708a5640 1 --2- 172.21.15.157:0/3191349501 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f0c5806be30 0x7f0c5806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:55.489 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.487+0000 7f0c708a5640 1 --2- 172.21.15.157:0/3191349501 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f0c5806be30 0x7f0c5806e2f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c149bc0 tx=0x7f0c5c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:55.491 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.491+0000 7f0c697fa640 1 -- 172.21.15.157:0/3191349501 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0c60079cb0 con 0x7f0c6c0c8bc0 2024-04-24T23:34:55.664 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:55.663+0000 7f0c72b09640 1 -- 172.21.15.157:0/3191349501 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f0c6c06d3d0 con 0x7f0c6c0c8bc0 2024-04-24T23:34:57.377 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.376+0000 7f0c697fa640 1 -- 172.21.15.157:0/3191349501 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v23) ==== 228+0+0 (secure 0 0 0) 0x7f0c6007dfb0 con 0x7f0c6c0c8bc0 2024-04-24T23:34:57.377 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_data_ec' created 2024-04-24T23:34:57.382 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.381+0000 7f0c72b09640 1 -- 172.21.15.157:0/3191349501 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f0c5806be30 msgr2=0x7f0c5806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:57.382 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.381+0000 7f0c72b09640 1 --2- 172.21.15.157:0/3191349501 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f0c5806be30 0x7f0c5806e2f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c149bc0 tx=0x7f0c5c008040 comp rx=0 tx=0).stop 2024-04-24T23:34:57.382 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.381+0000 7f0c72b09640 1 -- 172.21.15.157:0/3191349501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c6c0c8bc0 msgr2=0x7f0c6c157d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:57.382 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.381+0000 7f0c72b09640 1 --2- 172.21.15.157:0/3191349501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c6c0c8bc0 0x7f0c6c157d50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0c60000c00 tx=0x7f0c600046c0 comp rx=0 tx=0).stop 2024-04-24T23:34:57.382 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.382+0000 7f0c72b09640 1 -- 172.21.15.157:0/3191349501 shutdown_connections 2024-04-24T23:34:57.383 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.382+0000 7f0c72b09640 1 --2- 172.21.15.157:0/3191349501 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f0c5806be30 0x7f0c5806e2f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:57.383 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.382+0000 7f0c72b09640 1 --2- 172.21.15.157:0/3191349501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c6c152a30 0x7f0c6c0bec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:57.383 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.382+0000 7f0c72b09640 1 --2- 172.21.15.157:0/3191349501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c6c0c8bc0 0x7f0c6c157d50 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:57.383 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.382+0000 7f0c72b09640 1 --2- 172.21.15.157:0/3191349501 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c6c0c5b80 0x7f0c6c1eeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:57.383 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.382+0000 7f0c72b09640 1 -- 172.21.15.157:0/3191349501 >> 172.21.15.157:0/3191349501 conn(0x7f0c6c0acc90 msgr2=0x7f0c6c150ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:57.383 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.382+0000 7f0c72b09640 1 -- 172.21.15.157:0/3191349501 shutdown_connections 2024-04-24T23:34:57.383 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.382+0000 7f0c72b09640 1 -- 172.21.15.157:0/3191349501 wait complete. 2024-04-24T23:34:57.398 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-24T23:34:57.500 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.500+0000 7f6456509640 1 Processor -- start 2024-04-24T23:34:57.500 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.500+0000 7f6456509640 1 -- start start 2024-04-24T23:34:57.503 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.500+0000 7f6456509640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64500ac6f0 0x7f6450152b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:57.503 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.500+0000 7f6456509640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f64500c5f00 0x7f64500c1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:57.503 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.500+0000 7f6456509640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f645014fa00 con 0x7f64500c57a0 2024-04-24T23:34:57.503 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.500+0000 7f6456509640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f64501526f0 con 0x7f64500c5f00 2024-04-24T23:34:57.503 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.500+0000 7f6456509640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f64500a61f0 con 0x7f64500ac6f0 2024-04-24T23:34:57.503 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.500+0000 7f644f7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f64500c5f00 0x7f64500c1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:57.503 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.500+0000 7f644f7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f64500c5f00 0x7f64500c1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:59874/0 (socket says 172.21.15.157:59874) 2024-04-24T23:34:57.503 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.500+0000 7f644f7fe640 1 -- 172.21.15.157:0/914625019 learned_addr learned my addr 172.21.15.157:0/914625019 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:34:57.503 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.500+0000 7f6454aa6640 1 --1- 172.21.15.157:0/914625019 >> v1:172.21.15.157:6789/0 conn(0x7f64500c57a0 0x7f645006e270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:37624/0 (socket says 172.21.15.157:37624) 2024-04-24T23:34:57.503 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.501+0000 7f644ffff640 1 --2- 172.21.15.157:0/914625019 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64500ac6f0 0x7f6450152b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:57.503 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.501+0000 7f644f7fe640 1 -- 172.21.15.157:0/914625019 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64500ac6f0 msgr2=0x7f6450152b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:57.503 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.501+0000 7f644f7fe640 1 --2- 172.21.15.157:0/914625019 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64500ac6f0 0x7f6450152b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:57.504 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.501+0000 7f644f7fe640 1 -- 172.21.15.157:0/914625019 >> v1:172.21.15.157:6789/0 conn(0x7f64500c57a0 legacy=0x7f645006e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:57.504 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.501+0000 7f644f7fe640 1 -- 172.21.15.157:0/914625019 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6450156130 con 0x7f64500c5f00 2024-04-24T23:34:57.504 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.501+0000 7f644ffff640 1 --2- 172.21.15.157:0/914625019 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64500ac6f0 0x7f6450152b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:34:57.504 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.501+0000 7f644f7fe640 1 --2- 172.21.15.157:0/914625019 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64500c5f00 0x7f64500c1190 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6444009870 tx=0x7f644402a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18f6019399eb4a64 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:57.504 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.501+0000 7f644effd640 1 -- 172.21.15.157:0/914625019 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6444037070 con 0x7f64500c5f00 2024-04-24T23:34:57.504 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.501+0000 7f644effd640 1 -- 172.21.15.157:0/914625019 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6444032440 con 0x7f64500c5f00 2024-04-24T23:34:57.504 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.502+0000 7f644effd640 1 -- 172.21.15.157:0/914625019 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6444032810 con 0x7f64500c5f00 2024-04-24T23:34:57.504 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.502+0000 7f6456509640 1 -- 172.21.15.157:0/914625019 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64500c5f00 msgr2=0x7f64500c1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:57.504 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.502+0000 7f6456509640 1 --2- 172.21.15.157:0/914625019 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64500c5f00 0x7f64500c1190 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6444009870 tx=0x7f644402a130 comp rx=0 tx=0).stop 2024-04-24T23:34:57.504 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.502+0000 7f6456509640 1 -- 172.21.15.157:0/914625019 shutdown_connections 2024-04-24T23:34:57.504 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.502+0000 7f6456509640 1 --2- 172.21.15.157:0/914625019 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64500c5f00 0x7f64500c1190 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:57.504 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.502+0000 7f6456509640 1 --2- 172.21.15.157:0/914625019 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64500ac6f0 0x7f6450152b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:57.504 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.502+0000 7f6456509640 1 -- 172.21.15.157:0/914625019 >> 172.21.15.157:0/914625019 conn(0x7f64500c3290 msgr2=0x7f64500c3670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:57.505 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.502+0000 7f6456509640 1 -- 172.21.15.157:0/914625019 shutdown_connections 2024-04-24T23:34:57.505 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.502+0000 7f6456509640 1 -- 172.21.15.157:0/914625019 wait complete. 2024-04-24T23:34:57.507 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.503+0000 7f6456509640 1 Processor -- start 2024-04-24T23:34:57.508 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.503+0000 7f6456509640 1 -- start start 2024-04-24T23:34:57.508 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.504+0000 7f6456509640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64500ac6f0 0x7f64501ddcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:57.508 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.504+0000 7f6456509640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64500c57a0 0x7f645014af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:57.508 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.504+0000 7f6456509640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64500c5f00 0x7f645014b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:57.508 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.504+0000 7f6456509640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6450152380 con 0x7f64500c5f00 2024-04-24T23:34:57.508 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.504+0000 7f6456509640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f64501520a0 con 0x7f64500c57a0 2024-04-24T23:34:57.508 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.504+0000 7f6456509640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f645006e980 con 0x7f64500ac6f0 2024-04-24T23:34:57.508 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.504+0000 7f6454aa6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64500c5f00 0x7f645014b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:57.508 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.504+0000 7f6454aa6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64500c5f00 0x7f645014b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:59888/0 (socket says 172.21.15.157:59888) 2024-04-24T23:34:57.508 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.504+0000 7f6454aa6640 1 -- 172.21.15.157:0/2987379056 learned_addr learned my addr 172.21.15.157:0/2987379056 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:34:57.508 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.504+0000 7f644ffff640 1 --2- 172.21.15.157:0/2987379056 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64500ac6f0 0x7f64501ddcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:57.508 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.504+0000 7f644f7fe640 1 --2- 172.21.15.157:0/2987379056 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64500c57a0 0x7f645014af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:57.509 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.504+0000 7f6454aa6640 1 -- 172.21.15.157:0/2987379056 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64500ac6f0 msgr2=0x7f64501ddcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:57.509 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.504+0000 7f6454aa6640 1 --2- 172.21.15.157:0/2987379056 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64500ac6f0 0x7f64501ddcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:57.509 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.504+0000 7f6454aa6640 1 -- 172.21.15.157:0/2987379056 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64500c57a0 msgr2=0x7f645014af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:57.509 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.504+0000 7f6454aa6640 1 --2- 172.21.15.157:0/2987379056 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64500c57a0 0x7f645014af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:57.509 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.504+0000 7f6454aa6640 1 -- 172.21.15.157:0/2987379056 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64500733b0 con 0x7f64500c5f00 2024-04-24T23:34:57.509 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.504+0000 7f644f7fe640 1 --2- 172.21.15.157:0/2987379056 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64500c57a0 0x7f645014af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:34:57.509 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.505+0000 7f6454aa6640 1 --2- 172.21.15.157:0/2987379056 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64500c5f00 0x7f645014b4d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f644800eb40 tx=0x7f644800c060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:57.509 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.505+0000 7f644cff9640 1 -- 172.21.15.157:0/2987379056 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6448019070 con 0x7f64500c5f00 2024-04-24T23:34:57.509 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.505+0000 7f644cff9640 1 -- 172.21.15.157:0/2987379056 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f644800cc40 con 0x7f64500c5f00 2024-04-24T23:34:57.509 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.505+0000 7f644cff9640 1 -- 172.21.15.157:0/2987379056 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6448014400 con 0x7f64500c5f00 2024-04-24T23:34:57.509 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.505+0000 7f6456509640 1 -- 172.21.15.157:0/2987379056 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64501dbfa0 con 0x7f64500c5f00 2024-04-24T23:34:57.509 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.505+0000 7f6456509640 1 -- 172.21.15.157:0/2987379056 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6450072e90 con 0x7f64500c5f00 2024-04-24T23:34:57.509 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.506+0000 7f6456509640 1 -- 172.21.15.157:0/2987379056 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6450067c70 con 0x7f64500c5f00 2024-04-24T23:34:57.509 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.506+0000 7f644cff9640 1 -- 172.21.15.157:0/2987379056 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f644801c070 con 0x7f64500c5f00 2024-04-24T23:34:57.510 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.507+0000 7f644cff9640 1 --2- 172.21.15.157:0/2987379056 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f643c06be30 0x7f643c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:57.510 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.507+0000 7f644cff9640 1 -- 172.21.15.157:0/2987379056 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6500+0+0 (secure 0 0 0) 0x7f644808cd40 con 0x7f64500c5f00 2024-04-24T23:34:57.510 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.507+0000 7f644ffff640 1 --2- 172.21.15.157:0/2987379056 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f643c06be30 0x7f643c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:57.510 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.507+0000 7f644ffff640 1 --2- 172.21.15.157:0/2987379056 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f643c06be30 0x7f643c06e2f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6450155290 tx=0x7f6440008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:57.511 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.511+0000 7f644cff9640 1 -- 172.21.15.157:0/2987379056 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f644805b460 con 0x7f64500c5f00 2024-04-24T23:34:57.683 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:57.682+0000 7f6456509640 1 -- 172.21.15.157:0/2987379056 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f64500681c0 con 0x7f64500c5f00 2024-04-24T23:34:58.379 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.378+0000 7f644cff9640 1 -- 172.21.15.157:0/2987379056 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v24) ==== 169+0+0 (secure 0 0 0) 0x7f644805ad60 con 0x7f64500c5f00 2024-04-24T23:34:58.379 INFO:teuthology.orchestra.run.smithi157.stderr:set pool 4 allow_ec_overwrites to true 2024-04-24T23:34:58.384 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.382+0000 7f6456509640 1 -- 172.21.15.157:0/2987379056 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f643c06be30 msgr2=0x7f643c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:58.384 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.382+0000 7f6456509640 1 --2- 172.21.15.157:0/2987379056 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f643c06be30 0x7f643c06e2f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6450155290 tx=0x7f6440008040 comp rx=0 tx=0).stop 2024-04-24T23:34:58.384 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.382+0000 7f6456509640 1 -- 172.21.15.157:0/2987379056 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64500c5f00 msgr2=0x7f645014b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:58.384 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.382+0000 7f6456509640 1 --2- 172.21.15.157:0/2987379056 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64500c5f00 0x7f645014b4d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f644800eb40 tx=0x7f644800c060 comp rx=0 tx=0).stop 2024-04-24T23:34:58.384 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.382+0000 7f6456509640 1 -- 172.21.15.157:0/2987379056 shutdown_connections 2024-04-24T23:34:58.384 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.382+0000 7f6456509640 1 --2- 172.21.15.157:0/2987379056 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f643c06be30 0x7f643c06e2f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:58.384 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.382+0000 7f6456509640 1 --2- 172.21.15.157:0/2987379056 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64500c5f00 0x7f645014b4d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:58.385 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.382+0000 7f6456509640 1 --2- 172.21.15.157:0/2987379056 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64500c57a0 0x7f645014af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:58.385 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.382+0000 7f6456509640 1 --2- 172.21.15.157:0/2987379056 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64500ac6f0 0x7f64501ddcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:58.385 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.382+0000 7f6456509640 1 -- 172.21.15.157:0/2987379056 >> 172.21.15.157:0/2987379056 conn(0x7f64500c3290 msgr2=0x7f64500c1190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:58.385 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.384+0000 7f6456509640 1 -- 172.21.15.157:0/2987379056 shutdown_connections 2024-04-24T23:34:58.390 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.384+0000 7f6456509640 1 -- 172.21.15.157:0/2987379056 wait complete. 2024-04-24T23:34:58.415 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-24T23:34:58.517 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.516+0000 7f1453389640 1 Processor -- start 2024-04-24T23:34:58.517 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.517+0000 7f1453389640 1 -- start start 2024-04-24T23:34:58.519 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.517+0000 7f1453389640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f144c0c4280 0x7f144c155d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:58.519 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.517+0000 7f1453389640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f144c06d5e0 0x7f144c1562d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:58.519 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.517+0000 7f1453389640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f144c06df40 0x7f144c0c6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:58.519 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.517+0000 7f1453389640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f144c0c81d0 con 0x7f144c0c4280 2024-04-24T23:34:58.520 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.517+0000 7f1453389640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f144c0c69d0 con 0x7f144c06df40 2024-04-24T23:34:58.520 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.517+0000 7f1453389640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f144c0c3770 con 0x7f144c06d5e0 2024-04-24T23:34:58.520 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.518+0000 7f1451125640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f144c0c4280 0x7f144c155d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:58.520 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.518+0000 7f1451125640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f144c0c4280 0x7f144c155d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:59898/0 (socket says 172.21.15.157:59898) 2024-04-24T23:34:58.520 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.518+0000 7f1451125640 1 -- 172.21.15.157:0/546327466 learned_addr learned my addr 172.21.15.157:0/546327466 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:34:58.520 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.518+0000 7f1451926640 1 --2- 172.21.15.157:0/546327466 >> v2:172.21.15.169:3300/0 conn(0x7f144c06df40 0x7f144c0c6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:58.520 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.518+0000 7f1451125640 1 -- 172.21.15.157:0/546327466 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f144c06d5e0 msgr2=0x7f144c1562d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:58.520 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.518+0000 7f1450924640 1 --2- 172.21.15.157:0/546327466 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f144c06d5e0 0x7f144c1562d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:58.520 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.518+0000 7f1451125640 1 --2- 172.21.15.157:0/546327466 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f144c06d5e0 0x7f144c1562d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:58.520 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.518+0000 7f1451125640 1 -- 172.21.15.157:0/546327466 >> v2:172.21.15.169:3300/0 conn(0x7f144c06df40 msgr2=0x7f144c0c6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:58.520 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.518+0000 7f1451125640 1 --2- 172.21.15.157:0/546327466 >> v2:172.21.15.169:3300/0 conn(0x7f144c06df40 0x7f144c0c6e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:58.520 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.518+0000 7f1451125640 1 -- 172.21.15.157:0/546327466 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f144c0c8420 con 0x7f144c0c4280 2024-04-24T23:34:58.521 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.518+0000 7f1451125640 1 --2- 172.21.15.157:0/546327466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f144c0c4280 0x7f144c155d90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f144c0c5270 tx=0x7f144402a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9af9aacd0e94b0c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:58.521 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.519+0000 7f144bfff640 1 -- 172.21.15.157:0/546327466 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1444036070 con 0x7f144c0c4280 2024-04-24T23:34:58.521 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.519+0000 7f144bfff640 1 -- 172.21.15.157:0/546327466 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1444031600 con 0x7f144c0c4280 2024-04-24T23:34:58.521 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.519+0000 7f144bfff640 1 -- 172.21.15.157:0/546327466 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14440319d0 con 0x7f144c0c4280 2024-04-24T23:34:58.524 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.519+0000 7f1453389640 1 -- 172.21.15.157:0/546327466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f144c0c4280 msgr2=0x7f144c155d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:58.524 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.519+0000 7f1453389640 1 --2- 172.21.15.157:0/546327466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f144c0c4280 0x7f144c155d90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f144c0c5270 tx=0x7f144402a570 comp rx=0 tx=0).stop 2024-04-24T23:34:58.524 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.519+0000 7f1453389640 1 -- 172.21.15.157:0/546327466 shutdown_connections 2024-04-24T23:34:58.524 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.519+0000 7f1453389640 1 --2- 172.21.15.157:0/546327466 >> v2:172.21.15.169:3300/0 conn(0x7f144c06df40 0x7f144c0c6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:58.524 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.519+0000 7f1453389640 1 --2- 172.21.15.157:0/546327466 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f144c06d5e0 0x7f144c1562d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:58.524 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.519+0000 7f1453389640 1 --2- 172.21.15.157:0/546327466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f144c0c4280 0x7f144c155d90 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:58.524 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.519+0000 7f1453389640 1 -- 172.21.15.157:0/546327466 >> 172.21.15.157:0/546327466 conn(0x7f144c0c5500 msgr2=0x7f144c0c5910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:58.525 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.519+0000 7f1453389640 1 -- 172.21.15.157:0/546327466 shutdown_connections 2024-04-24T23:34:58.525 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.519+0000 7f1453389640 1 -- 172.21.15.157:0/546327466 wait complete. 2024-04-24T23:34:58.525 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.520+0000 7f1453389640 1 Processor -- start 2024-04-24T23:34:58.525 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.520+0000 7f1453389640 1 -- start start 2024-04-24T23:34:58.525 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.521+0000 7f1453389640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f144c06d5e0 0x7f144c151cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:58.525 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.521+0000 7f1453389640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f144c06df40 0x7f144c0adb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:58.525 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.521+0000 7f1453389640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f144c0c4280 0x7f144c0ae090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:58.525 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.521+0000 7f1453389640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f144c0c4660 con 0x7f144c06d5e0 2024-04-24T23:34:58.525 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.521+0000 7f1453389640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f144c0c3400 con 0x7f144c06df40 2024-04-24T23:34:58.525 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.521+0000 7f1453389640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f144c0c47e0 con 0x7f144c0c4280 2024-04-24T23:34:58.525 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.521+0000 7f1451125640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f144c06d5e0 0x7f144c151cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:58.525 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.521+0000 7f1451125640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f144c06d5e0 0x7f144c151cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:59902/0 (socket says 172.21.15.157:59902) 2024-04-24T23:34:58.525 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.521+0000 7f1451125640 1 -- 172.21.15.157:0/939082001 learned_addr learned my addr 172.21.15.157:0/939082001 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:34:58.526 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.521+0000 7f1450924640 1 --2- 172.21.15.157:0/939082001 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f144c06df40 0x7f144c0adb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:58.526 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.521+0000 7f1451926640 1 --2- 172.21.15.157:0/939082001 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f144c0c4280 0x7f144c0ae090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:58.526 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.521+0000 7f1451125640 1 -- 172.21.15.157:0/939082001 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f144c0c4280 msgr2=0x7f144c0ae090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:58.526 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.521+0000 7f1451125640 1 --2- 172.21.15.157:0/939082001 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f144c0c4280 0x7f144c0ae090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:58.526 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.521+0000 7f1451125640 1 -- 172.21.15.157:0/939082001 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f144c06df40 msgr2=0x7f144c0adb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:58.526 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.521+0000 7f1451125640 1 --2- 172.21.15.157:0/939082001 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f144c06df40 0x7f144c0adb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:58.526 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.521+0000 7f1451125640 1 -- 172.21.15.157:0/939082001 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f144c1ead90 con 0x7f144c06d5e0 2024-04-24T23:34:58.526 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.521+0000 7f1450924640 1 --2- 172.21.15.157:0/939082001 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f144c06df40 0x7f144c0adb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:34:58.526 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.522+0000 7f1451125640 1 --2- 172.21.15.157:0/939082001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f144c06d5e0 0x7f144c151cf0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1444034040 tx=0x7f1444004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:58.526 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.522+0000 7f1449ffb640 1 -- 172.21.15.157:0/939082001 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144403f020 con 0x7f144c06d5e0 2024-04-24T23:34:58.526 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.522+0000 7f1449ffb640 1 -- 172.21.15.157:0/939082001 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1444037040 con 0x7f144c06d5e0 2024-04-24T23:34:58.526 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.522+0000 7f1453389640 1 -- 172.21.15.157:0/939082001 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f144c1eb070 con 0x7f144c06d5e0 2024-04-24T23:34:58.527 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.522+0000 7f1449ffb640 1 -- 172.21.15.157:0/939082001 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1444036070 con 0x7f144c06d5e0 2024-04-24T23:34:58.527 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.522+0000 7f1453389640 1 -- 172.21.15.157:0/939082001 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f144c1eb630 con 0x7f144c06d5e0 2024-04-24T23:34:58.527 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.523+0000 7f1453389640 1 -- 172.21.15.157:0/939082001 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f144c0681c0 con 0x7f144c06d5e0 2024-04-24T23:34:58.527 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.523+0000 7f1449ffb640 1 -- 172.21.15.157:0/939082001 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1444042020 con 0x7f144c06d5e0 2024-04-24T23:34:58.527 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.523+0000 7f1449ffb640 1 --2- 172.21.15.157:0/939082001 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f143806be30 0x7f143806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:58.527 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.524+0000 7f1449ffb640 1 -- 172.21.15.157:0/939082001 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6500+0+0 (secure 0 0 0) 0x7f14440ab7d0 con 0x7f144c06d5e0 2024-04-24T23:34:58.527 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.524+0000 7f1450924640 1 --2- 172.21.15.157:0/939082001 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f143806be30 0x7f143806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:58.527 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.524+0000 7f1450924640 1 --2- 172.21.15.157:0/939082001 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f143806be30 0x7f143806e2f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f144c0a7f90 tx=0x7f1440008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:58.528 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.528+0000 7f1449ffb640 1 -- 172.21.15.157:0/939082001 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1444049030 con 0x7f144c06d5e0 2024-04-24T23:34:58.743 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:58.742+0000 7f1453389640 1 -- 172.21.15.157:0/939082001 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f144c0a8720 con 0x7f144c06d5e0 2024-04-24T23:34:58.963 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:34:58.966 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:34:59.077 INFO:tasks.ceph.mds.b.smithi169.stderr:2024-04-24T23:34:59.077+0000 7f12d9d96640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T23:34:59.392 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.392+0000 7f1449ffb640 1 -- 172.21.15.157:0/939082001 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) ==== 137+0+0 (secure 0 0 0) 0x7f1444079ef0 con 0x7f144c06d5e0 2024-04-24T23:34:59.393 INFO:teuthology.orchestra.run.smithi157.stderr:added data pool 4 to fsmap 2024-04-24T23:34:59.397 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.396+0000 7f1453389640 1 -- 172.21.15.157:0/939082001 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f143806be30 msgr2=0x7f143806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:59.397 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.396+0000 7f1453389640 1 --2- 172.21.15.157:0/939082001 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f143806be30 0x7f143806e2f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f144c0a7f90 tx=0x7f1440008040 comp rx=0 tx=0).stop 2024-04-24T23:34:59.397 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.396+0000 7f1453389640 1 -- 172.21.15.157:0/939082001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f144c06d5e0 msgr2=0x7f144c151cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:59.397 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.396+0000 7f1453389640 1 --2- 172.21.15.157:0/939082001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f144c06d5e0 0x7f144c151cf0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1444034040 tx=0x7f1444004a40 comp rx=0 tx=0).stop 2024-04-24T23:34:59.397 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.396+0000 7f1453389640 1 -- 172.21.15.157:0/939082001 shutdown_connections 2024-04-24T23:34:59.397 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.396+0000 7f1453389640 1 --2- 172.21.15.157:0/939082001 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f143806be30 0x7f143806e2f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:59.397 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.396+0000 7f1453389640 1 --2- 172.21.15.157:0/939082001 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f144c0c4280 0x7f144c0ae090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:59.397 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.396+0000 7f1453389640 1 --2- 172.21.15.157:0/939082001 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f144c06df40 0x7f144c0adb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:59.397 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.396+0000 7f1453389640 1 --2- 172.21.15.157:0/939082001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f144c06d5e0 0x7f144c151cf0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:59.397 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.396+0000 7f1453389640 1 -- 172.21.15.157:0/939082001 >> 172.21.15.157:0/939082001 conn(0x7f144c0c5500 msgr2=0x7f144c0c7160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:59.397 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.396+0000 7f1453389640 1 -- 172.21.15.157:0/939082001 shutdown_connections 2024-04-24T23:34:59.397 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.396+0000 7f1453389640 1 -- 172.21.15.157:0/939082001 wait complete. 2024-04-24T23:34:59.409 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:34:59.509 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.509+0000 7f715f163640 1 Processor -- start 2024-04-24T23:34:59.509 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.509+0000 7f715f163640 1 -- start start 2024-04-24T23:34:59.513 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.509+0000 7f715f163640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f71580c4b20 0x7f71580c2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:59.513 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.509+0000 7f715f163640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f715806d3c0 con 0x7f715814b720 2024-04-24T23:34:59.513 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.509+0000 7f715f163640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f715806c400 con 0x7f7158073130 2024-04-24T23:34:59.513 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.509+0000 7f715f163640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f7158073510 con 0x7f71580c4b20 2024-04-24T23:34:59.513 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.509+0000 7f715d700640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f715814b720 0x7f715814bb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:37640/0 (socket says 172.21.15.157:37640) 2024-04-24T23:34:59.513 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.510+0000 7f715d700640 1 -- 172.21.15.157:0/3865569948 learned_addr learned my addr 172.21.15.157:0/3865569948 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-24T23:34:59.513 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.510+0000 7f71577fe640 1 -- 172.21.15.157:0/3865569948 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1274888491 0 0) 0x7f715806d3c0 con 0x7f715814b720 2024-04-24T23:34:59.513 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.510+0000 7f71577fe640 1 -- 172.21.15.157:0/3865569948 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7144003520 con 0x7f715814b720 2024-04-24T23:34:59.513 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.510+0000 7f7157fff640 1 --2- 172.21.15.157:0/3865569948 >> v2:172.21.15.169:3300/0 conn(0x7f71580c4b20 0x7f71580c2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:59.513 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.510+0000 7f71577fe640 1 -- 172.21.15.157:0/3865569948 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2392258780 0 0) 0x7f7144003520 con 0x7f715814b720 2024-04-24T23:34:59.513 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.510+0000 7f71577fe640 1 -- 172.21.15.157:0/3865569948 >> v2:172.21.15.169:3300/0 conn(0x7f71580c4b20 msgr2=0x7f71580c2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:59.513 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.510+0000 7f71577fe640 1 --2- 172.21.15.157:0/3865569948 >> v2:172.21.15.169:3300/0 conn(0x7f71580c4b20 0x7f71580c2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:59.514 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.510+0000 7f71577fe640 1 -- 172.21.15.157:0/3865569948 >> v1:172.21.15.169:6789/0 conn(0x7f7158073130 legacy=0x7f715806ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:59.514 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.510+0000 7f71577fe640 1 -- 172.21.15.157:0/3865569948 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f715806c6a0 con 0x7f715814b720 2024-04-24T23:34:59.514 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.510+0000 7f71577fe640 1 -- 172.21.15.157:0/3865569948 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1480525340 0 0) 0x7f7140001670 con 0x7f715814b720 2024-04-24T23:34:59.514 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.510+0000 7f71577fe640 1 -- 172.21.15.157:0/3865569948 >> v1:172.21.15.157:6789/0 conn(0x7f715814b720 legacy=0x7f715814bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:59.514 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.510+0000 7f71577fe640 1 --2- 172.21.15.157:0/3865569948 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71440039b0 0x7f7144003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:59.514 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.510+0000 7f71577fe640 1 --2- 172.21.15.157:0/3865569948 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7144004800 0x7f7144004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:59.514 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.510+0000 7f71577fe640 1 --2- 172.21.15.157:0/3865569948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71440052e0 0x7f714400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:59.514 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.510+0000 7f71577fe640 1 -- 172.21.15.157:0/3865569948 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7158073510 con 0x7f71440052e0 2024-04-24T23:34:59.514 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.510+0000 7f71577fe640 1 -- 172.21.15.157:0/3865569948 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f714400c190 con 0x7f7144004800 2024-04-24T23:34:59.514 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.510+0000 7f71577fe640 1 -- 172.21.15.157:0/3865569948 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f714400c390 con 0x7f71440039b0 2024-04-24T23:34:59.514 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.510+0000 7f715d700640 1 --2- 172.21.15.157:0/3865569948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71440052e0 0x7f714400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:59.514 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.510+0000 7f7157fff640 1 --2- 172.21.15.157:0/3865569948 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7144004800 0x7f7144004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:59.514 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.510+0000 7f715ceff640 1 --2- 172.21.15.157:0/3865569948 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71440039b0 0x7f7144003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:59.515 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.510+0000 7f715d700640 1 -- 172.21.15.157:0/3865569948 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71440039b0 msgr2=0x7f7144003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:59.515 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.510+0000 7f715d700640 1 --2- 172.21.15.157:0/3865569948 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71440039b0 0x7f7144003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:59.515 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.510+0000 7f715d700640 1 -- 172.21.15.157:0/3865569948 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7144004800 msgr2=0x7f7144004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:59.515 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.510+0000 7f715d700640 1 --2- 172.21.15.157:0/3865569948 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7144004800 0x7f7144004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:59.515 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.510+0000 7f715d700640 1 -- 172.21.15.157:0/3865569948 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7144003520 con 0x7f71440052e0 2024-04-24T23:34:59.515 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.511+0000 7f715d700640 1 -- 172.21.15.157:0/3865569948 reap_dead start 2024-04-24T23:34:59.515 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.511+0000 7f715d700640 1 --2- 172.21.15.157:0/3865569948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71440052e0 0x7f714400b9c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7140007fd0 tx=0x7f7144005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a7cf549d9a58b47 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:59.515 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.511+0000 7f71577fe640 1 -- 172.21.15.157:0/3865569948 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7140007d50 con 0x7f71440052e0 2024-04-24T23:34:59.515 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.511+0000 7f71577fe640 1 -- 172.21.15.157:0/3865569948 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71400058d0 con 0x7f71440052e0 2024-04-24T23:34:59.515 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.511+0000 7f71577fe640 1 -- 172.21.15.157:0/3865569948 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7140005ca0 con 0x7f71440052e0 2024-04-24T23:34:59.515 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.512+0000 7f715f163640 1 -- 172.21.15.157:0/3865569948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71440052e0 msgr2=0x7f714400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:59.515 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.512+0000 7f715f163640 1 --2- 172.21.15.157:0/3865569948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71440052e0 0x7f714400b9c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7140007fd0 tx=0x7f7144005830 comp rx=0 tx=0).stop 2024-04-24T23:34:59.515 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.512+0000 7f715f163640 1 -- 172.21.15.157:0/3865569948 shutdown_connections 2024-04-24T23:34:59.515 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.512+0000 7f715f163640 1 --2- 172.21.15.157:0/3865569948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71440052e0 0x7f714400b9c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:59.516 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.512+0000 7f715f163640 1 -- 172.21.15.157:0/3865569948 >> 172.21.15.157:0/3865569948 conn(0x7f71580a8720 msgr2=0x7f71580a8b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:59.516 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.512+0000 7f715f163640 1 -- 172.21.15.157:0/3865569948 shutdown_connections 2024-04-24T23:34:59.516 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.512+0000 7f715f163640 1 -- 172.21.15.157:0/3865569948 wait complete. 2024-04-24T23:34:59.519 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.513+0000 7f715f163640 1 Processor -- start 2024-04-24T23:34:59.519 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.513+0000 7f715f163640 1 -- start start 2024-04-24T23:34:59.519 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.514+0000 7f715f163640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71440052e0 0x7f715814d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:59.519 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.514+0000 7f715f163640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71581529e0 0x7f71581ea820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:59.519 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.514+0000 7f715f163640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71581ead60 0x7f7158154ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:59.519 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.514+0000 7f715f163640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f71580c2470 con 0x7f71440052e0 2024-04-24T23:34:59.519 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.514+0000 7f715f163640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f71580c2190 con 0x7f71581ead60 2024-04-24T23:34:59.519 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.514+0000 7f715f163640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f71580c2990 con 0x7f71581529e0 2024-04-24T23:34:59.519 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.514+0000 7f715ceff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71440052e0 0x7f715814d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:59.519 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.514+0000 7f715ceff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71440052e0 0x7f715814d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:59920/0 (socket says 172.21.15.157:59920) 2024-04-24T23:34:59.519 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.514+0000 7f715ceff640 1 -- 172.21.15.157:0/3681472653 learned_addr learned my addr 172.21.15.157:0/3681472653 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:34:59.520 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.514+0000 7f715d700640 1 --2- 172.21.15.157:0/3681472653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71581ead60 0x7f7158154ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:59.520 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.514+0000 7f7157fff640 1 --2- 172.21.15.157:0/3681472653 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71581529e0 0x7f71581ea820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:59.520 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.514+0000 7f715ceff640 1 -- 172.21.15.157:0/3681472653 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71581529e0 msgr2=0x7f71581ea820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:59.520 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.514+0000 7f715ceff640 1 --2- 172.21.15.157:0/3681472653 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71581529e0 0x7f71581ea820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:59.520 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.514+0000 7f715ceff640 1 -- 172.21.15.157:0/3681472653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71581ead60 msgr2=0x7f7158154ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:59.520 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.514+0000 7f715ceff640 1 --2- 172.21.15.157:0/3681472653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71581ead60 0x7f7158154ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:59.520 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.514+0000 7f715ceff640 1 -- 172.21.15.157:0/3681472653 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71580ac450 con 0x7f71440052e0 2024-04-24T23:34:59.520 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.514+0000 7f7157fff640 1 --2- 172.21.15.157:0/3681472653 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71581529e0 0x7f71581ea820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:34:59.520 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.514+0000 7f715ceff640 1 --2- 172.21.15.157:0/3681472653 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71440052e0 0x7f715814d000 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f715000f9b0 tx=0x7f715000fe70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:59.520 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.514+0000 7f71557fa640 1 -- 172.21.15.157:0/3681472653 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7150017020 con 0x7f71440052e0 2024-04-24T23:34:59.520 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.514+0000 7f71557fa640 1 -- 172.21.15.157:0/3681472653 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f715000d9a0 con 0x7f71440052e0 2024-04-24T23:34:59.520 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.514+0000 7f71557fa640 1 -- 172.21.15.157:0/3681472653 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715000dd90 con 0x7f71440052e0 2024-04-24T23:34:59.520 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.514+0000 7f715f163640 1 -- 172.21.15.157:0/3681472653 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71580ac730 con 0x7f71440052e0 2024-04-24T23:34:59.520 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.514+0000 7f715f163640 1 -- 172.21.15.157:0/3681472653 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f715814ed50 con 0x7f71440052e0 2024-04-24T23:34:59.520 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.515+0000 7f715f163640 1 -- 172.21.15.157:0/3681472653 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71580681c0 con 0x7f71440052e0 2024-04-24T23:34:59.521 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.516+0000 7f71557fa640 1 -- 172.21.15.157:0/3681472653 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f715001a020 con 0x7f71440052e0 2024-04-24T23:34:59.521 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.516+0000 7f71557fa640 1 --2- 172.21.15.157:0/3681472653 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f713806be60 0x7f713806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:59.521 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.516+0000 7f71557fa640 1 -- 172.21.15.157:0/3681472653 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f715008b450 con 0x7f71440052e0 2024-04-24T23:34:59.521 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.516+0000 7f7157fff640 1 --2- 172.21.15.157:0/3681472653 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f713806be60 0x7f713806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:59.521 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.517+0000 7f7157fff640 1 --2- 172.21.15.157:0/3681472653 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f713806be60 0x7f713806e320 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f71581eb8e0 tx=0x7f7148008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:59.521 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.520+0000 7f71557fa640 1 -- 172.21.15.157:0/3681472653 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7150021020 con 0x7f71440052e0 2024-04-24T23:34:59.732 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.731+0000 7f715f163640 1 -- 172.21.15.157:0/3681472653 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f71580c09e0 con 0x7f71440052e0 2024-04-24T23:34:59.732 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.732+0000 7f71557fa640 1 -- 172.21.15.157:0/3681472653 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5189 (secure 0 0 0) 0x7f7150059500 con 0x7f71440052e0 2024-04-24T23:34:59.732 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:34:59.732 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1276438765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1276438765},{"type":"v1","addr":"172.21.15.169:6837","nonce":1276438765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1398599086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":1398599086},{"type":"v1","addr":"172.21.15.157:6835","nonce":1398599086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4258,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3795840339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3795840339},{"type":"v1","addr":"172.21.15.157:6834","nonce":3795840339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:34:53.701085+0000","modified":"2024-04-24T23:34:58.744647+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6836/3782746080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6835","nonce":3782746080},{"type":"v1","addr":"172.21.15.169:6836","nonce":3782746080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4270,"qdb_cluster":[4270]},"id":1}]} 2024-04-24T23:34:59.733 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 7 2024-04-24T23:34:59.735 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.733+0000 7f715f163640 1 -- 172.21.15.157:0/3681472653 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f713806be60 msgr2=0x7f713806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:59.735 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.733+0000 7f715f163640 1 --2- 172.21.15.157:0/3681472653 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f713806be60 0x7f713806e320 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f71581eb8e0 tx=0x7f7148008040 comp rx=0 tx=0).stop 2024-04-24T23:34:59.735 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.733+0000 7f715f163640 1 -- 172.21.15.157:0/3681472653 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71440052e0 msgr2=0x7f715814d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:59.735 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.733+0000 7f715f163640 1 --2- 172.21.15.157:0/3681472653 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71440052e0 0x7f715814d000 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f715000f9b0 tx=0x7f715000fe70 comp rx=0 tx=0).stop 2024-04-24T23:34:59.735 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.734+0000 7f715f163640 1 -- 172.21.15.157:0/3681472653 shutdown_connections 2024-04-24T23:34:59.735 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.734+0000 7f715f163640 1 --2- 172.21.15.157:0/3681472653 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f713806be60 0x7f713806e320 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:59.735 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.734+0000 7f715f163640 1 --2- 172.21.15.157:0/3681472653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71581ead60 0x7f7158154ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:59.735 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.734+0000 7f715f163640 1 --2- 172.21.15.157:0/3681472653 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71581529e0 0x7f71581ea820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:59.735 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.734+0000 7f715f163640 1 --2- 172.21.15.157:0/3681472653 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71440052e0 0x7f715814d000 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:59.735 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.734+0000 7f715f163640 1 -- 172.21.15.157:0/3681472653 >> 172.21.15.157:0/3681472653 conn(0x7f71580a8720 msgr2=0x7f71580a6920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:59.735 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.734+0000 7f715f163640 1 -- 172.21.15.157:0/3681472653 shutdown_connections 2024-04-24T23:34:59.735 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.734+0000 7f715f163640 1 -- 172.21.15.157:0/3681472653 wait complete. 2024-04-24T23:34:59.745 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:34:59.842 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.841+0000 7f5b3ab63640 1 Processor -- start 2024-04-24T23:34:59.842 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.841+0000 7f5b3ab63640 1 -- start start 2024-04-24T23:34:59.845 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.842+0000 7f5b3ab63640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5b34152b40 0x7f5b340c5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:59.845 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.842+0000 7f5b3ab63640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b340be360 0x7f5b34155350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:59.845 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.842+0000 7f5b3ab63640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b340c8bb0 con 0x7f5b340c5b70 2024-04-24T23:34:59.845 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.842+0000 7f5b3ab63640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f5b340c8e90 con 0x7f5b34152b40 2024-04-24T23:34:59.845 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.842+0000 7f5b3ab63640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5b34159480 con 0x7f5b340be360 2024-04-24T23:34:59.845 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.842+0000 7f5b388ff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5b34152b40 0x7f5b340c5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:59.845 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.842+0000 7f5b39100640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f5b340c5b70 0x7f5b34155890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:37656/0 (socket says 172.21.15.157:37656) 2024-04-24T23:34:59.845 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.842+0000 7f5b39100640 1 -- 172.21.15.157:0/2640491562 learned_addr learned my addr 172.21.15.157:0/2640491562 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-24T23:34:59.845 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.842+0000 7f5b33fff640 1 --2- 172.21.15.157:0/2640491562 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b340be360 0x7f5b34155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:59.845 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.843+0000 7f5b388ff640 1 -- 172.21.15.157:0/2640491562 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b340be360 msgr2=0x7f5b34155350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:59.845 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.843+0000 7f5b388ff640 1 --2- 172.21.15.157:0/2640491562 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b340be360 0x7f5b34155350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:59.846 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.843+0000 7f5b388ff640 1 -- 172.21.15.157:0/2640491562 >> v1:172.21.15.157:6789/0 conn(0x7f5b340c5b70 legacy=0x7f5b34155890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:59.846 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.843+0000 7f5b337fe640 1 -- 172.21.15.157:0/2640491562 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3572502134 0 0) 0x7f5b340c8bb0 con 0x7f5b340c5b70 2024-04-24T23:34:59.846 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.843+0000 7f5b388ff640 1 -- 172.21.15.157:0/2640491562 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b34159760 con 0x7f5b34152b40 2024-04-24T23:34:59.846 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.843+0000 7f5b388ff640 1 --2- 172.21.15.157:0/2640491562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b34152b40 0x7f5b340c5630 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5b240097a0 tx=0x7f5b2402a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de370fcfe905768d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:59.846 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.843+0000 7f5b337fe640 1 -- 172.21.15.157:0/2640491562 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b24037070 con 0x7f5b34152b40 2024-04-24T23:34:59.846 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.843+0000 7f5b337fe640 1 -- 172.21.15.157:0/2640491562 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b2402adb0 con 0x7f5b34152b40 2024-04-24T23:34:59.846 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.843+0000 7f5b337fe640 1 -- 172.21.15.157:0/2640491562 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b24032660 con 0x7f5b34152b40 2024-04-24T23:34:59.846 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.844+0000 7f5b3ab63640 1 -- 172.21.15.157:0/2640491562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b34152b40 msgr2=0x7f5b340c5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:59.846 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.844+0000 7f5b3ab63640 1 --2- 172.21.15.157:0/2640491562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b34152b40 0x7f5b340c5630 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5b240097a0 tx=0x7f5b2402a060 comp rx=0 tx=0).stop 2024-04-24T23:34:59.846 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.844+0000 7f5b3ab63640 1 -- 172.21.15.157:0/2640491562 shutdown_connections 2024-04-24T23:34:59.846 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.844+0000 7f5b3ab63640 1 --2- 172.21.15.157:0/2640491562 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b340be360 0x7f5b34155350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:59.846 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.844+0000 7f5b3ab63640 1 --2- 172.21.15.157:0/2640491562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b34152b40 0x7f5b340c5630 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:59.847 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.844+0000 7f5b3ab63640 1 -- 172.21.15.157:0/2640491562 >> 172.21.15.157:0/2640491562 conn(0x7f5b340672c0 msgr2=0x7f5b340676d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:59.847 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.844+0000 7f5b3ab63640 1 -- 172.21.15.157:0/2640491562 shutdown_connections 2024-04-24T23:34:59.847 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.844+0000 7f5b3ab63640 1 -- 172.21.15.157:0/2640491562 wait complete. 2024-04-24T23:34:59.850 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.845+0000 7f5b3ab63640 1 Processor -- start 2024-04-24T23:34:59.850 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.845+0000 7f5b3ab63640 1 -- start start 2024-04-24T23:34:59.850 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.846+0000 7f5b3ab63640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b340be360 0x7f5b340be7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:59.850 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.846+0000 7f5b3ab63640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b340c5b70 0x7f5b341ef110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:59.850 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.846+0000 7f5b3ab63640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b34152b40 0x7f5b341577e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:59.850 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.846+0000 7f5b3ab63640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5b340caa00 con 0x7f5b34152b40 2024-04-24T23:34:59.850 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.846+0000 7f5b3ab63640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5b340cb210 con 0x7f5b340be360 2024-04-24T23:34:59.850 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.846+0000 7f5b3ab63640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5b340cba40 con 0x7f5b340c5b70 2024-04-24T23:34:59.850 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.846+0000 7f5b39100640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b34152b40 0x7f5b341577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:59.850 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.846+0000 7f5b39100640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b34152b40 0x7f5b341577e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:59946/0 (socket says 172.21.15.157:59946) 2024-04-24T23:34:59.850 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.846+0000 7f5b39100640 1 -- 172.21.15.157:0/641199612 learned_addr learned my addr 172.21.15.157:0/641199612 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:34:59.851 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.846+0000 7f5b33fff640 1 --2- 172.21.15.157:0/641199612 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b340c5b70 0x7f5b341ef110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:59.851 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.846+0000 7f5b388ff640 1 --2- 172.21.15.157:0/641199612 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b340be360 0x7f5b340be7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:59.851 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.846+0000 7f5b39100640 1 -- 172.21.15.157:0/641199612 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b340c5b70 msgr2=0x7f5b341ef110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:59.851 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.846+0000 7f5b39100640 1 --2- 172.21.15.157:0/641199612 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b340c5b70 0x7f5b341ef110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:59.851 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.846+0000 7f5b39100640 1 -- 172.21.15.157:0/641199612 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b340be360 msgr2=0x7f5b340be7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:59.851 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.846+0000 7f5b39100640 1 --2- 172.21.15.157:0/641199612 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b340be360 0x7f5b340be7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:59.851 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.846+0000 7f5b39100640 1 -- 172.21.15.157:0/641199612 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b340bf770 con 0x7f5b34152b40 2024-04-24T23:34:59.851 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.846+0000 7f5b33fff640 1 --2- 172.21.15.157:0/641199612 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b340c5b70 0x7f5b341ef110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:34:59.851 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.846+0000 7f5b388ff640 1 --2- 172.21.15.157:0/641199612 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b340be360 0x7f5b340be7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:34:59.851 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.847+0000 7f5b39100640 1 --2- 172.21.15.157:0/641199612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b34152b40 0x7f5b341577e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c00c7a0 tx=0x7f5b2c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:59.851 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.847+0000 7f5b317fa640 1 -- 172.21.15.157:0/641199612 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c018070 con 0x7f5b34152b40 2024-04-24T23:34:59.851 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.847+0000 7f5b317fa640 1 -- 172.21.15.157:0/641199612 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b2c00a640 con 0x7f5b34152b40 2024-04-24T23:34:59.852 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.847+0000 7f5b317fa640 1 -- 172.21.15.157:0/641199612 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c00aa70 con 0x7f5b34152b40 2024-04-24T23:34:59.852 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.847+0000 7f5b3ab63640 1 -- 172.21.15.157:0/641199612 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b340acdb0 con 0x7f5b34152b40 2024-04-24T23:34:59.852 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.847+0000 7f5b3ab63640 1 -- 172.21.15.157:0/641199612 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b340ad370 con 0x7f5b34152b40 2024-04-24T23:34:59.852 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.848+0000 7f5b3ab63640 1 -- 172.21.15.157:0/641199612 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b340c5070 con 0x7f5b34152b40 2024-04-24T23:34:59.852 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.848+0000 7f5b317fa640 1 -- 172.21.15.157:0/641199612 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5b2c01b070 con 0x7f5b34152b40 2024-04-24T23:34:59.852 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.849+0000 7f5b317fa640 1 --2- 172.21.15.157:0/641199612 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f5b2006be30 0x7f5b2006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:59.852 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.849+0000 7f5b317fa640 1 -- 172.21.15.157:0/641199612 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f5b2c08a6e0 con 0x7f5b34152b40 2024-04-24T23:34:59.852 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.849+0000 7f5b388ff640 1 --2- 172.21.15.157:0/641199612 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f5b2006be30 0x7f5b2006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:59.852 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.849+0000 7f5b388ff640 1 --2- 172.21.15.157:0/641199612 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f5b2006be30 0x7f5b2006e2f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5b24034000 tx=0x7f5b2402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:59.853 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:34:59.853+0000 7f5b317fa640 1 -- 172.21.15.157:0/641199612 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5b2c058de0 con 0x7f5b34152b40 2024-04-24T23:35:00.027 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.027+0000 7f5b3ab63640 1 -- 172.21.15.157:0/641199612 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5b3414e640 con 0x7f5b34152b40 2024-04-24T23:35:00.029 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.028+0000 7f5b317fa640 1 -- 172.21.15.157:0/641199612 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18538 (secure 0 0 0) 0x7f5b2c0586e0 con 0x7f5b34152b40 2024-04-24T23:35:00.029 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:35:00.029 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":25,"fsid":"223685fb-5b64-46a5-9857-3791c9c07df2","created":"2024-04-24T23:34:37.504202+0000","modified":"2024-04-24T23:34:58.744500+0000","last_up_change":"2024-04-24T23:34:47.584963+0000","last_in_change":"2024-04-24T23:34:39.632954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:34:47.796655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:34:52.177208+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:34:52.667022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:34:56.395517+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"95d14ac8-fccf-4261-be8e-edafa746f629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6809","nonce":4003822160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6811","nonce":4003822160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6815","nonce":4003822160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6813","nonce":4003822160}]},"public_addr":"172.21.15.157:6809/4003822160","cluster_addr":"172.21.15.157:6811/4003822160","heartbeat_back_addr":"172.21.15.157:6815/4003822160","heartbeat_front_addr":"172.21.15.157:6813/4003822160","state":["exists","up"]},{"osd":1,"uuid":"7be79da9-8eec-4990-9178-71d602e39d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6817","nonce":1557575576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6819","nonce":1557575576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6823","nonce":1557575576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6821","nonce":1557575576}]},"public_addr":"172.21.15.157:6817/1557575576","cluster_addr":"172.21.15.157:6819/1557575576","heartbeat_back_addr":"172.21.15.157:6823/1557575576","heartbeat_front_addr":"172.21.15.157:6821/1557575576","state":["exists","up"]},{"osd":2,"uuid":"0127ed00-b7c3-466d-bfb0-01b0fd315f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6825","nonce":2793948092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6827","nonce":2793948092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6831","nonce":2793948092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6829","nonce":2793948092}]},"public_addr":"172.21.15.157:6825/2793948092","cluster_addr":"172.21.15.157:6827/2793948092","heartbeat_back_addr":"172.21.15.157:6831/2793948092","heartbeat_front_addr":"172.21.15.157:6829/2793948092","state":["exists","up"]},{"osd":3,"uuid":"f2331acc-aa37-433e-868e-3561b6039562","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6801","nonce":422758549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6803","nonce":422758549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6807","nonce":422758549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6805","nonce":422758549}]},"public_addr":"172.21.15.157:6801/422758549","cluster_addr":"172.21.15.157:6803/422758549","heartbeat_back_addr":"172.21.15.157:6807/422758549","heartbeat_front_addr":"172.21.15.157:6805/422758549","state":["exists","up"]},{"osd":4,"uuid":"1998854b-7398-43f5-ab7d-79fa0debcf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6801","nonce":321910099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6803","nonce":321910099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6807","nonce":321910099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6805","nonce":321910099}]},"public_addr":"172.21.15.169:6801/321910099","cluster_addr":"172.21.15.169:6803/321910099","heartbeat_back_addr":"172.21.15.169:6807/321910099","heartbeat_front_addr":"172.21.15.169:6805/321910099","state":["exists","up"]},{"osd":5,"uuid":"c37153ff-9b2b-48b9-a684-2c82b6f39f15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6825","nonce":243321711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6827","nonce":243321711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6831","nonce":243321711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6829","nonce":243321711}]},"public_addr":"172.21.15.169:6825/243321711","cluster_addr":"172.21.15.169:6827/243321711","heartbeat_back_addr":"172.21.15.169:6831/243321711","heartbeat_front_addr":"172.21.15.169:6829/243321711","state":["exists","up"]},{"osd":6,"uuid":"8db79e1c-f36b-49cb-8d8a-a20c2aec57c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6809","nonce":3441420785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6811","nonce":3441420785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6815","nonce":3441420785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6813","nonce":3441420785}]},"public_addr":"172.21.15.169:6809/3441420785","cluster_addr":"172.21.15.169:6811/3441420785","heartbeat_back_addr":"172.21.15.169:6815/3441420785","heartbeat_front_addr":"172.21.15.169:6813/3441420785","state":["exists","up"]},{"osd":7,"uuid":"928586e9-296c-4b8b-9b27-42f9f155dbdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6817","nonce":1663179020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6819","nonce":1663179020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6823","nonce":1663179020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6821","nonce":1663179020}]},"public_addr":"172.21.15.169:6817/1663179020","cluster_addr":"172.21.15.169:6819/1663179020","heartbeat_back_addr":"172.21.15.169:6823/1663179020","heartbeat_front_addr":"172.21.15.169:6821/1663179020","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.848039+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.826489+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.801120+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.727706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.824343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.791915+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.878073+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.810831+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:35:00.031 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.030+0000 7f5b3ab63640 1 -- 172.21.15.157:0/641199612 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f5b2006be30 msgr2=0x7f5b2006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:00.031 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.030+0000 7f5b3ab63640 1 --2- 172.21.15.157:0/641199612 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f5b2006be30 0x7f5b2006e2f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5b24034000 tx=0x7f5b2402e000 comp rx=0 tx=0).stop 2024-04-24T23:35:00.031 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.030+0000 7f5b3ab63640 1 -- 172.21.15.157:0/641199612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b34152b40 msgr2=0x7f5b341577e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:00.031 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.030+0000 7f5b3ab63640 1 --2- 172.21.15.157:0/641199612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b34152b40 0x7f5b341577e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c00c7a0 tx=0x7f5b2c00cc60 comp rx=0 tx=0).stop 2024-04-24T23:35:00.031 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.030+0000 7f5b3ab63640 1 -- 172.21.15.157:0/641199612 shutdown_connections 2024-04-24T23:35:00.031 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.030+0000 7f5b3ab63640 1 --2- 172.21.15.157:0/641199612 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f5b2006be30 0x7f5b2006e2f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:00.031 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.030+0000 7f5b3ab63640 1 --2- 172.21.15.157:0/641199612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b34152b40 0x7f5b341577e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:00.032 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.030+0000 7f5b3ab63640 1 --2- 172.21.15.157:0/641199612 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b340c5b70 0x7f5b341ef110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:00.032 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.030+0000 7f5b3ab63640 1 --2- 172.21.15.157:0/641199612 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b340be360 0x7f5b340be7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:00.032 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.030+0000 7f5b3ab63640 1 -- 172.21.15.157:0/641199612 >> 172.21.15.157:0/641199612 conn(0x7f5b340672c0 msgr2=0x7f5b340c9e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:00.032 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.030+0000 7f5b3ab63640 1 -- 172.21.15.157:0/641199612 shutdown_connections 2024-04-24T23:35:00.032 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.030+0000 7f5b3ab63640 1 -- 172.21.15.157:0/641199612 wait complete. 2024-04-24T23:35:00.042 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:35:00.139 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.139+0000 7fadcf3ee640 1 Processor -- start 2024-04-24T23:35:00.140 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.139+0000 7fadcf3ee640 1 -- start start 2024-04-24T23:35:00.141 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.140+0000 7fadcf3ee640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fadc814b950 0x7fadc80c2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:00.141 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.140+0000 7fadcf3ee640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fadc8154e60 0x7fadc80c5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:00.141 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.140+0000 7fadcf3ee640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadc814bd30 con 0x7fadc80c5640 2024-04-24T23:35:00.141 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.140+0000 7fadcf3ee640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fadc806c480 con 0x7fadc8154e60 2024-04-24T23:35:00.141 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.140+0000 7fadcf3ee640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fadc81552f0 con 0x7fadc814b950 2024-04-24T23:35:00.141 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.140+0000 7fadcc989640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fadc8154e60 0x7fadc80c5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:00.141 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.140+0000 7fadcd18a640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fadc814b950 0x7fadc80c2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:00.141 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.140+0000 7fadcd98b640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fadc80c5640 0x7fadc8151ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.157:44316/0 (socket says 172.21.15.157:44316) 2024-04-24T23:35:00.142 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.140+0000 7fadcc989640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fadc8154e60 0x7fadc80c5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.157:47724/0 (socket says 172.21.15.157:47724) 2024-04-24T23:35:00.142 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.140+0000 7fadcd98b640 1 -- 172.21.15.157:0/2446317837 learned_addr learned my addr 172.21.15.157:0/2446317837 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-24T23:35:00.142 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.140+0000 7fadcd18a640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fadc814b950 0x7fadc80c2b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.157:60738/0 (socket says 172.21.15.157:60738) 2024-04-24T23:35:00.142 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.140+0000 7fadcc989640 1 -- 172.21.15.157:0/2446317837 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fadc814b950 msgr2=0x7fadc80c2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:00.142 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.140+0000 7fadcc989640 1 --2- 172.21.15.157:0/2446317837 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fadc814b950 0x7fadc80c2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:00.142 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.140+0000 7fadcc989640 1 -- 172.21.15.157:0/2446317837 >> v1:172.21.15.169:6789/0 conn(0x7fadc80c5640 legacy=0x7fadc8151ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:00.142 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.140+0000 7fadcc989640 1 -- 172.21.15.157:0/2446317837 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadc81555d0 con 0x7fadc8154e60 2024-04-24T23:35:00.142 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.140+0000 7fadbffff640 1 -- 172.21.15.157:0/2446317837 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 955601650 0 0) 0x7fadc814bd30 con 0x7fadc80c5640 2024-04-24T23:35:00.142 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.140+0000 7fadcd18a640 1 --2- 172.21.15.157:0/2446317837 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fadc814b950 0x7fadc80c2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:35:00.144 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.141+0000 7fadcc989640 1 --2- 172.21.15.157:0/2446317837 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fadc8154e60 0x7fadc80c5100 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fadb800ba40 tx=0x7fadb802c7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=496123186b0b0073 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:00.144 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.141+0000 7fadbffff640 1 -- 172.21.15.157:0/2446317837 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb8031710 con 0x7fadc8154e60 2024-04-24T23:35:00.144 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.141+0000 7fadbffff640 1 -- 172.21.15.157:0/2446317837 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadb80318b0 con 0x7fadc8154e60 2024-04-24T23:35:00.144 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.141+0000 7fadbffff640 1 -- 172.21.15.157:0/2446317837 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb8031c80 con 0x7fadc8154e60 2024-04-24T23:35:00.144 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.142+0000 7fadcf3ee640 1 -- 172.21.15.157:0/2446317837 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fadc8154e60 msgr2=0x7fadc80c5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:00.144 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.142+0000 7fadcf3ee640 1 --2- 172.21.15.157:0/2446317837 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fadc8154e60 0x7fadc80c5100 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fadb800ba40 tx=0x7fadb802c7b0 comp rx=0 tx=0).stop 2024-04-24T23:35:00.144 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.142+0000 7fadcf3ee640 1 -- 172.21.15.157:0/2446317837 shutdown_connections 2024-04-24T23:35:00.144 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.142+0000 7fadcf3ee640 1 --2- 172.21.15.157:0/2446317837 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fadc8154e60 0x7fadc80c5100 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:00.144 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.142+0000 7fadcf3ee640 1 --2- 172.21.15.157:0/2446317837 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fadc814b950 0x7fadc80c2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:00.144 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.142+0000 7fadcf3ee640 1 -- 172.21.15.157:0/2446317837 >> 172.21.15.157:0/2446317837 conn(0x7fadc80a8720 msgr2=0x7fadc80a8b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:00.144 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.142+0000 7fadcf3ee640 1 -- 172.21.15.157:0/2446317837 shutdown_connections 2024-04-24T23:35:00.144 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.142+0000 7fadcf3ee640 1 -- 172.21.15.157:0/2446317837 wait complete. 2024-04-24T23:35:00.145 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.143+0000 7fadcf3ee640 1 Processor -- start 2024-04-24T23:35:00.145 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.143+0000 7fadcf3ee640 1 -- start start 2024-04-24T23:35:00.147 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.144+0000 7fadcf3ee640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fadc80c5640 0x7fadc814c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:00.147 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.144+0000 7fadcf3ee640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadc814b950 0x7fadc814d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:00.147 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.144+0000 7fadcf3ee640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fadc81e7090 0x7fadc80720e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:00.147 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.144+0000 7fadcf3ee640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fadc80c3070 con 0x7fadc814b950 2024-04-24T23:35:00.147 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.144+0000 7fadcf3ee640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fadc8155b90 con 0x7fadc80c5640 2024-04-24T23:35:00.147 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.144+0000 7fadcf3ee640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fadc80c31f0 con 0x7fadc81e7090 2024-04-24T23:35:00.147 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.144+0000 7fadcc989640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadc814b950 0x7fadc814d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:00.147 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.144+0000 7fadcc989640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadc814b950 0x7fadc814d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:59948/0 (socket says 172.21.15.157:59948) 2024-04-24T23:35:00.147 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.144+0000 7fadcc989640 1 -- 172.21.15.157:0/3950011135 learned_addr learned my addr 172.21.15.157:0/3950011135 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:00.148 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.144+0000 7fadcd18a640 1 --2- 172.21.15.157:0/3950011135 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fadc80c5640 0x7fadc814c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:00.148 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.144+0000 7fadcd98b640 1 --2- 172.21.15.157:0/3950011135 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fadc81e7090 0x7fadc80720e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:00.148 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.144+0000 7fadcc989640 1 -- 172.21.15.157:0/3950011135 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fadc81e7090 msgr2=0x7fadc80720e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:00.148 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.144+0000 7fadcc989640 1 --2- 172.21.15.157:0/3950011135 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fadc81e7090 0x7fadc80720e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:00.148 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.144+0000 7fadcc989640 1 -- 172.21.15.157:0/3950011135 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fadc80c5640 msgr2=0x7fadc814c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:00.148 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.144+0000 7fadcc989640 1 --2- 172.21.15.157:0/3950011135 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fadc80c5640 0x7fadc814c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:00.148 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.144+0000 7fadcc989640 1 -- 172.21.15.157:0/3950011135 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadc8071940 con 0x7fadc814b950 2024-04-24T23:35:00.148 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.144+0000 7fadcc989640 1 --2- 172.21.15.157:0/3950011135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadc814b950 0x7fadc814d130 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fadb800ba40 tx=0x7fadb80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:00.148 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.144+0000 7fadbdffb640 1 -- 172.21.15.157:0/3950011135 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb8041070 con 0x7fadc814b950 2024-04-24T23:35:00.148 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.144+0000 7fadbdffb640 1 -- 172.21.15.157:0/3950011135 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadb8033c40 con 0x7fadc814b950 2024-04-24T23:35:00.148 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.144+0000 7fadbdffb640 1 -- 172.21.15.157:0/3950011135 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb8006e10 con 0x7fadc814b950 2024-04-24T23:35:00.148 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.144+0000 7fadcf3ee640 1 -- 172.21.15.157:0/3950011135 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadc814ced0 con 0x7fadc814b950 2024-04-24T23:35:00.148 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.145+0000 7fadcf3ee640 1 -- 172.21.15.157:0/3950011135 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadc81e7bb0 con 0x7fadc814b950 2024-04-24T23:35:00.149 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.146+0000 7fadcf3ee640 1 -- 172.21.15.157:0/3950011135 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadc80681c0 con 0x7fadc814b950 2024-04-24T23:35:00.149 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.146+0000 7fadbdffb640 1 -- 172.21.15.157:0/3950011135 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fadb8004030 con 0x7fadc814b950 2024-04-24T23:35:00.149 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.146+0000 7fadbdffb640 1 --2- 172.21.15.157:0/3950011135 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fadac06be30 0x7fadac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:00.149 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.146+0000 7fadbdffb640 1 -- 172.21.15.157:0/3950011135 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fadb80ae490 con 0x7fadc814b950 2024-04-24T23:35:00.149 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.147+0000 7fadcd18a640 1 --2- 172.21.15.157:0/3950011135 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fadac06be30 0x7fadac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:00.149 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.147+0000 7fadcd18a640 1 --2- 172.21.15.157:0/3950011135 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fadac06be30 0x7fadac06e2f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fadc4009a30 tx=0x7fadc4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:00.151 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.150+0000 7fadbdffb640 1 -- 172.21.15.157:0/3950011135 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fadc8071940 con 0x7fadc814b950 2024-04-24T23:35:00.326 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.325+0000 7fadcf3ee640 1 -- 172.21.15.157:0/3950011135 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fadc80c6620 con 0x7fadc814b950 2024-04-24T23:35:00.327 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.326+0000 7fadbdffb640 1 -- 172.21.15.157:0/3950011135 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18538 (secure 0 0 0) 0x7fadb807cb90 con 0x7fadc814b950 2024-04-24T23:35:00.327 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:35:00.327 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":25,"fsid":"223685fb-5b64-46a5-9857-3791c9c07df2","created":"2024-04-24T23:34:37.504202+0000","modified":"2024-04-24T23:34:58.744500+0000","last_up_change":"2024-04-24T23:34:47.584963+0000","last_in_change":"2024-04-24T23:34:39.632954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:34:47.796655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:34:52.177208+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:34:52.667022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:34:56.395517+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"95d14ac8-fccf-4261-be8e-edafa746f629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6809","nonce":4003822160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6811","nonce":4003822160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6815","nonce":4003822160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6813","nonce":4003822160}]},"public_addr":"172.21.15.157:6809/4003822160","cluster_addr":"172.21.15.157:6811/4003822160","heartbeat_back_addr":"172.21.15.157:6815/4003822160","heartbeat_front_addr":"172.21.15.157:6813/4003822160","state":["exists","up"]},{"osd":1,"uuid":"7be79da9-8eec-4990-9178-71d602e39d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6817","nonce":1557575576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6819","nonce":1557575576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6823","nonce":1557575576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6821","nonce":1557575576}]},"public_addr":"172.21.15.157:6817/1557575576","cluster_addr":"172.21.15.157:6819/1557575576","heartbeat_back_addr":"172.21.15.157:6823/1557575576","heartbeat_front_addr":"172.21.15.157:6821/1557575576","state":["exists","up"]},{"osd":2,"uuid":"0127ed00-b7c3-466d-bfb0-01b0fd315f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6825","nonce":2793948092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6827","nonce":2793948092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6831","nonce":2793948092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6829","nonce":2793948092}]},"public_addr":"172.21.15.157:6825/2793948092","cluster_addr":"172.21.15.157:6827/2793948092","heartbeat_back_addr":"172.21.15.157:6831/2793948092","heartbeat_front_addr":"172.21.15.157:6829/2793948092","state":["exists","up"]},{"osd":3,"uuid":"f2331acc-aa37-433e-868e-3561b6039562","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6801","nonce":422758549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6803","nonce":422758549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6807","nonce":422758549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6805","nonce":422758549}]},"public_addr":"172.21.15.157:6801/422758549","cluster_addr":"172.21.15.157:6803/422758549","heartbeat_back_addr":"172.21.15.157:6807/422758549","heartbeat_front_addr":"172.21.15.157:6805/422758549","state":["exists","up"]},{"osd":4,"uuid":"1998854b-7398-43f5-ab7d-79fa0debcf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6801","nonce":321910099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6803","nonce":321910099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6807","nonce":321910099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6805","nonce":321910099}]},"public_addr":"172.21.15.169:6801/321910099","cluster_addr":"172.21.15.169:6803/321910099","heartbeat_back_addr":"172.21.15.169:6807/321910099","heartbeat_front_addr":"172.21.15.169:6805/321910099","state":["exists","up"]},{"osd":5,"uuid":"c37153ff-9b2b-48b9-a684-2c82b6f39f15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6825","nonce":243321711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6827","nonce":243321711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6831","nonce":243321711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6829","nonce":243321711}]},"public_addr":"172.21.15.169:6825/243321711","cluster_addr":"172.21.15.169:6827/243321711","heartbeat_back_addr":"172.21.15.169:6831/243321711","heartbeat_front_addr":"172.21.15.169:6829/243321711","state":["exists","up"]},{"osd":6,"uuid":"8db79e1c-f36b-49cb-8d8a-a20c2aec57c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6809","nonce":3441420785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6811","nonce":3441420785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6815","nonce":3441420785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6813","nonce":3441420785}]},"public_addr":"172.21.15.169:6809/3441420785","cluster_addr":"172.21.15.169:6811/3441420785","heartbeat_back_addr":"172.21.15.169:6815/3441420785","heartbeat_front_addr":"172.21.15.169:6813/3441420785","state":["exists","up"]},{"osd":7,"uuid":"928586e9-296c-4b8b-9b27-42f9f155dbdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6817","nonce":1663179020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6819","nonce":1663179020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6823","nonce":1663179020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6821","nonce":1663179020}]},"public_addr":"172.21.15.169:6817/1663179020","cluster_addr":"172.21.15.169:6819/1663179020","heartbeat_back_addr":"172.21.15.169:6823/1663179020","heartbeat_front_addr":"172.21.15.169:6821/1663179020","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.848039+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.826489+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.801120+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.727706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.824343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.791915+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.878073+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.810831+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:35:00.329 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.328+0000 7fadcf3ee640 1 -- 172.21.15.157:0/3950011135 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fadac06be30 msgr2=0x7fadac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:00.329 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.328+0000 7fadcf3ee640 1 --2- 172.21.15.157:0/3950011135 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fadac06be30 0x7fadac06e2f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fadc4009a30 tx=0x7fadc4008040 comp rx=0 tx=0).stop 2024-04-24T23:35:00.330 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.328+0000 7fadcf3ee640 1 -- 172.21.15.157:0/3950011135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadc814b950 msgr2=0x7fadc814d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:00.330 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.328+0000 7fadcf3ee640 1 --2- 172.21.15.157:0/3950011135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadc814b950 0x7fadc814d130 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fadb800ba40 tx=0x7fadb80046c0 comp rx=0 tx=0).stop 2024-04-24T23:35:00.330 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.328+0000 7fadcf3ee640 1 -- 172.21.15.157:0/3950011135 shutdown_connections 2024-04-24T23:35:00.330 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.328+0000 7fadcf3ee640 1 --2- 172.21.15.157:0/3950011135 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fadac06be30 0x7fadac06e2f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:00.330 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.328+0000 7fadcf3ee640 1 --2- 172.21.15.157:0/3950011135 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fadc81e7090 0x7fadc80720e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:00.330 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.328+0000 7fadcf3ee640 1 --2- 172.21.15.157:0/3950011135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadc814b950 0x7fadc814d130 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:00.330 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.328+0000 7fadcf3ee640 1 --2- 172.21.15.157:0/3950011135 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fadc80c5640 0x7fadc814c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:00.330 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.328+0000 7fadcf3ee640 1 -- 172.21.15.157:0/3950011135 >> 172.21.15.157:0/3950011135 conn(0x7fadc80a8720 msgr2=0x7fadc80c3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:00.330 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.328+0000 7fadcf3ee640 1 -- 172.21.15.157:0/3950011135 shutdown_connections 2024-04-24T23:35:00.330 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.328+0000 7fadcf3ee640 1 -- 172.21.15.157:0/3950011135 wait complete. 2024-04-24T23:35:00.355 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:35:00.454 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.453+0000 7f1633d34640 1 Processor -- start 2024-04-24T23:35:00.454 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.454+0000 7f1633d34640 1 -- start start 2024-04-24T23:35:00.458 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.454+0000 7f1633d34640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f162c152b40 0x7f162c0c5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:00.458 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.454+0000 7f1633d34640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f162c0be360 0x7f162c155350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:00.458 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.454+0000 7f1633d34640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f162c0c8bb0 con 0x7f162c0c5b70 2024-04-24T23:35:00.458 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.454+0000 7f1633d34640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f162c0c8e90 con 0x7f162c152b40 2024-04-24T23:35:00.458 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.454+0000 7f1633d34640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f162c159480 con 0x7f162c0be360 2024-04-24T23:35:00.458 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.454+0000 7f16322d1640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f162c0c5b70 0x7f162c155890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:37660/0 (socket says 172.21.15.157:37660) 2024-04-24T23:35:00.458 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.454+0000 7f16322d1640 1 -- 172.21.15.157:0/689964869 learned_addr learned my addr 172.21.15.157:0/689964869 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-24T23:35:00.458 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.455+0000 7f16312cf640 1 --2- 172.21.15.157:0/689964869 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f162c0be360 0x7f162c155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:00.458 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.455+0000 7f1631ad0640 1 --2- 172.21.15.157:0/689964869 >> v2:172.21.15.169:3300/0 conn(0x7f162c152b40 0x7f162c0c5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:00.458 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.455+0000 7f1630ace640 1 -- 172.21.15.157:0/689964869 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2343089094 0 0) 0x7f162c0c8bb0 con 0x7f162c0c5b70 2024-04-24T23:35:00.459 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.455+0000 7f1630ace640 1 -- 172.21.15.157:0/689964869 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1618003520 con 0x7f162c0c5b70 2024-04-24T23:35:00.459 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.455+0000 7f1630ace640 1 -- 172.21.15.157:0/689964869 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2474525094 0 0) 0x7f1618003520 con 0x7f162c0c5b70 2024-04-24T23:35:00.459 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.455+0000 7f1630ace640 1 -- 172.21.15.157:0/689964869 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f162c0be360 msgr2=0x7f162c155350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:00.459 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.455+0000 7f1630ace640 1 --2- 172.21.15.157:0/689964869 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f162c0be360 0x7f162c155350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:00.459 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.455+0000 7f1630ace640 1 -- 172.21.15.157:0/689964869 >> v2:172.21.15.169:3300/0 conn(0x7f162c152b40 msgr2=0x7f162c0c5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:00.459 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.455+0000 7f1630ace640 1 --2- 172.21.15.157:0/689964869 >> v2:172.21.15.169:3300/0 conn(0x7f162c152b40 0x7f162c0c5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:00.459 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.455+0000 7f1630ace640 1 -- 172.21.15.157:0/689964869 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f162c159760 con 0x7f162c0c5b70 2024-04-24T23:35:00.459 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.455+0000 7f1630ace640 1 -- 172.21.15.157:0/689964869 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1480525340 0 0) 0x7f16240019f0 con 0x7f162c0c5b70 2024-04-24T23:35:00.459 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.455+0000 7f1630ace640 1 -- 172.21.15.157:0/689964869 >> v1:172.21.15.157:6789/0 conn(0x7f162c0c5b70 legacy=0x7f162c155890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:00.459 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.455+0000 7f1630ace640 1 --2- 172.21.15.157:0/689964869 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f162c0be360 0x7f16180039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:00.459 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.455+0000 7f1630ace640 1 --2- 172.21.15.157:0/689964869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1618003f30 0x7f1618004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:00.459 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.455+0000 7f1630ace640 1 --2- 172.21.15.157:0/689964869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1618004d40 0x7f16180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:00.459 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.455+0000 7f1630ace640 1 -- 172.21.15.157:0/689964869 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f162c0c8e90 con 0x7f1618004d40 2024-04-24T23:35:00.459 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.455+0000 7f1630ace640 1 -- 172.21.15.157:0/689964869 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f162c159480 con 0x7f1618003f30 2024-04-24T23:35:00.460 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.455+0000 7f1630ace640 1 -- 172.21.15.157:0/689964869 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f161800baa0 con 0x7f162c0be360 2024-04-24T23:35:00.460 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.455+0000 7f1631ad0640 1 --2- 172.21.15.157:0/689964869 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f162c0be360 0x7f16180039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:00.460 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.455+0000 7f16322d1640 1 --2- 172.21.15.157:0/689964869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1618004d40 0x7f16180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:00.460 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.455+0000 7f16312cf640 1 --2- 172.21.15.157:0/689964869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1618003f30 0x7f1618004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:00.460 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.455+0000 7f16322d1640 1 -- 172.21.15.157:0/689964869 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f162c0be360 msgr2=0x7f16180039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:00.460 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.455+0000 7f16322d1640 1 --2- 172.21.15.157:0/689964869 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f162c0be360 0x7f16180039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:00.460 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.455+0000 7f16322d1640 1 -- 172.21.15.157:0/689964869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1618003f30 msgr2=0x7f1618004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:00.460 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.455+0000 7f16322d1640 1 --2- 172.21.15.157:0/689964869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1618003f30 0x7f1618004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:00.460 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.455+0000 7f16322d1640 1 -- 172.21.15.157:0/689964869 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1618003520 con 0x7f1618004d40 2024-04-24T23:35:00.460 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.455+0000 7f1631ad0640 1 --2- 172.21.15.157:0/689964869 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f162c0be360 0x7f16180039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:35:00.460 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.456+0000 7f16322d1640 1 --2- 172.21.15.157:0/689964869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1618004d40 0x7f16180051c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f162400a870 tx=0x7f1618005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dd2d529820847ed server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:00.460 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.456+0000 7f1630ace640 1 -- 172.21.15.157:0/689964869 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162400a4c0 con 0x7f1618004d40 2024-04-24T23:35:00.460 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.456+0000 7f1630ace640 1 -- 172.21.15.157:0/689964869 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f162402f070 con 0x7f1618004d40 2024-04-24T23:35:00.461 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.456+0000 7f1630ace640 1 -- 172.21.15.157:0/689964869 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162402bcc0 con 0x7f1618004d40 2024-04-24T23:35:00.461 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.457+0000 7f1633d34640 1 -- 172.21.15.157:0/689964869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1618004d40 msgr2=0x7f16180051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:00.461 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.457+0000 7f1633d34640 1 --2- 172.21.15.157:0/689964869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1618004d40 0x7f16180051c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f162400a870 tx=0x7f1618005830 comp rx=0 tx=0).stop 2024-04-24T23:35:00.461 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.457+0000 7f16322d1640 1 -- 172.21.15.157:0/689964869 reap_dead start 2024-04-24T23:35:00.461 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.457+0000 7f1633d34640 1 -- 172.21.15.157:0/689964869 shutdown_connections 2024-04-24T23:35:00.461 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.457+0000 7f1633d34640 1 -- 172.21.15.157:0/689964869 >> 172.21.15.157:0/689964869 conn(0x7f162c0672c0 msgr2=0x7f162c0676d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:00.461 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.457+0000 7f1633d34640 1 -- 172.21.15.157:0/689964869 shutdown_connections 2024-04-24T23:35:00.461 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.457+0000 7f1633d34640 1 -- 172.21.15.157:0/689964869 wait complete. 2024-04-24T23:35:00.464 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.458+0000 7f1633d34640 1 Processor -- start 2024-04-24T23:35:00.464 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.458+0000 7f1633d34640 1 -- start start 2024-04-24T23:35:00.464 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.459+0000 7f1633d34640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1618004d40 0x7f162c0c49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:00.464 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.459+0000 7f1633d34640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f162c14bbf0 0x7f162c155990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:00.464 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.459+0000 7f1633d34640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f162c0c1180 0x7f162c0c1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:00.464 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.459+0000 7f1633d34640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f162c0caa00 con 0x7f1618004d40 2024-04-24T23:35:00.464 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.459+0000 7f1633d34640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f162c0cb210 con 0x7f162c14bbf0 2024-04-24T23:35:00.464 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.459+0000 7f1633d34640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f162c0cba40 con 0x7f162c0c1180 2024-04-24T23:35:00.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.459+0000 7f1631ad0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1618004d40 0x7f162c0c49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:00.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.459+0000 7f1631ad0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1618004d40 0x7f162c0c49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:59968/0 (socket says 172.21.15.157:59968) 2024-04-24T23:35:00.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.459+0000 7f1631ad0640 1 -- 172.21.15.157:0/2087302162 learned_addr learned my addr 172.21.15.157:0/2087302162 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:00.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.459+0000 7f1631ad0640 1 -- 172.21.15.157:0/2087302162 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f162c0c1180 msgr2=0x7f162c0c1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:00.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.459+0000 7f1631ad0640 1 --2- 172.21.15.157:0/2087302162 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f162c0c1180 0x7f162c0c1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:00.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.459+0000 7f1631ad0640 1 -- 172.21.15.157:0/2087302162 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f162c14bbf0 msgr2=0x7f162c155990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:00.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.459+0000 7f16312cf640 1 --2- 172.21.15.157:0/2087302162 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f162c14bbf0 0x7f162c155990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:00.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.459+0000 7f1631ad0640 1 --2- 172.21.15.157:0/2087302162 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f162c14bbf0 0x7f162c155990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:00.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.459+0000 7f1631ad0640 1 -- 172.21.15.157:0/2087302162 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f162c0bde70 con 0x7f1618004d40 2024-04-24T23:35:00.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.459+0000 7f16312cf640 1 --2- 172.21.15.157:0/2087302162 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f162c14bbf0 0x7f162c155990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:35:00.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.459+0000 7f1631ad0640 1 --2- 172.21.15.157:0/2087302162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1618004d40 0x7f162c0c49e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1628011c20 tx=0x7f162800f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:00.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.459+0000 7f16227fc640 1 -- 172.21.15.157:0/2087302162 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162800fcc0 con 0x7f1618004d40 2024-04-24T23:35:00.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.459+0000 7f16227fc640 1 -- 172.21.15.157:0/2087302162 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f162800fe60 con 0x7f1618004d40 2024-04-24T23:35:00.466 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.459+0000 7f16227fc640 1 -- 172.21.15.157:0/2087302162 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1628014680 con 0x7f1618004d40 2024-04-24T23:35:00.466 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.459+0000 7f1633d34640 1 -- 172.21.15.157:0/2087302162 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f162c0be110 con 0x7f1618004d40 2024-04-24T23:35:00.466 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.459+0000 7f1633d34640 1 -- 172.21.15.157:0/2087302162 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f162c0c4f20 con 0x7f1618004d40 2024-04-24T23:35:00.466 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.460+0000 7f1633d34640 1 -- 172.21.15.157:0/2087302162 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f162c0681c0 con 0x7f1618004d40 2024-04-24T23:35:00.466 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.461+0000 7f16227fc640 1 -- 172.21.15.157:0/2087302162 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1628012040 con 0x7f1618004d40 2024-04-24T23:35:00.466 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.461+0000 7f16227fc640 1 --2- 172.21.15.157:0/2087302162 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f161006be30 0x7f161006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:00.466 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.461+0000 7f16227fc640 1 -- 172.21.15.157:0/2087302162 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f162808ba20 con 0x7f1618004d40 2024-04-24T23:35:00.466 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.461+0000 7f16312cf640 1 --2- 172.21.15.157:0/2087302162 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f161006be30 0x7f161006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:00.466 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.462+0000 7f16312cf640 1 --2- 172.21.15.157:0/2087302162 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f161006be30 0x7f161006e2f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f161c0093d0 tx=0x7f161c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:00.466 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.465+0000 7f16227fc640 1 -- 172.21.15.157:0/2087302162 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1628019030 con 0x7f1618004d40 2024-04-24T23:35:00.675 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.674+0000 7f1633d34640 1 -- 172.21.15.157:0/2087302162 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f162c0c6590 con 0x7f1618004d40 2024-04-24T23:35:00.675 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.675+0000 7f16227fc640 1 -- 172.21.15.157:0/2087302162 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5189 (secure 0 0 0) 0x7f162805a120 con 0x7f1618004d40 2024-04-24T23:35:00.676 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:35:00.676 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1276438765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1276438765},{"type":"v1","addr":"172.21.15.169:6837","nonce":1276438765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1398599086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":1398599086},{"type":"v1","addr":"172.21.15.157:6835","nonce":1398599086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4258,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3795840339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3795840339},{"type":"v1","addr":"172.21.15.157:6834","nonce":3795840339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:34:53.701085+0000","modified":"2024-04-24T23:34:58.744647+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6836/3782746080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6835","nonce":3782746080},{"type":"v1","addr":"172.21.15.169:6836","nonce":3782746080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4270,"qdb_cluster":[4270]},"id":1}]} 2024-04-24T23:35:00.676 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 7 2024-04-24T23:35:00.678 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.677+0000 7f1633d34640 1 -- 172.21.15.157:0/2087302162 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f161006be30 msgr2=0x7f161006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:00.678 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.677+0000 7f1633d34640 1 --2- 172.21.15.157:0/2087302162 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f161006be30 0x7f161006e2f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f161c0093d0 tx=0x7f161c008040 comp rx=0 tx=0).stop 2024-04-24T23:35:00.678 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.677+0000 7f1633d34640 1 -- 172.21.15.157:0/2087302162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1618004d40 msgr2=0x7f162c0c49e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:00.678 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.677+0000 7f1633d34640 1 --2- 172.21.15.157:0/2087302162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1618004d40 0x7f162c0c49e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1628011c20 tx=0x7f162800f610 comp rx=0 tx=0).stop 2024-04-24T23:35:00.678 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.677+0000 7f1633d34640 1 -- 172.21.15.157:0/2087302162 shutdown_connections 2024-04-24T23:35:00.678 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.677+0000 7f1633d34640 1 --2- 172.21.15.157:0/2087302162 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f161006be30 0x7f161006e2f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:00.678 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.677+0000 7f1633d34640 1 --2- 172.21.15.157:0/2087302162 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f162c0c1180 0x7f162c0c1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:00.678 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.677+0000 7f1633d34640 1 --2- 172.21.15.157:0/2087302162 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f162c14bbf0 0x7f162c155990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:00.679 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.677+0000 7f1633d34640 1 --2- 172.21.15.157:0/2087302162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1618004d40 0x7f162c0c49e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:00.679 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.677+0000 7f1633d34640 1 -- 172.21.15.157:0/2087302162 >> 172.21.15.157:0/2087302162 conn(0x7f162c0672c0 msgr2=0x7f162c0ca100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:00.679 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.677+0000 7f1633d34640 1 -- 172.21.15.157:0/2087302162 shutdown_connections 2024-04-24T23:35:00.679 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.677+0000 7f1633d34640 1 -- 172.21.15.157:0/2087302162 wait complete. 2024-04-24T23:35:00.688 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:35:00.786 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.785+0000 7f0d43563640 1 Processor -- start 2024-04-24T23:35:00.786 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.785+0000 7f0d43563640 1 -- start start 2024-04-24T23:35:00.787 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.786+0000 7f0d43563640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0d3c0c8b70 0x7f0d3c155d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:00.787 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.786+0000 7f0d43563640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0d3c158ec0 0x7f0d3c159330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:00.787 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.786+0000 7f0d43563640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d3c159870 con 0x7f0d3c152a70 2024-04-24T23:35:00.787 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.786+0000 7f0d43563640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f0d3c0c55f0 con 0x7f0d3c0c8b70 2024-04-24T23:35:00.788 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.786+0000 7f0d43563640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f0d3c0c5840 con 0x7f0d3c158ec0 2024-04-24T23:35:00.788 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.786+0000 7f0d40afe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0d3c0c8b70 0x7f0d3c155d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:00.788 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.786+0000 7f0d40afe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0d3c0c8b70 0x7f0d3c155d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:59974/0 (socket says 172.21.15.157:59974) 2024-04-24T23:35:00.788 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.786+0000 7f0d40afe640 1 -- 172.21.15.157:0/2765762233 learned_addr learned my addr 172.21.15.157:0/2765762233 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:00.788 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.786+0000 7f0d41b00640 1 --2- 172.21.15.157:0/2765762233 >> v2:172.21.15.169:3300/0 conn(0x7f0d3c158ec0 0x7f0d3c159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:00.788 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.786+0000 7f0d40afe640 1 -- 172.21.15.157:0/2765762233 >> v2:172.21.15.169:3300/0 conn(0x7f0d3c158ec0 msgr2=0x7f0d3c159330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:00.788 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.786+0000 7f0d40afe640 1 --2- 172.21.15.157:0/2765762233 >> v2:172.21.15.169:3300/0 conn(0x7f0d3c158ec0 0x7f0d3c159330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:00.788 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.786+0000 7f0d40afe640 1 -- 172.21.15.157:0/2765762233 >> v1:172.21.15.169:6789/0 conn(0x7f0d3c152a70 legacy=0x7f0d3c0cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:00.788 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.787+0000 7f0d40afe640 1 -- 172.21.15.157:0/2765762233 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d3c0c5b20 con 0x7f0d3c0c8b70 2024-04-24T23:35:00.789 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.787+0000 7f0d40afe640 1 --2- 172.21.15.157:0/2765762233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d3c0c8b70 0x7f0d3c155d90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c0c5770 tx=0x7f0d2c02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f4516f084b76d66 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:00.790 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.787+0000 7f0d33fff640 1 -- 172.21.15.157:0/2765762233 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c037070 con 0x7f0d3c0c8b70 2024-04-24T23:35:00.790 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.787+0000 7f0d33fff640 1 -- 172.21.15.157:0/2765762233 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d2c02ae10 con 0x7f0d3c0c8b70 2024-04-24T23:35:00.790 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.787+0000 7f0d33fff640 1 -- 172.21.15.157:0/2765762233 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c032660 con 0x7f0d3c0c8b70 2024-04-24T23:35:00.790 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.788+0000 7f0d43563640 1 -- 172.21.15.157:0/2765762233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d3c0c8b70 msgr2=0x7f0d3c155d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:00.790 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.788+0000 7f0d43563640 1 --2- 172.21.15.157:0/2765762233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d3c0c8b70 0x7f0d3c155d90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c0c5770 tx=0x7f0d2c02a100 comp rx=0 tx=0).stop 2024-04-24T23:35:00.790 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.788+0000 7f0d43563640 1 -- 172.21.15.157:0/2765762233 shutdown_connections 2024-04-24T23:35:00.790 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.788+0000 7f0d43563640 1 --2- 172.21.15.157:0/2765762233 >> v2:172.21.15.169:3300/0 conn(0x7f0d3c158ec0 0x7f0d3c159330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:00.790 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.788+0000 7f0d43563640 1 --2- 172.21.15.157:0/2765762233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d3c0c8b70 0x7f0d3c155d90 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:00.790 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.788+0000 7f0d43563640 1 -- 172.21.15.157:0/2765762233 >> 172.21.15.157:0/2765762233 conn(0x7f0d3c0ace30 msgr2=0x7f0d3c0cac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:00.790 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.788+0000 7f0d43563640 1 -- 172.21.15.157:0/2765762233 shutdown_connections 2024-04-24T23:35:00.790 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.788+0000 7f0d43563640 1 -- 172.21.15.157:0/2765762233 wait complete. 2024-04-24T23:35:00.790 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.789+0000 7f0d43563640 1 Processor -- start 2024-04-24T23:35:00.791 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.789+0000 7f0d43563640 1 -- start start 2024-04-24T23:35:00.792 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.790+0000 7f0d43563640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d3c0c8b70 0x7f0d3c153180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:00.792 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.790+0000 7f0d43563640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d3c152a70 0x7f0d3c1578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:00.792 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.790+0000 7f0d43563640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d3c158ec0 0x7f0d3c0bf500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:00.792 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.790+0000 7f0d43563640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c0cba90 con 0x7f0d3c0c8b70 2024-04-24T23:35:00.792 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.790+0000 7f0d43563640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c06dfa0 con 0x7f0d3c158ec0 2024-04-24T23:35:00.793 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.790+0000 7f0d43563640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0d3c1536c0 con 0x7f0d3c152a70 2024-04-24T23:35:00.793 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.790+0000 7f0d412ff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d3c0c8b70 0x7f0d3c153180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:00.793 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.790+0000 7f0d412ff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d3c0c8b70 0x7f0d3c153180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:59982/0 (socket says 172.21.15.157:59982) 2024-04-24T23:35:00.793 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.790+0000 7f0d412ff640 1 -- 172.21.15.157:0/4013992731 learned_addr learned my addr 172.21.15.157:0/4013992731 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:00.793 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.790+0000 7f0d412ff640 1 -- 172.21.15.157:0/4013992731 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d3c152a70 msgr2=0x7f0d3c1578b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:00.793 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.790+0000 7f0d412ff640 1 --2- 172.21.15.157:0/4013992731 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d3c152a70 0x7f0d3c1578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:00.793 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.790+0000 7f0d412ff640 1 -- 172.21.15.157:0/4013992731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d3c158ec0 msgr2=0x7f0d3c0bf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:00.793 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.790+0000 7f0d412ff640 1 --2- 172.21.15.157:0/4013992731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d3c158ec0 0x7f0d3c0bf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:00.793 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.790+0000 7f0d412ff640 1 -- 172.21.15.157:0/4013992731 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d3c14f650 con 0x7f0d3c0c8b70 2024-04-24T23:35:00.793 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.790+0000 7f0d412ff640 1 --2- 172.21.15.157:0/4013992731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d3c0c8b70 0x7f0d3c153180 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0d3800c7c0 tx=0x7f0d3800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:00.793 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.790+0000 7f0d31ffb640 1 -- 172.21.15.157:0/4013992731 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d38016020 con 0x7f0d3c0c8b70 2024-04-24T23:35:00.793 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.790+0000 7f0d31ffb640 1 -- 172.21.15.157:0/4013992731 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d3800aa50 con 0x7f0d3c0c8b70 2024-04-24T23:35:00.793 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.790+0000 7f0d43563640 1 -- 172.21.15.157:0/4013992731 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d3c0c1250 con 0x7f0d3c0c8b70 2024-04-24T23:35:00.793 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.790+0000 7f0d31ffb640 1 -- 172.21.15.157:0/4013992731 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3800ae40 con 0x7f0d3c0c8b70 2024-04-24T23:35:00.793 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.791+0000 7f0d43563640 1 -- 172.21.15.157:0/4013992731 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d3c157df0 con 0x7f0d3c0c8b70 2024-04-24T23:35:00.794 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.792+0000 7f0d43563640 1 -- 172.21.15.157:0/4013992731 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d3c06e120 con 0x7f0d3c0c8b70 2024-04-24T23:35:00.794 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.792+0000 7f0d31ffb640 1 -- 172.21.15.157:0/4013992731 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0d38019020 con 0x7f0d3c0c8b70 2024-04-24T23:35:00.794 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.792+0000 7f0d31ffb640 1 --2- 172.21.15.157:0/4013992731 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f0d2006be30 0x7f0d2006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:00.794 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.792+0000 7f0d31ffb640 1 -- 172.21.15.157:0/4013992731 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f0d3808b230 con 0x7f0d3c0c8b70 2024-04-24T23:35:00.794 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.793+0000 7f0d40afe640 1 --2- 172.21.15.157:0/4013992731 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f0d2006be30 0x7f0d2006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:00.795 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.793+0000 7f0d40afe640 1 --2- 172.21.15.157:0/4013992731 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f0d2006be30 0x7f0d2006e2f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c034000 tx=0x7f0d2c02c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:00.797 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.797+0000 7f0d31ffb640 1 -- 172.21.15.157:0/4013992731 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0d3c06e120 con 0x7f0d3c0c8b70 2024-04-24T23:35:00.974 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.973+0000 7f0d43563640 1 -- 172.21.15.157:0/4013992731 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0d3c0681c0 con 0x7f0d3c0c8b70 2024-04-24T23:35:00.975 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.974+0000 7f0d31ffb640 1 -- 172.21.15.157:0/4013992731 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18538 (secure 0 0 0) 0x7f0d38020030 con 0x7f0d3c0c8b70 2024-04-24T23:35:00.975 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:35:00.975 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":25,"fsid":"223685fb-5b64-46a5-9857-3791c9c07df2","created":"2024-04-24T23:34:37.504202+0000","modified":"2024-04-24T23:34:58.744500+0000","last_up_change":"2024-04-24T23:34:47.584963+0000","last_in_change":"2024-04-24T23:34:39.632954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:34:47.796655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:34:52.177208+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:34:52.667022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:34:56.395517+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"95d14ac8-fccf-4261-be8e-edafa746f629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6809","nonce":4003822160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6811","nonce":4003822160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6815","nonce":4003822160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6813","nonce":4003822160}]},"public_addr":"172.21.15.157:6809/4003822160","cluster_addr":"172.21.15.157:6811/4003822160","heartbeat_back_addr":"172.21.15.157:6815/4003822160","heartbeat_front_addr":"172.21.15.157:6813/4003822160","state":["exists","up"]},{"osd":1,"uuid":"7be79da9-8eec-4990-9178-71d602e39d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6817","nonce":1557575576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6819","nonce":1557575576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6823","nonce":1557575576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6821","nonce":1557575576}]},"public_addr":"172.21.15.157:6817/1557575576","cluster_addr":"172.21.15.157:6819/1557575576","heartbeat_back_addr":"172.21.15.157:6823/1557575576","heartbeat_front_addr":"172.21.15.157:6821/1557575576","state":["exists","up"]},{"osd":2,"uuid":"0127ed00-b7c3-466d-bfb0-01b0fd315f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6825","nonce":2793948092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6827","nonce":2793948092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6831","nonce":2793948092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6829","nonce":2793948092}]},"public_addr":"172.21.15.157:6825/2793948092","cluster_addr":"172.21.15.157:6827/2793948092","heartbeat_back_addr":"172.21.15.157:6831/2793948092","heartbeat_front_addr":"172.21.15.157:6829/2793948092","state":["exists","up"]},{"osd":3,"uuid":"f2331acc-aa37-433e-868e-3561b6039562","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6801","nonce":422758549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6803","nonce":422758549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6807","nonce":422758549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6805","nonce":422758549}]},"public_addr":"172.21.15.157:6801/422758549","cluster_addr":"172.21.15.157:6803/422758549","heartbeat_back_addr":"172.21.15.157:6807/422758549","heartbeat_front_addr":"172.21.15.157:6805/422758549","state":["exists","up"]},{"osd":4,"uuid":"1998854b-7398-43f5-ab7d-79fa0debcf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6801","nonce":321910099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6803","nonce":321910099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6807","nonce":321910099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6805","nonce":321910099}]},"public_addr":"172.21.15.169:6801/321910099","cluster_addr":"172.21.15.169:6803/321910099","heartbeat_back_addr":"172.21.15.169:6807/321910099","heartbeat_front_addr":"172.21.15.169:6805/321910099","state":["exists","up"]},{"osd":5,"uuid":"c37153ff-9b2b-48b9-a684-2c82b6f39f15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6825","nonce":243321711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6827","nonce":243321711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6831","nonce":243321711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6829","nonce":243321711}]},"public_addr":"172.21.15.169:6825/243321711","cluster_addr":"172.21.15.169:6827/243321711","heartbeat_back_addr":"172.21.15.169:6831/243321711","heartbeat_front_addr":"172.21.15.169:6829/243321711","state":["exists","up"]},{"osd":6,"uuid":"8db79e1c-f36b-49cb-8d8a-a20c2aec57c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6809","nonce":3441420785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6811","nonce":3441420785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6815","nonce":3441420785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6813","nonce":3441420785}]},"public_addr":"172.21.15.169:6809/3441420785","cluster_addr":"172.21.15.169:6811/3441420785","heartbeat_back_addr":"172.21.15.169:6815/3441420785","heartbeat_front_addr":"172.21.15.169:6813/3441420785","state":["exists","up"]},{"osd":7,"uuid":"928586e9-296c-4b8b-9b27-42f9f155dbdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6817","nonce":1663179020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6819","nonce":1663179020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6823","nonce":1663179020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6821","nonce":1663179020}]},"public_addr":"172.21.15.169:6817/1663179020","cluster_addr":"172.21.15.169:6819/1663179020","heartbeat_back_addr":"172.21.15.169:6823/1663179020","heartbeat_front_addr":"172.21.15.169:6821/1663179020","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.848039+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.826489+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.801120+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.727706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.824343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.791915+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.878073+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.810831+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:35:00.977 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.976+0000 7f0d43563640 1 -- 172.21.15.157:0/4013992731 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f0d2006be30 msgr2=0x7f0d2006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:00.977 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.976+0000 7f0d43563640 1 --2- 172.21.15.157:0/4013992731 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f0d2006be30 0x7f0d2006e2f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c034000 tx=0x7f0d2c02c000 comp rx=0 tx=0).stop 2024-04-24T23:35:00.977 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.976+0000 7f0d43563640 1 -- 172.21.15.157:0/4013992731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d3c0c8b70 msgr2=0x7f0d3c153180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:00.977 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.976+0000 7f0d43563640 1 --2- 172.21.15.157:0/4013992731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d3c0c8b70 0x7f0d3c153180 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0d3800c7c0 tx=0x7f0d3800cc80 comp rx=0 tx=0).stop 2024-04-24T23:35:00.977 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.976+0000 7f0d43563640 1 -- 172.21.15.157:0/4013992731 shutdown_connections 2024-04-24T23:35:00.978 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.976+0000 7f0d43563640 1 --2- 172.21.15.157:0/4013992731 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f0d2006be30 0x7f0d2006e2f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:00.978 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.976+0000 7f0d43563640 1 --2- 172.21.15.157:0/4013992731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d3c158ec0 0x7f0d3c0bf500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:00.978 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.976+0000 7f0d43563640 1 --2- 172.21.15.157:0/4013992731 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d3c152a70 0x7f0d3c1578b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:00.978 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.976+0000 7f0d43563640 1 --2- 172.21.15.157:0/4013992731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d3c0c8b70 0x7f0d3c153180 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:00.978 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.976+0000 7f0d43563640 1 -- 172.21.15.157:0/4013992731 >> 172.21.15.157:0/4013992731 conn(0x7f0d3c0ace30 msgr2=0x7f0d3c0c9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:00.978 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.976+0000 7f0d43563640 1 -- 172.21.15.157:0/4013992731 shutdown_connections 2024-04-24T23:35:00.978 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:00.977+0000 7f0d43563640 1 -- 172.21.15.157:0/4013992731 wait complete. 2024-04-24T23:35:01.390 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T23:34:53.701085+0000', 'modified': '2024-04-24T23:34:58.744647+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4270}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4270': {'gid': 4270, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.169:6836/3782746080', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6835', 'nonce': 3782746080}, {'type': 'v1', 'addr': '172.21.15.169:6836', 'nonce': 3782746080}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4270, 'qdb_cluster': [4270]} 2024-04-24T23:35:01.390 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-24T23:35:01.390 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-24T23:35:01.490 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.489+0000 7efdba82c640 1 Processor -- start 2024-04-24T23:35:01.490 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.489+0000 7efdba82c640 1 -- start start 2024-04-24T23:35:01.492 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.490+0000 7efdba82c640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7efdb4156130 0x7efdb406dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:01.492 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.490+0000 7efdba82c640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdb40a5e00 0x7efdb406e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:01.492 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.490+0000 7efdba82c640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdb4158b20 con 0x7efdb4152aa0 2024-04-24T23:35:01.492 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.490+0000 7efdba82c640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7efdb4155400 con 0x7efdb4156130 2024-04-24T23:35:01.492 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.490+0000 7efdba82c640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efdb4152e80 con 0x7efdb40a5e00 2024-04-24T23:35:01.492 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.490+0000 7efdb8dc9640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdb40a5e00 0x7efdb406e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:01.492 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.490+0000 7efdb37fe640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7efdb4156130 0x7efdb406dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:01.493 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.490+0000 7efdb8dc9640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdb40a5e00 0x7efdb406e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.157:60796/0 (socket says 172.21.15.157:60796) 2024-04-24T23:35:01.493 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.490+0000 7efdb8dc9640 1 -- 172.21.15.157:0/4041117673 learned_addr learned my addr 172.21.15.157:0/4041117673 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:01.493 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.491+0000 7efdb37fe640 1 -- 172.21.15.157:0/4041117673 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdb40a5e00 msgr2=0x7efdb406e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:01.493 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.491+0000 7efdb37fe640 1 --2- 172.21.15.157:0/4041117673 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdb40a5e00 0x7efdb406e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:01.493 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.491+0000 7efdb37fe640 1 -- 172.21.15.157:0/4041117673 >> v1:172.21.15.169:6789/0 conn(0x7efdb4152aa0 legacy=0x7efdb40bfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:01.493 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.491+0000 7efdb37fe640 1 -- 172.21.15.157:0/4041117673 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdb4155650 con 0x7efdb4156130 2024-04-24T23:35:01.493 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.491+0000 7efdb37fe640 1 --2- 172.21.15.157:0/4041117673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdb4156130 0x7efdb406dbb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efda80097a0 tx=0x7efda802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c6064a0675c8cd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:01.493 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.492+0000 7efdb2ffd640 1 -- 172.21.15.157:0/4041117673 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda8037070 con 0x7efdb4156130 2024-04-24T23:35:01.493 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.492+0000 7efdb2ffd640 1 -- 172.21.15.157:0/4041117673 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efda802ae50 con 0x7efdb4156130 2024-04-24T23:35:01.493 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.492+0000 7efdb2ffd640 1 -- 172.21.15.157:0/4041117673 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda8032660 con 0x7efdb4156130 2024-04-24T23:35:01.496 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.492+0000 7efdba82c640 1 -- 172.21.15.157:0/4041117673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdb4156130 msgr2=0x7efdb406dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:01.496 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.492+0000 7efdba82c640 1 --2- 172.21.15.157:0/4041117673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdb4156130 0x7efdb406dbb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efda80097a0 tx=0x7efda802a100 comp rx=0 tx=0).stop 2024-04-24T23:35:01.496 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.492+0000 7efdba82c640 1 -- 172.21.15.157:0/4041117673 shutdown_connections 2024-04-24T23:35:01.496 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.492+0000 7efdba82c640 1 --2- 172.21.15.157:0/4041117673 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdb40a5e00 0x7efdb406e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:01.496 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.492+0000 7efdba82c640 1 --2- 172.21.15.157:0/4041117673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdb4156130 0x7efdb406dbb0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:01.497 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.492+0000 7efdba82c640 1 -- 172.21.15.157:0/4041117673 >> 172.21.15.157:0/4041117673 conn(0x7efdb40a8690 msgr2=0x7efdb40a8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:01.497 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.492+0000 7efdba82c640 1 -- 172.21.15.157:0/4041117673 shutdown_connections 2024-04-24T23:35:01.497 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.492+0000 7efdba82c640 1 -- 172.21.15.157:0/4041117673 wait complete. 2024-04-24T23:35:01.497 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.493+0000 7efdba82c640 1 Processor -- start 2024-04-24T23:35:01.497 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.493+0000 7efdba82c640 1 -- start start 2024-04-24T23:35:01.497 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.494+0000 7efdba82c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdb40a5e00 0x7efdb41ee430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:01.497 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.494+0000 7efdba82c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdb4152aa0 0x7efdb41ee970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:01.497 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.494+0000 7efdba82c640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdb4156130 0x7efdb4144e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:01.497 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.494+0000 7efdba82c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7efdb40c0420 con 0x7efdb40a5e00 2024-04-24T23:35:01.497 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.494+0000 7efdba82c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efdb4155c10 con 0x7efdb4152aa0 2024-04-24T23:35:01.497 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.494+0000 7efdba82c640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efdb40c05a0 con 0x7efdb4156130 2024-04-24T23:35:01.497 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.494+0000 7efdb3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdb40a5e00 0x7efdb41ee430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:01.497 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.494+0000 7efdb8dc9640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdb4156130 0x7efdb4144e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:01.497 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.494+0000 7efdb3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdb40a5e00 0x7efdb41ee430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:59984/0 (socket says 172.21.15.157:59984) 2024-04-24T23:35:01.498 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.494+0000 7efdb3fff640 1 -- 172.21.15.157:0/2198055569 learned_addr learned my addr 172.21.15.157:0/2198055569 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:01.498 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.494+0000 7efdb37fe640 1 --2- 172.21.15.157:0/2198055569 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdb4152aa0 0x7efdb41ee970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:01.498 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.494+0000 7efdb3fff640 1 -- 172.21.15.157:0/2198055569 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdb4156130 msgr2=0x7efdb4144e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:01.498 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.494+0000 7efdb3fff640 1 --2- 172.21.15.157:0/2198055569 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdb4156130 0x7efdb4144e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:01.498 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.494+0000 7efdb3fff640 1 -- 172.21.15.157:0/2198055569 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdb4152aa0 msgr2=0x7efdb41ee970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:01.498 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.494+0000 7efdb3fff640 1 --2- 172.21.15.157:0/2198055569 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdb4152aa0 0x7efdb41ee970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:01.498 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.494+0000 7efdb3fff640 1 -- 172.21.15.157:0/2198055569 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdb4156920 con 0x7efdb40a5e00 2024-04-24T23:35:01.498 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.494+0000 7efdb37fe640 1 --2- 172.21.15.157:0/2198055569 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdb4152aa0 0x7efdb41ee970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:35:01.498 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.495+0000 7efdb3fff640 1 --2- 172.21.15.157:0/2198055569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdb40a5e00 0x7efdb41ee430 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efda400c400 tx=0x7efda400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:01.498 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.495+0000 7efdb0ff9640 1 -- 172.21.15.157:0/2198055569 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda40052d0 con 0x7efdb40a5e00 2024-04-24T23:35:01.498 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.495+0000 7efdb0ff9640 1 -- 172.21.15.157:0/2198055569 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efda4005490 con 0x7efdb40a5e00 2024-04-24T23:35:01.498 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.495+0000 7efdba82c640 1 -- 172.21.15.157:0/2198055569 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdb40c3090 con 0x7efdb40a5e00 2024-04-24T23:35:01.498 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.495+0000 7efdb0ff9640 1 -- 172.21.15.157:0/2198055569 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda4007260 con 0x7efdb40a5e00 2024-04-24T23:35:01.499 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.495+0000 7efdba82c640 1 -- 172.21.15.157:0/2198055569 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdb41eeeb0 con 0x7efdb40a5e00 2024-04-24T23:35:01.506 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.496+0000 7efdb0ff9640 1 -- 172.21.15.157:0/2198055569 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efda4007400 con 0x7efdb40a5e00 2024-04-24T23:35:01.506 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.497+0000 7efdb0ff9640 1 --2- 172.21.15.157:0/2198055569 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7efda006be30 0x7efda006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:01.506 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.497+0000 7efdb0ff9640 1 -- 172.21.15.157:0/2198055569 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7efda408a5f0 con 0x7efdb40a5e00 2024-04-24T23:35:01.506 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.497+0000 7efdb37fe640 1 --2- 172.21.15.157:0/2198055569 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7efda006be30 0x7efda006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:01.506 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.497+0000 7efdb37fe640 1 --2- 172.21.15.157:0/2198055569 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7efda006be30 0x7efda006e2f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7efda8034000 tx=0x7efda802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:01.506 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.498+0000 7efdbd60e740 1 Processor -- start 2024-04-24T23:35:01.506 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.498+0000 7efdbd60e740 1 -- start start 2024-04-24T23:35:01.507 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.498+0000 7efdbd60e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddecbefad0 0x55ddecbec940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:01.507 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.498+0000 7efdbd60e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ddecbb28e0 0x55ddecbece80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:01.507 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.498+0000 7efdbd60e740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ddecbd3510 0x55ddecbedbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:01.507 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.498+0000 7efdbd60e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ddecb14f00 con 0x55ddecbefad0 2024-04-24T23:35:01.507 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.498+0000 7efdbd60e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ddecabd750 con 0x55ddecbb28e0 2024-04-24T23:35:01.507 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.498+0000 7efdbd60e740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55ddecb10ec0 con 0x55ddecbd3510 2024-04-24T23:35:01.507 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.499+0000 7efdb3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddecbefad0 0x55ddecbec940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:01.507 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.499+0000 7efdb3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddecbefad0 0x55ddecbec940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:59994/0 (socket says 172.21.15.157:59994) 2024-04-24T23:35:01.507 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.499+0000 7efdb3fff640 1 -- 172.21.15.157:0/1777446410 learned_addr learned my addr 172.21.15.157:0/1777446410 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:01.507 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.499+0000 7efdb37fe640 1 --2- 172.21.15.157:0/1777446410 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ddecbb28e0 0x55ddecbece80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:01.507 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.499+0000 7efdb8dc9640 1 --2- 172.21.15.157:0/1777446410 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ddecbd3510 0x55ddecbedbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:01.507 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.499+0000 7efdb3fff640 1 -- 172.21.15.157:0/1777446410 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ddecbd3510 msgr2=0x55ddecbedbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:01.507 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.499+0000 7efdb3fff640 1 --2- 172.21.15.157:0/1777446410 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ddecbd3510 0x55ddecbedbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:01.507 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.499+0000 7efdb3fff640 1 -- 172.21.15.157:0/1777446410 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ddecbb28e0 msgr2=0x55ddecbece80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:01.508 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.499+0000 7efdb3fff640 1 --2- 172.21.15.157:0/1777446410 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ddecbb28e0 0x55ddecbece80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:01.508 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.499+0000 7efdb3fff640 1 -- 172.21.15.157:0/1777446410 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddec8292d0 con 0x55ddecbefad0 2024-04-24T23:35:01.508 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.499+0000 7efdb8dc9640 1 --2- 172.21.15.157:0/1777446410 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ddecbd3510 0x55ddecbedbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:35:01.508 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.499+0000 7efdb3fff640 1 --2- 172.21.15.157:0/1777446410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddecbefad0 0x55ddecbec940 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7efda4088870 tx=0x7efda400a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=666b19c1692527bc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:01.508 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.499+0000 7efd99ffb640 1 -- 172.21.15.157:0/1777446410 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda40569a0 con 0x55ddecbefad0 2024-04-24T23:35:01.508 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.499+0000 7efd99ffb640 1 -- 172.21.15.157:0/1777446410 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efda4054d10 con 0x55ddecbefad0 2024-04-24T23:35:01.508 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.499+0000 7efd99ffb640 1 -- 172.21.15.157:0/1777446410 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda405dd50 con 0x55ddecbefad0 2024-04-24T23:35:01.508 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.500+0000 7efdbd60e740 1 -- 172.21.15.157:0/1777446410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddecbefad0 msgr2=0x55ddecbec940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:01.508 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.500+0000 7efdbd60e740 1 --2- 172.21.15.157:0/1777446410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddecbefad0 0x55ddecbec940 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7efda4088870 tx=0x7efda400a000 comp rx=0 tx=0).stop 2024-04-24T23:35:01.508 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.500+0000 7efdbd60e740 1 -- 172.21.15.157:0/1777446410 shutdown_connections 2024-04-24T23:35:01.508 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.500+0000 7efdbd60e740 1 --2- 172.21.15.157:0/1777446410 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ddecbd3510 0x55ddecbedbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:01.508 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.500+0000 7efdbd60e740 1 --2- 172.21.15.157:0/1777446410 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ddecbb28e0 0x55ddecbece80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:01.509 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.500+0000 7efdbd60e740 1 --2- 172.21.15.157:0/1777446410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddecbefad0 0x55ddecbec940 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:01.509 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.500+0000 7efdbd60e740 1 -- 172.21.15.157:0/1777446410 >> 172.21.15.157:0/1777446410 conn(0x55ddecbae640 msgr2=0x55ddecbd2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:01.509 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.500+0000 7efdbd60e740 1 -- 172.21.15.157:0/1777446410 shutdown_connections 2024-04-24T23:35:01.509 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.500+0000 7efdbd60e740 1 -- 172.21.15.157:0/1777446410 wait complete. 2024-04-24T23:35:01.509 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.500+0000 7efdbd60e740 1 -- start start 2024-04-24T23:35:01.509 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.500+0000 7efdbd60e740 1 Processor -- start 2024-04-24T23:35:01.509 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.501+0000 7efdbd60e740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ddecbd3510 0x55ddecbe5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:01.509 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.501+0000 7efdbd60e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddecbefad0 0x55ddecbfed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:01.509 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.501+0000 7efdbd60e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ddecbe63d0 0x55ddecbe6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:01.509 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.501+0000 7efdbd60e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ddecbb2dc0 con 0x55ddecbefad0 2024-04-24T23:35:01.509 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.501+0000 7efdbd60e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ddecad0cf0 con 0x55ddecbe63d0 2024-04-24T23:35:01.509 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.501+0000 7efdbd60e740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55ddecbec4b0 con 0x55ddecbd3510 2024-04-24T23:35:01.509 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.501+0000 7efdbd60e740 10 client.0 _ll_register_callbacks cb 0x55ddecad06e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:35:01.509 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.501+0000 7efdb37fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddecbefad0 0x55ddecbfed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:01.509 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.501+0000 7efdb8dc9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ddecbe63d0 0x55ddecbe6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:01.510 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.501+0000 7efdb37fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddecbefad0 0x55ddecbfed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:59996/0 (socket says 172.21.15.157:59996) 2024-04-24T23:35:01.510 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.501+0000 7efdb37fe640 1 -- 172.21.15.157:0/2197343655 learned_addr learned my addr 172.21.15.157:0/2197343655 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:01.510 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.501+0000 7efdb3fff640 1 --2- 172.21.15.157:0/2197343655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ddecbd3510 0x55ddecbe5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:01.510 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.501+0000 7efdb37fe640 1 -- 172.21.15.157:0/2197343655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ddecbd3510 msgr2=0x55ddecbe5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:01.510 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.501+0000 7efdb37fe640 1 --2- 172.21.15.157:0/2197343655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ddecbd3510 0x55ddecbe5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:01.510 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.501+0000 7efdb37fe640 1 -- 172.21.15.157:0/2197343655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ddecbe63d0 msgr2=0x55ddecbe6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:01.510 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.501+0000 7efdb37fe640 1 --2- 172.21.15.157:0/2197343655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ddecbe63d0 0x55ddecbe6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:01.510 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.501+0000 7efdb37fe640 1 -- 172.21.15.157:0/2197343655 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddec8f36b0 con 0x55ddecbefad0 2024-04-24T23:35:01.510 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.502+0000 7efdb8dc9640 1 --2- 172.21.15.157:0/2197343655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ddecbe63d0 0x55ddecbe6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:35:01.510 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.502+0000 7efdb37fe640 1 --2- 172.21.15.157:0/2197343655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddecbefad0 0x55ddecbfed20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efda8035040 tx=0x7efda8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a461d29d93102fd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:01.510 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.502+0000 7efd92ffd640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-04-24T23:35:01.510 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.502+0000 7efd92ffd640 1 -- 172.21.15.157:0/2197343655 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda8037040 con 0x55ddecbefad0 2024-04-24T23:35:01.510 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.502+0000 7efd92ffd640 1 -- 172.21.15.157:0/2197343655 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7efda8003810 con 0x55ddecbefad0 2024-04-24T23:35:01.511 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.502+0000 7efdbd60e740 1 -- 172.21.15.157:0/2197343655 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efda4054d10 con 0x55ddecbefad0 2024-04-24T23:35:01.511 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.502+0000 7efd92ffd640 1 -- 172.21.15.157:0/2197343655 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7efda8055050 con 0x55ddecbefad0 2024-04-24T23:35:01.511 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.502+0000 7efdbd60e740 10 client.4401 fetch_fsmap learned FSMap version 7 2024-04-24T23:35:01.511 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.502+0000 7efdbd60e740 1 -- 172.21.15.157:0/2197343655 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55ddecbe58f0 con 0x55ddecbefad0 2024-04-24T23:35:01.511 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.502+0000 7efd92ffd640 1 -- 172.21.15.157:0/2197343655 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efda4054d10 con 0x55ddecbefad0 2024-04-24T23:35:01.511 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.502+0000 7efd92ffd640 1 -- 172.21.15.157:0/2197343655 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7efda8058020 con 0x55ddecbefad0 2024-04-24T23:35:01.511 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.502+0000 7efdbd60e740 10 client.4401 fetch_fsmap finished waiting for FSMap version 7 2024-04-24T23:35:01.511 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.502+0000 7efdbd60e740 10 client.4401 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T23:35:01.511 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.502+0000 7efdbd60e740 20 client.4401 populate_metadata read hostname 'smithi157' 2024-04-24T23:35:01.511 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.502+0000 7efdbd60e740 1 --2- 172.21.15.157:0/2197343655 >> [v2:172.21.15.169:6835/3782746080,v1:172.21.15.169:6836/3782746080] conn(0x55ddecbce200 0x55ddecc02230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:01.511 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.502+0000 7efdbd60e740 4 client.4401 mds_command: new command op to 4270 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:35:01.511 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.502+0000 7efdbd60e740 1 -- 172.21.15.157:0/2197343655 --> [v2:172.21.15.169:6835/3782746080,v1:172.21.15.169:6836/3782746080] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ddecb22440 con 0x55ddecbce200 2024-04-24T23:35:01.511 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.502+0000 7efdb3fff640 1 --2- 172.21.15.157:0/2197343655 >> [v2:172.21.15.169:6835/3782746080,v1:172.21.15.169:6836/3782746080] conn(0x55ddecbce200 0x55ddecc02230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:01.512 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.503+0000 7efdb3fff640 1 --2- 172.21.15.157:0/2197343655 >> [v2:172.21.15.169:6835/3782746080,v1:172.21.15.169:6836/3782746080] conn(0x55ddecbce200 0x55ddecc02230 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18a1f2d5fe140ec7 server_cookie=f4c860224b213eb2 in_seq=0 out_seq=0 2024-04-24T23:35:01.512 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.503+0000 7efd92ffd640 10 client.4401 ms_handle_connect on v2:172.21.15.169:6835/3782746080 2024-04-24T23:35:01.512 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.504+0000 7efd92ffd640 1 -- 172.21.15.157:0/2197343655 <== mds.0 v2:172.21.15.169:6835/3782746080 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ddecb22440 con 0x55ddecbce200 2024-04-24T23:35:01.512 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.504+0000 7efd92ffd640 10 client.4401 handle_command_reply: tid=0 2024-04-24T23:35:01.512 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.504+0000 7efdbd60e740 1 client.4401 shutdown 2024-04-24T23:35:01.512 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.504+0000 7efdbd60e740 1 -- 172.21.15.157:0/2197343655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddecbefad0 msgr2=0x55ddecbfed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:01.512 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.504+0000 7efdbd60e740 1 --2- 172.21.15.157:0/2197343655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddecbefad0 0x55ddecbfed20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efda8035040 tx=0x7efda8004730 comp rx=0 tx=0).stop 2024-04-24T23:35:01.512 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.504+0000 7efdbd60e740 1 -- 172.21.15.157:0/2197343655 shutdown_connections 2024-04-24T23:35:01.512 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.504+0000 7efdbd60e740 1 --2- 172.21.15.157:0/2197343655 >> [v2:172.21.15.169:6835/3782746080,v1:172.21.15.169:6836/3782746080] conn(0x55ddecbce200 0x55ddecc02230 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:01.512 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.504+0000 7efdbd60e740 1 --2- 172.21.15.157:0/2197343655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ddecbe63d0 0x55ddecbe6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:01.512 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.504+0000 7efd92ffd640 0 client.4401 ms_handle_reset on v2:172.21.15.169:6835/3782746080 2024-04-24T23:35:01.512 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.504+0000 7efdbd60e740 1 --2- 172.21.15.157:0/2197343655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddecbefad0 0x55ddecbfed20 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:01.512 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.504+0000 7efdbd60e740 1 --2- 172.21.15.157:0/2197343655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ddecbd3510 0x55ddecbe5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:01.512 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.504+0000 7efdbd60e740 1 -- 172.21.15.157:0/2197343655 >> 172.21.15.157:0/2197343655 conn(0x55ddecbb28e0 msgr2=0x55ddecbf1200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:01.513 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.504+0000 7efdbd60e740 1 -- 172.21.15.157:0/2197343655 shutdown_connections 2024-04-24T23:35:01.513 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.504+0000 7efdbd60e740 1 -- 172.21.15.157:0/2197343655 wait complete. 2024-04-24T23:35:01.513 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.504+0000 7efdbd60e740 20 client.4401 trim_cache size 0 max 16384 2024-04-24T23:35:01.523 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.522+0000 7efdbd60e740 1 Processor -- start 2024-04-24T23:35:01.523 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.522+0000 7efdbd60e740 1 -- start start 2024-04-24T23:35:01.525 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.523+0000 7efdbd60e740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ddecbce200 0x55ddecbfbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:01.525 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.523+0000 7efdbd60e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ddecbd3510 0x55ddecbd0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:01.526 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.523+0000 7efdbd60e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddecbe63d0 0x55ddecbfa2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:01.526 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.523+0000 7efdbd60e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7efdb41f8180 con 0x55ddecbe63d0 2024-04-24T23:35:01.526 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.523+0000 7efdbd60e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efdb4155010 con 0x55ddecbd3510 2024-04-24T23:35:01.526 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.523+0000 7efdbd60e740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efdb41f7b90 con 0x55ddecbce200 2024-04-24T23:35:01.526 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.523+0000 7efdb8dc9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddecbe63d0 0x55ddecbfa2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:01.526 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.523+0000 7efdb8dc9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddecbe63d0 0x55ddecbfa2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60008/0 (socket says 172.21.15.157:60008) 2024-04-24T23:35:01.526 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.523+0000 7efdb8dc9640 1 -- 172.21.15.157:0/306383580 learned_addr learned my addr 172.21.15.157:0/306383580 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:01.526 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.523+0000 7efdb37fe640 1 --2- 172.21.15.157:0/306383580 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ddecbd3510 0x55ddecbd0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:01.526 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.523+0000 7efdb8dc9640 1 -- 172.21.15.157:0/306383580 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ddecbce200 msgr2=0x55ddecbfbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:01.526 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.523+0000 7efdb8dc9640 1 --2- 172.21.15.157:0/306383580 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ddecbce200 0x55ddecbfbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:01.526 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.523+0000 7efdb8dc9640 1 -- 172.21.15.157:0/306383580 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ddecbd3510 msgr2=0x55ddecbd0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:01.526 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.523+0000 7efdb8dc9640 1 --2- 172.21.15.157:0/306383580 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ddecbd3510 0x55ddecbd0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:01.527 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.523+0000 7efdb8dc9640 1 -- 172.21.15.157:0/306383580 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddecbe8aa0 con 0x55ddecbe63d0 2024-04-24T23:35:01.527 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.523+0000 7efdb37fe640 1 --2- 172.21.15.157:0/306383580 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ddecbd3510 0x55ddecbd0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:35:01.527 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.523+0000 7efdb8dc9640 1 --2- 172.21.15.157:0/306383580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddecbe63d0 0x55ddecbfa2c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efdac007770 tx=0x7efdac009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a73467a3a6315876 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:01.527 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.523+0000 7efd997fa640 1 -- 172.21.15.157:0/306383580 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdac003b90 con 0x55ddecbe63d0 2024-04-24T23:35:01.527 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.523+0000 7efd997fa640 1 -- 172.21.15.157:0/306383580 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdac01b070 con 0x55ddecbe63d0 2024-04-24T23:35:01.527 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.523+0000 7efd997fa640 1 -- 172.21.15.157:0/306383580 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdac004020 con 0x55ddecbe63d0 2024-04-24T23:35:01.527 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.524+0000 7efdbd60e740 1 -- 172.21.15.157:0/306383580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddecbe63d0 msgr2=0x55ddecbfa2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:01.527 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.524+0000 7efdbd60e740 1 --2- 172.21.15.157:0/306383580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddecbe63d0 0x55ddecbfa2c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efdac007770 tx=0x7efdac009810 comp rx=0 tx=0).stop 2024-04-24T23:35:01.527 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.524+0000 7efdbd60e740 1 -- 172.21.15.157:0/306383580 shutdown_connections 2024-04-24T23:35:01.527 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.524+0000 7efdbd60e740 1 --2- 172.21.15.157:0/306383580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddecbe63d0 0x55ddecbfa2c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:01.527 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.524+0000 7efdbd60e740 1 --2- 172.21.15.157:0/306383580 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ddecbd3510 0x55ddecbd0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:01.527 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.524+0000 7efdbd60e740 1 --2- 172.21.15.157:0/306383580 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ddecbce200 0x55ddecbfbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:01.527 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.524+0000 7efdbd60e740 1 -- 172.21.15.157:0/306383580 >> 172.21.15.157:0/306383580 conn(0x55ddecbb28e0 msgr2=0x55ddecbf1200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:01.527 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.524+0000 7efdbd60e740 1 -- 172.21.15.157:0/306383580 shutdown_connections 2024-04-24T23:35:01.528 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.524+0000 7efdbd60e740 1 -- 172.21.15.157:0/306383580 wait complete. 2024-04-24T23:35:01.528 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.524+0000 7efdbd60e740 1 -- start start 2024-04-24T23:35:01.528 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.525+0000 7efdbd60e740 1 Processor -- start 2024-04-24T23:35:01.531 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.525+0000 7efdbd60e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddecbce200 0x55ddecc04ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:01.532 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.525+0000 7efdbd60e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ddecbd3510 0x55ddecbd0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:01.532 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.525+0000 7efdbd60e740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ddecbe63d0 0x55ddecc65390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:01.532 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.525+0000 7efdbd60e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ddecbe6f00 con 0x55ddecbce200 2024-04-24T23:35:01.532 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.525+0000 7efdbd60e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ddecbe5bd0 con 0x55ddecbd3510 2024-04-24T23:35:01.532 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.525+0000 7efdbd60e740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55ddecbce680 con 0x55ddecbe63d0 2024-04-24T23:35:01.532 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.525+0000 7efdbd60e740 10 client.0 _ll_register_callbacks cb 0x7efda8032f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:35:01.532 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.525+0000 7efdb3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddecbce200 0x55ddecc04ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:01.532 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.525+0000 7efdb3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddecbce200 0x55ddecc04ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60010/0 (socket says 172.21.15.157:60010) 2024-04-24T23:35:01.532 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.525+0000 7efdb3fff640 1 -- 172.21.15.157:0/2416165035 learned_addr learned my addr 172.21.15.157:0/2416165035 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:01.532 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.525+0000 7efdb37fe640 1 --2- 172.21.15.157:0/2416165035 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ddecbd3510 0x55ddecbd0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:01.532 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.526+0000 7efdb8dc9640 1 --2- 172.21.15.157:0/2416165035 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ddecbe63d0 0x55ddecc65390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:01.532 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.526+0000 7efdb3fff640 1 -- 172.21.15.157:0/2416165035 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ddecbe63d0 msgr2=0x55ddecc65390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:01.532 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.526+0000 7efdb3fff640 1 --2- 172.21.15.157:0/2416165035 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ddecbe63d0 0x55ddecc65390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:01.532 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.526+0000 7efdb3fff640 1 -- 172.21.15.157:0/2416165035 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ddecbd3510 msgr2=0x55ddecbd0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:01.533 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.526+0000 7efdb3fff640 1 --2- 172.21.15.157:0/2416165035 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ddecbd3510 0x55ddecbd0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:01.533 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.526+0000 7efdb3fff640 1 -- 172.21.15.157:0/2416165035 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddecbad7f0 con 0x55ddecbce200 2024-04-24T23:35:01.533 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.526+0000 7efdb3fff640 1 --2- 172.21.15.157:0/2416165035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddecbce200 0x55ddecc04ec0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efda4058630 tx=0x7efda4088740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13bc1f63f87194cd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:01.533 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.526+0000 7efd92ffd640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-04-24T23:35:01.533 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.526+0000 7efd92ffd640 1 -- 172.21.15.157:0/2416165035 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda4001120 con 0x55ddecbce200 2024-04-24T23:35:01.533 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.526+0000 7efd92ffd640 1 -- 172.21.15.157:0/2416165035 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7efda40569a0 con 0x55ddecbce200 2024-04-24T23:35:01.533 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.526+0000 7efdbd60e740 1 -- 172.21.15.157:0/2416165035 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdac01b070 con 0x55ddecbce200 2024-04-24T23:35:01.533 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.526+0000 7efd92ffd640 1 -- 172.21.15.157:0/2416165035 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7efda405db30 con 0x55ddecbce200 2024-04-24T23:35:01.533 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.526+0000 7efdbd60e740 10 client.4407 fetch_fsmap learned FSMap version 7 2024-04-24T23:35:01.533 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.526+0000 7efdbd60e740 1 -- 172.21.15.157:0/2416165035 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55ddecbb0e30 con 0x55ddecbce200 2024-04-24T23:35:01.533 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.526+0000 7efd92ffd640 1 -- 172.21.15.157:0/2416165035 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdac01b070 con 0x55ddecbce200 2024-04-24T23:35:01.533 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.526+0000 7efd92ffd640 1 -- 172.21.15.157:0/2416165035 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7efda405dcd0 con 0x55ddecbce200 2024-04-24T23:35:01.533 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.526+0000 7efdbd60e740 10 client.4407 fetch_fsmap finished waiting for FSMap version 7 2024-04-24T23:35:01.533 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.526+0000 7efdbd60e740 10 client.4407 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T23:35:01.534 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.526+0000 7efdbd60e740 20 client.4407 populate_metadata read hostname 'smithi157' 2024-04-24T23:35:01.534 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.526+0000 7efdbd60e740 1 --2- 172.21.15.157:0/2416165035 >> [v2:172.21.15.169:6835/3782746080,v1:172.21.15.169:6836/3782746080] conn(0x55ddecbefad0 0x55ddecbfa2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:01.534 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.526+0000 7efdbd60e740 4 client.4407 mds_command: new command op to 4270 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:35:01.534 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.526+0000 7efdbd60e740 1 -- 172.21.15.157:0/2416165035 --> [v2:172.21.15.169:6835/3782746080,v1:172.21.15.169:6836/3782746080] -- command(tid 0: {"prefix": "status"}) -- 0x55ddecb1c790 con 0x55ddecbefad0 2024-04-24T23:35:01.534 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.526+0000 7efdb37fe640 1 --2- 172.21.15.157:0/2416165035 >> [v2:172.21.15.169:6835/3782746080,v1:172.21.15.169:6836/3782746080] conn(0x55ddecbefad0 0x55ddecbfa2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:01.534 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.527+0000 7efd92ffd640 10 client.4407 ms_handle_connect on v2:172.21.15.169:6835/3782746080 2024-04-24T23:35:01.534 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.527+0000 7efdb37fe640 1 --2- 172.21.15.157:0/2416165035 >> [v2:172.21.15.169:6835/3782746080,v1:172.21.15.169:6836/3782746080] conn(0x55ddecbefad0 0x55ddecbfa2c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a486071524608105 server_cookie=5401eb0a43d5ad4e in_seq=0 out_seq=0 2024-04-24T23:35:01.534 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.527+0000 7efd92ffd640 1 -- 172.21.15.157:0/2416165035 <== mds.0 v2:172.21.15.169:6835/3782746080 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x55ddecb1c790 con 0x55ddecbefad0 2024-04-24T23:35:01.534 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.527+0000 7efd92ffd640 10 client.4407 handle_command_reply: tid=0 2024-04-24T23:35:01.534 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.527+0000 7efdbd60e740 1 client.4407 shutdown 2024-04-24T23:35:01.534 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.527+0000 7efdbd60e740 1 -- 172.21.15.157:0/2416165035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddecbce200 msgr2=0x55ddecc04ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:01.534 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.527+0000 7efdbd60e740 1 --2- 172.21.15.157:0/2416165035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddecbce200 0x55ddecc04ec0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efda4058630 tx=0x7efda4088740 comp rx=0 tx=0).stop 2024-04-24T23:35:01.534 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.527+0000 7efdbd60e740 1 -- 172.21.15.157:0/2416165035 shutdown_connections 2024-04-24T23:35:01.535 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.527+0000 7efdbd60e740 1 --2- 172.21.15.157:0/2416165035 >> [v2:172.21.15.169:6835/3782746080,v1:172.21.15.169:6836/3782746080] conn(0x55ddecbefad0 0x55ddecbfa2c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:01.535 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.527+0000 7efdbd60e740 1 --2- 172.21.15.157:0/2416165035 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ddecbe63d0 0x55ddecc65390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:01.535 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.527+0000 7efdbd60e740 1 --2- 172.21.15.157:0/2416165035 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ddecbd3510 0x55ddecbd0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:01.535 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.527+0000 7efd92ffd640 0 client.4407 ms_handle_reset on v2:172.21.15.169:6835/3782746080 2024-04-24T23:35:01.535 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.527+0000 7efdbd60e740 1 --2- 172.21.15.157:0/2416165035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddecbce200 0x55ddecc04ec0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:01.535 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.527+0000 7efdbd60e740 1 -- 172.21.15.157:0/2416165035 >> 172.21.15.157:0/2416165035 conn(0x55ddecbb28e0 msgr2=0x55ddecc12110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:01.535 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.527+0000 7efdbd60e740 1 -- 172.21.15.157:0/2416165035 shutdown_connections 2024-04-24T23:35:01.535 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.527+0000 7efdbd60e740 1 -- 172.21.15.157:0/2416165035 wait complete. 2024-04-24T23:35:01.535 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.527+0000 7efdbd60e740 20 client.4407 trim_cache size 0 max 16384 2024-04-24T23:35:01.535 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-04-24T23:35:01.535 INFO:teuthology.orchestra.run.smithi157.stdout: "cluster_fsid": "223685fb-5b64-46a5-9857-3791c9c07df2", 2024-04-24T23:35:01.536 INFO:teuthology.orchestra.run.smithi157.stdout: "whoami": 0, 2024-04-24T23:35:01.536 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 4270, 2024-04-24T23:35:01.536 INFO:teuthology.orchestra.run.smithi157.stdout: "want_state": "up:active", 2024-04-24T23:35:01.536 INFO:teuthology.orchestra.run.smithi157.stdout: "state": "up:active", 2024-04-24T23:35:01.536 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs", 2024-04-24T23:35:01.536 INFO:teuthology.orchestra.run.smithi157.stdout: "rank_uptime": 7.1692768290000002, 2024-04-24T23:35:01.536 INFO:teuthology.orchestra.run.smithi157.stdout: "mdsmap_epoch": 7, 2024-04-24T23:35:01.536 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_epoch": 25, 2024-04-24T23:35:01.536 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_epoch_barrier": 24, 2024-04-24T23:35:01.536 INFO:teuthology.orchestra.run.smithi157.stdout: "uptime": 13.452693526999999 2024-04-24T23:35:01.536 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-04-24T23:35:01.537 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.528+0000 7efdba82c640 1 -- 172.21.15.157:0/2198055569 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7efda006be30 msgr2=0x7efda006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:01.537 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.528+0000 7efdba82c640 1 --2- 172.21.15.157:0/2198055569 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7efda006be30 0x7efda006e2f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7efda8034000 tx=0x7efda802e000 comp rx=0 tx=0).stop 2024-04-24T23:35:01.537 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.528+0000 7efdba82c640 1 -- 172.21.15.157:0/2198055569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdb40a5e00 msgr2=0x7efdb41ee430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:01.537 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.528+0000 7efdba82c640 1 --2- 172.21.15.157:0/2198055569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdb40a5e00 0x7efdb41ee430 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efda400c400 tx=0x7efda400c8c0 comp rx=0 tx=0).stop 2024-04-24T23:35:01.538 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.528+0000 7efdba82c640 1 -- 172.21.15.157:0/2198055569 shutdown_connections 2024-04-24T23:35:01.538 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.528+0000 7efdba82c640 1 --2- 172.21.15.157:0/2198055569 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7efda006be30 0x7efda006e2f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:01.538 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.528+0000 7efdba82c640 1 --2- 172.21.15.157:0/2198055569 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdb4156130 0x7efdb4144e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:01.538 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.528+0000 7efdba82c640 1 --2- 172.21.15.157:0/2198055569 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdb4152aa0 0x7efdb41ee970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:01.538 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.528+0000 7efdba82c640 1 --2- 172.21.15.157:0/2198055569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdb40a5e00 0x7efdb41ee430 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:01.538 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.528+0000 7efdba82c640 1 -- 172.21.15.157:0/2198055569 >> 172.21.15.157:0/2198055569 conn(0x7efdb40a8690 msgr2=0x7efdb40a8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:01.538 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.528+0000 7efdba82c640 1 -- 172.21.15.157:0/2198055569 shutdown_connections 2024-04-24T23:35:01.538 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.528+0000 7efdba82c640 1 -- 172.21.15.157:0/2198055569 wait complete. 2024-04-24T23:35:01.539 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-24T23:35:01.539 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-24T23:35:01.539 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T23:35:01.637 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.637+0000 7f126aa2a640 1 Processor -- start 2024-04-24T23:35:01.637 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.637+0000 7f126aa2a640 1 -- start start 2024-04-24T23:35:01.639 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.637+0000 7f126aa2a640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12640be360 0x7f1264155350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:01.639 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.637+0000 7f126aa2a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12640c8bb0 con 0x7f1264155890 2024-04-24T23:35:01.639 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.637+0000 7f126aa2a640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1264159480 con 0x7f1264152b40 2024-04-24T23:35:01.639 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.637+0000 7f126aa2a640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f12640c8e90 con 0x7f12640be360 2024-04-24T23:35:01.639 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.638+0000 7f1268fc7640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f1264155890 0x7f126415fcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:37676/0 (socket says 172.21.15.157:37676) 2024-04-24T23:35:01.640 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.638+0000 7f1268fc7640 1 -- 172.21.15.157:0/884684114 learned_addr learned my addr 172.21.15.157:0/884684114 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-24T23:35:01.640 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.638+0000 7f12637fe640 1 --2- 172.21.15.157:0/884684114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12640be360 0x7f1264155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:01.640 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.638+0000 7f1262ffd640 1 -- 172.21.15.157:0/884684114 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 847889548 0 0) 0x7f12640c8bb0 con 0x7f1264155890 2024-04-24T23:35:01.640 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.638+0000 7f1262ffd640 1 -- 172.21.15.157:0/884684114 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f124c003520 con 0x7f1264155890 2024-04-24T23:35:01.640 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.638+0000 7f1262ffd640 1 -- 172.21.15.157:0/884684114 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1536456882 0 0) 0x7f1264159480 con 0x7f1264152b40 2024-04-24T23:35:01.640 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.638+0000 7f1262ffd640 1 -- 172.21.15.157:0/884684114 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f12640c8bb0 con 0x7f1264152b40 2024-04-24T23:35:01.640 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.638+0000 7f12637fe640 1 -- 172.21.15.157:0/884684114 >> v1:172.21.15.169:6789/0 conn(0x7f1264152b40 legacy=0x7f12640c5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:01.640 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.638+0000 7f12637fe640 1 -- 172.21.15.157:0/884684114 >> v1:172.21.15.157:6789/0 conn(0x7f1264155890 legacy=0x7f126415fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:01.640 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.638+0000 7f12637fe640 1 -- 172.21.15.157:0/884684114 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1264159760 con 0x7f12640be360 2024-04-24T23:35:01.640 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.639+0000 7f12637fe640 1 --2- 172.21.15.157:0/884684114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12640be360 0x7f1264155350 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f12540097d0 tx=0x7f125402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b31081683500a693 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:01.640 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.639+0000 7f1262ffd640 1 -- 172.21.15.157:0/884684114 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1254037070 con 0x7f12640be360 2024-04-24T23:35:01.640 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.639+0000 7f1262ffd640 1 -- 172.21.15.157:0/884684114 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f125402ae50 con 0x7f12640be360 2024-04-24T23:35:01.640 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.639+0000 7f1262ffd640 1 -- 172.21.15.157:0/884684114 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1254032640 con 0x7f12640be360 2024-04-24T23:35:01.643 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.639+0000 7f126aa2a640 1 -- 172.21.15.157:0/884684114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12640be360 msgr2=0x7f1264155350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:01.643 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.639+0000 7f126aa2a640 1 --2- 172.21.15.157:0/884684114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12640be360 0x7f1264155350 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f12540097d0 tx=0x7f125402a100 comp rx=0 tx=0).stop 2024-04-24T23:35:01.643 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.639+0000 7f126aa2a640 1 -- 172.21.15.157:0/884684114 shutdown_connections 2024-04-24T23:35:01.643 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.639+0000 7f126aa2a640 1 --2- 172.21.15.157:0/884684114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12640be360 0x7f1264155350 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:01.643 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.639+0000 7f126aa2a640 1 -- 172.21.15.157:0/884684114 >> 172.21.15.157:0/884684114 conn(0x7f12640672c0 msgr2=0x7f12640676d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:01.643 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.639+0000 7f126aa2a640 1 -- 172.21.15.157:0/884684114 shutdown_connections 2024-04-24T23:35:01.644 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.639+0000 7f126aa2a640 1 -- 172.21.15.157:0/884684114 wait complete. 2024-04-24T23:35:01.644 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.640+0000 7f126aa2a640 1 Processor -- start 2024-04-24T23:35:01.644 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.641+0000 7f126aa2a640 1 -- start start 2024-04-24T23:35:01.644 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.641+0000 7f126aa2a640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12640be360 0x7f1264154490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:01.644 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.641+0000 7f126aa2a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1264152b40 0x7f12640c0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:01.644 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.641+0000 7f126aa2a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1264155890 0x7f12640c1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:01.644 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.641+0000 7f126aa2a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1264155d40 con 0x7f1264155890 2024-04-24T23:35:01.644 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.641+0000 7f126aa2a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f12640cb210 con 0x7f1264152b40 2024-04-24T23:35:01.644 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.641+0000 7f126aa2a640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f12640caa00 con 0x7f12640be360 2024-04-24T23:35:01.644 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.641+0000 7f1268fc7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1264155890 0x7f12640c1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:01.644 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.641+0000 7f1268fc7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1264155890 0x7f12640c1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60018/0 (socket says 172.21.15.157:60018) 2024-04-24T23:35:01.644 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.641+0000 7f1268fc7640 1 -- 172.21.15.157:0/3100417248 learned_addr learned my addr 172.21.15.157:0/3100417248 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:01.644 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.642+0000 7f1263fff640 1 --2- 172.21.15.157:0/3100417248 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12640be360 0x7f1264154490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:01.644 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.642+0000 7f12637fe640 1 --2- 172.21.15.157:0/3100417248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1264152b40 0x7f12640c0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:01.645 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.642+0000 7f1268fc7640 1 -- 172.21.15.157:0/3100417248 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12640be360 msgr2=0x7f1264154490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:01.645 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.642+0000 7f1268fc7640 1 --2- 172.21.15.157:0/3100417248 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12640be360 0x7f1264154490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:01.645 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.642+0000 7f1268fc7640 1 -- 172.21.15.157:0/3100417248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1264152b40 msgr2=0x7f12640c0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:01.645 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.642+0000 7f1268fc7640 1 --2- 172.21.15.157:0/3100417248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1264152b40 0x7f12640c0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:01.645 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.642+0000 7f1268fc7640 1 -- 172.21.15.157:0/3100417248 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12641efdb0 con 0x7f1264155890 2024-04-24T23:35:01.645 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.642+0000 7f1268fc7640 1 --2- 172.21.15.157:0/3100417248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1264155890 0x7f12640c1190 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f125c00cbd0 tx=0x7f125c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:01.645 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.642+0000 7f1260ff9640 1 -- 172.21.15.157:0/3100417248 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125c017070 con 0x7f1264155890 2024-04-24T23:35:01.645 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.642+0000 7f1260ff9640 1 -- 172.21.15.157:0/3100417248 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f125c00ace0 con 0x7f1264155890 2024-04-24T23:35:01.645 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.642+0000 7f126aa2a640 1 -- 172.21.15.157:0/3100417248 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12640ac900 con 0x7f1264155890 2024-04-24T23:35:01.645 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.642+0000 7f1260ff9640 1 -- 172.21.15.157:0/3100417248 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125c012620 con 0x7f1264155890 2024-04-24T23:35:01.645 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.642+0000 7f126aa2a640 1 -- 172.21.15.157:0/3100417248 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12641549d0 con 0x7f1264155890 2024-04-24T23:35:01.646 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.643+0000 7f126aa2a640 1 -- 172.21.15.157:0/3100417248 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12640681c0 con 0x7f1264155890 2024-04-24T23:35:01.646 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.643+0000 7f1260ff9640 1 -- 172.21.15.157:0/3100417248 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f125c01a070 con 0x7f1264155890 2024-04-24T23:35:01.646 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.644+0000 7f1260ff9640 1 --2- 172.21.15.157:0/3100417248 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f125006be30 0x7f125006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:01.646 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.644+0000 7f1260ff9640 1 -- 172.21.15.157:0/3100417248 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f125c08b180 con 0x7f1264155890 2024-04-24T23:35:01.646 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.644+0000 7f1263fff640 1 --2- 172.21.15.157:0/3100417248 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f125006be30 0x7f125006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:01.646 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.644+0000 7f1263fff640 1 --2- 172.21.15.157:0/3100417248 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f125006be30 0x7f125006e2f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f12580011e0 tx=0x7f1258009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:01.648 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.648+0000 7f1260ff9640 1 -- 172.21.15.157:0/3100417248 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f125c059190 con 0x7f1264155890 2024-04-24T23:35:01.860 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.860+0000 7f126aa2a640 1 -- 172.21.15.157:0/3100417248 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f126406d3f0 con 0x7f1264155890 2024-04-24T23:35:01.861 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.860+0000 7f1260ff9640 1 -- 172.21.15.157:0/3100417248 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+88 (secure 0 0 0) 0x7f125c05d490 con 0x7f1264155890 2024-04-24T23:35:01.861 INFO:teuthology.orchestra.run.smithi157.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-24T23:35:01.863 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.862+0000 7f126aa2a640 1 -- 172.21.15.157:0/3100417248 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f125006be30 msgr2=0x7f125006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:01.863 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.862+0000 7f126aa2a640 1 --2- 172.21.15.157:0/3100417248 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f125006be30 0x7f125006e2f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f12580011e0 tx=0x7f1258009040 comp rx=0 tx=0).stop 2024-04-24T23:35:01.863 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.862+0000 7f126aa2a640 1 -- 172.21.15.157:0/3100417248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1264155890 msgr2=0x7f12640c1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:01.863 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.862+0000 7f126aa2a640 1 --2- 172.21.15.157:0/3100417248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1264155890 0x7f12640c1190 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f125c00cbd0 tx=0x7f125c00a430 comp rx=0 tx=0).stop 2024-04-24T23:35:01.863 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.862+0000 7f126aa2a640 1 -- 172.21.15.157:0/3100417248 shutdown_connections 2024-04-24T23:35:01.863 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.862+0000 7f126aa2a640 1 --2- 172.21.15.157:0/3100417248 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f125006be30 0x7f125006e2f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:01.864 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.862+0000 7f126aa2a640 1 --2- 172.21.15.157:0/3100417248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1264155890 0x7f12640c1190 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:01.864 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.862+0000 7f126aa2a640 1 --2- 172.21.15.157:0/3100417248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1264152b40 0x7f12640c0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:01.864 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.862+0000 7f126aa2a640 1 --2- 172.21.15.157:0/3100417248 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12640be360 0x7f1264154490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:01.864 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.862+0000 7f126aa2a640 1 -- 172.21.15.157:0/3100417248 >> 172.21.15.157:0/3100417248 conn(0x7f12640672c0 msgr2=0x7f1264151b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:01.864 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.862+0000 7f126aa2a640 1 -- 172.21.15.157:0/3100417248 shutdown_connections 2024-04-24T23:35:01.864 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01.862+0000 7f126aa2a640 1 -- 172.21.15.157:0/3100417248 wait complete. 2024-04-24T23:35:01.874 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-24T23:35:01.874 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T23:35:01.874 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2024-04-24T23:35:01.874 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-24T23:35:01.874 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-24T23:35:01.874 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-24T23:35:01.874 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T23:35:01.874 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T23:35:01.874 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-24T23:35:01.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:35:01.875 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2024-04-24T23:35:01.890 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T23:35:01.890 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T23:35:01.890 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T23:35:01.890 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:35:01.890 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-04-24T23:35:01.890 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:35:01.890 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:35:01.890 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:35:01.890 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2024-04-24T23:35:01.891 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:35:01.891 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:35:01.891 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2024-04-24T23:35:01.891 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:35:01.891 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:35:01.891 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T23:35:01.891 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:35:01.891 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-24T23:35:01.891 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 42743sec preferred_lft 42743sec 2024-04-24T23:35:01.891 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::3a09:adda:a6cf:8530/64 scope link noprefixroute 2024-04-24T23:35:01.891 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:35:01.892 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-24T23:35:01.892 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:35:01.892 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-24T23:35:01.892 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link add name ceph-brx type bridge 2024-04-24T23:35:01.892 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip addr flush dev ceph-brx 2024-04-24T23:35:01.892 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set ceph-brx up 2024-04-24T23:35:01.892 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-24T23:35:01.892 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-24T23:35:01.977 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:35:02.064 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:35:02.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:35:02.071 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T23:35:02.094 INFO:teuthology.orchestra.run.smithi157.stdout:1 2024-04-24T23:35:02.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:35:02.096 DEBUG:teuthology.orchestra.run.smithi157:> route 2024-04-24T23:35:02.162 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2024-04-24T23:35:02.162 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T23:35:02.162 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-24T23:35:02.163 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-24T23:35:02.163 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T23:35:02.163 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:35:02.163 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-24T23:35:02.163 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-24T23:35:02.163 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-24T23:35:02.164 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-24T23:35:02.164 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-24T23:35:02.250 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:35:02.416 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:35:02.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:35:02.422 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-04-24T23:35:02.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:35:02.438 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list-id 2024-04-24T23:35:02.492 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:35:02.492 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-24T23:35:02.492 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-24T23:35:02.492 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-24T23:35:02.492 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-24T23:35:02.580 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:35:02.602 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:35:02.608 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-24T23:35:02.608 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:35:02.608 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-24T23:35:02.609 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-24T23:35:02.609 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-24T23:35:02.609 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-24T23:35:02.609 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-24T23:35:02.609 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-24T23:35:02.609 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-24T23:35:02.657 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:35:02.726 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:35:02.733 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:35:02.733 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-24T23:35:02.733 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2024-04-24T23:35:02.733 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set dev brx.0 master ceph-brx 2024-04-24T23:35:02.733 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-24T23:35:02.780 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:35:02.801 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:35:02.807 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-24T23:35:02.807 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:35:02.807 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-24T23:35:02.821 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-24T23:35:02.821 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:35:02.822 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-24T23:35:02.874 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2024-04-24T23:35:02.938 DEBUG:teuthology.orchestra.run.smithi157:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T23:35:02.993 INFO:teuthology.orchestra.run.smithi157.stdout:/proc 2024-04-24T23:35:02.993 INFO:teuthology.orchestra.run.smithi157.stdout:/sys 2024-04-24T23:35:02.993 INFO:teuthology.orchestra.run.smithi157.stdout:/dev 2024-04-24T23:35:02.993 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/security 2024-04-24T23:35:02.993 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/shm 2024-04-24T23:35:02.993 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/pts 2024-04-24T23:35:02.993 INFO:teuthology.orchestra.run.smithi157.stdout:/run 2024-04-24T23:35:02.993 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/cgroup 2024-04-24T23:35:02.993 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/pstore 2024-04-24T23:35:02.993 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/bpf 2024-04-24T23:35:02.993 INFO:teuthology.orchestra.run.smithi157.stdout:/ 2024-04-24T23:35:02.993 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/selinux 2024-04-24T23:35:02.993 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T23:35:02.994 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/hugepages 2024-04-24T23:35:02.994 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/mqueue 2024-04-24T23:35:02.994 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/debug 2024-04-24T23:35:02.994 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/tracing 2024-04-24T23:35:02.994 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/config 2024-04-24T23:35:02.994 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-sysusers.service 2024-04-24T23:35:02.994 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/fuse/connections 2024-04-24T23:35:02.994 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T23:35:02.994 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T23:35:02.994 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T23:35:02.994 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T23:35:02.994 INFO:teuthology.orchestra.run.smithi157.stdout:/run/user/1000 2024-04-24T23:35:02.994 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph 2024-04-24T23:35:02.994 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-0 2024-04-24T23:35:02.995 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-1 2024-04-24T23:35:02.995 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-2 2024-04-24T23:35:02.995 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-3 2024-04-24T23:35:02.995 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns 2024-04-24T23:35:02.995 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-24T23:35:02.995 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-24T23:35:02.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:35:02.995 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2024-04-24T23:35:03.048 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-24T23:35:03.048 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-24T23:35:03.090 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2024-04-24T23:35:03.114 DEBUG:teuthology.orchestra.run.smithi157:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T23:35:03.169 INFO:teuthology.orchestra.run.smithi157.stdout:/proc 2024-04-24T23:35:03.169 INFO:teuthology.orchestra.run.smithi157.stdout:/sys 2024-04-24T23:35:03.170 INFO:teuthology.orchestra.run.smithi157.stdout:/dev 2024-04-24T23:35:03.170 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/security 2024-04-24T23:35:03.170 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/shm 2024-04-24T23:35:03.170 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/pts 2024-04-24T23:35:03.170 INFO:teuthology.orchestra.run.smithi157.stdout:/run 2024-04-24T23:35:03.170 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/cgroup 2024-04-24T23:35:03.170 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/pstore 2024-04-24T23:35:03.170 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/bpf 2024-04-24T23:35:03.170 INFO:teuthology.orchestra.run.smithi157.stdout:/ 2024-04-24T23:35:03.170 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/selinux 2024-04-24T23:35:03.170 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T23:35:03.170 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/hugepages 2024-04-24T23:35:03.170 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/mqueue 2024-04-24T23:35:03.171 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/debug 2024-04-24T23:35:03.171 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/tracing 2024-04-24T23:35:03.171 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/config 2024-04-24T23:35:03.171 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-sysusers.service 2024-04-24T23:35:03.171 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/fuse/connections 2024-04-24T23:35:03.171 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T23:35:03.171 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T23:35:03.171 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T23:35:03.171 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T23:35:03.171 INFO:teuthology.orchestra.run.smithi157.stdout:/run/user/1000 2024-04-24T23:35:03.171 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph 2024-04-24T23:35:03.171 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-0 2024-04-24T23:35:03.171 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-1 2024-04-24T23:35:03.172 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-2 2024-04-24T23:35:03.172 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-3 2024-04-24T23:35:03.172 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns 2024-04-24T23:35:03.172 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-24T23:35:03.172 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-24T23:35:03.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:35:03.172 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2024-04-24T23:35:03.203 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi157.stderr:2024-04-24T23:35:03.203+0000 7f1336761480 -1 init, newargv = 0x55e2e357c5f0 newargc=15 2024-04-24T23:35:03.203 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi157.stderr:ceph-fuse[39497]: starting ceph client 2024-04-24T23:35:03.209 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi157.stderr:ceph-fuse[39497]: starting fuse 2024-04-24T23:35:03.217 INFO:teuthology.orchestra.run.smithi157.stdout:43 2024-04-24T23:35:03.218 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-04-24T23:35:03.218 DEBUG:teuthology.orchestra.run.smithi157:> sudo stdin-killer -- python3 -c ' 2024-04-24T23:35:03.218 DEBUG:teuthology.orchestra.run.smithi157:> import glob 2024-04-24T23:35:03.218 DEBUG:teuthology.orchestra.run.smithi157:> import re 2024-04-24T23:35:03.218 DEBUG:teuthology.orchestra.run.smithi157:> import os 2024-04-24T23:35:03.218 DEBUG:teuthology.orchestra.run.smithi157:> import subprocess 2024-04-24T23:35:03.218 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-24T23:35:03.218 DEBUG:teuthology.orchestra.run.smithi157:> def _find_admin_socket(client_name): 2024-04-24T23:35:03.218 DEBUG:teuthology.orchestra.run.smithi157:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-24T23:35:03.219 DEBUG:teuthology.orchestra.run.smithi157:> files = glob.glob(asok_path) 2024-04-24T23:35:03.219 DEBUG:teuthology.orchestra.run.smithi157:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-24T23:35:03.219 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-24T23:35:03.219 DEBUG:teuthology.orchestra.run.smithi157:> # Given a non-glob path, it better be there 2024-04-24T23:35:03.219 DEBUG:teuthology.orchestra.run.smithi157:> if "*" not in asok_path: 2024-04-24T23:35:03.219 DEBUG:teuthology.orchestra.run.smithi157:> assert(len(files) == 1) 2024-04-24T23:35:03.219 DEBUG:teuthology.orchestra.run.smithi157:> return files[0] 2024-04-24T23:35:03.219 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-24T23:35:03.219 DEBUG:teuthology.orchestra.run.smithi157:> for f in files: 2024-04-24T23:35:03.219 DEBUG:teuthology.orchestra.run.smithi157:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-24T23:35:03.219 DEBUG:teuthology.orchestra.run.smithi157:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-24T23:35:03.219 DEBUG:teuthology.orchestra.run.smithi157:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-24T23:35:03.219 DEBUG:teuthology.orchestra.run.smithi157:> contents = proc_f.read() 2024-04-24T23:35:03.220 DEBUG:teuthology.orchestra.run.smithi157:> if mountpoint in contents: 2024-04-24T23:35:03.220 DEBUG:teuthology.orchestra.run.smithi157:> return f 2024-04-24T23:35:03.220 DEBUG:teuthology.orchestra.run.smithi157:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-24T23:35:03.220 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-24T23:35:03.220 DEBUG:teuthology.orchestra.run.smithi157:> print(_find_admin_socket("client.admin")) 2024-04-24T23:35:03.220 DEBUG:teuthology.orchestra.run.smithi157:> ' 2024-04-24T23:35:03.334 INFO:teuthology.orchestra.run.smithi157.stdout:/var/run/ceph/ceph-client.admin.39497.asok 2024-04-24T23:35:03.337 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:35:03.345 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.39497.asok 2024-04-24T23:35:03.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:35:03.345 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.39497.asok status 2024-04-24T23:35:03.478 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-04-24T23:35:03.478 INFO:teuthology.orchestra.run.smithi157.stdout: "metadata": { 2024-04-24T23:35:03.478 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", 2024-04-24T23:35:03.478 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-24T23:35:03.478 INFO:teuthology.orchestra.run.smithi157.stdout: "entity_id": "admin", 2024-04-24T23:35:03.478 INFO:teuthology.orchestra.run.smithi157.stdout: "hostname": "smithi157", 2024-04-24T23:35:03.478 INFO:teuthology.orchestra.run.smithi157.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-24T23:35:03.478 INFO:teuthology.orchestra.run.smithi157.stdout: "pid": "39497", 2024-04-24T23:35:03.478 INFO:teuthology.orchestra.run.smithi157.stdout: "root": "/" 2024-04-24T23:35:03.478 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-24T23:35:03.479 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_count": 0, 2024-04-24T23:35:03.479 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_pinned_count": 0, 2024-04-24T23:35:03.479 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 4422, 2024-04-24T23:35:03.479 INFO:teuthology.orchestra.run.smithi157.stdout: "inst": { 2024-04-24T23:35:03.479 INFO:teuthology.orchestra.run.smithi157.stdout: "name": { 2024-04-24T23:35:03.479 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "client", 2024-04-24T23:35:03.479 INFO:teuthology.orchestra.run.smithi157.stdout: "num": 4422 2024-04-24T23:35:03.479 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-24T23:35:03.479 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2024-04-24T23:35:03.479 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2024-04-24T23:35:03.479 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2024-04-24T23:35:03.479 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 1964036766 2024-04-24T23:35:03.479 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-24T23:35:03.479 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-24T23:35:03.480 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2024-04-24T23:35:03.480 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2024-04-24T23:35:03.480 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2024-04-24T23:35:03.480 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 1964036766 2024-04-24T23:35:03.480 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-24T23:35:03.480 INFO:teuthology.orchestra.run.smithi157.stdout: "inst_str": "client.4422 192.168.0.1:0/1964036766", 2024-04-24T23:35:03.480 INFO:teuthology.orchestra.run.smithi157.stdout: "addr_str": "192.168.0.1:0/1964036766", 2024-04-24T23:35:03.480 INFO:teuthology.orchestra.run.smithi157.stdout: "inode_count": 1, 2024-04-24T23:35:03.480 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_epoch": 7, 2024-04-24T23:35:03.480 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch": 25, 2024-04-24T23:35:03.480 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch_barrier": 0, 2024-04-24T23:35:03.480 INFO:teuthology.orchestra.run.smithi157.stdout: "blocklisted": false, 2024-04-24T23:35:03.480 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs" 2024-04-24T23:35:03.480 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-04-24T23:35:03.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:35:03.486 DEBUG:teuthology.orchestra.run.smithi157:> stat --file-system '--printf=%T 2024-04-24T23:35:03.486 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-24T23:35:03.501 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2024-04-24T23:35:03.501 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-24T23:35:03.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:35:03.501 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-24T23:35:03.565 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-24T23:35:03.656 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:35:03.676 INFO:teuthology.orchestra.run.smithi157.stdout:# file: . 2024-04-24T23:35:03.676 INFO:teuthology.orchestra.run.smithi157.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-24T23:35:03.676 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:35:03.676 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:35:03.682 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-24T23:35:03.682 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:35:03.697 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:35:03.834 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.833+0000 7f8ce3d42640 1 Processor -- start 2024-04-24T23:35:03.834 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.833+0000 7f8ce3d42640 1 -- start start 2024-04-24T23:35:03.835 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.834+0000 7f8ce3d42640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8cdc070860 0x7f8cdc070c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:03.835 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.834+0000 7f8ce3d42640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8cdc0a5c50 0x7f8cdc0c2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:03.835 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.834+0000 7f8ce3d42640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8cdc153110 con 0x7f8cdc0c1590 2024-04-24T23:35:03.835 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.834+0000 7f8ce3d42640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f8cdc153d00 con 0x7f8cdc070860 2024-04-24T23:35:03.835 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.834+0000 7f8ce3d42640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8cdc1533d0 con 0x7f8cdc0a5c50 2024-04-24T23:35:03.836 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.834+0000 7f8ce1ade640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8cdc070860 0x7f8cdc070c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:03.836 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.834+0000 7f8ce1ade640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8cdc070860 0x7f8cdc070c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39594/0 (socket says 172.21.15.157:39594) 2024-04-24T23:35:03.836 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.834+0000 7f8ce1ade640 1 -- 172.21.15.157:0/1054130781 learned_addr learned my addr 172.21.15.157:0/1054130781 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:03.836 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.834+0000 7f8ce12dd640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f8cdc0c1590 0x7f8cdc0c1e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:41120/0 (socket says 172.21.15.157:41120) 2024-04-24T23:35:03.836 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.835+0000 7f8ce22df640 1 --2- 172.21.15.157:0/1054130781 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8cdc0a5c50 0x7f8cdc0c2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:03.836 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.835+0000 7f8ce1ade640 1 -- 172.21.15.157:0/1054130781 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8cdc0a5c50 msgr2=0x7f8cdc0c2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:03.836 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.835+0000 7f8ce1ade640 1 --2- 172.21.15.157:0/1054130781 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8cdc0a5c50 0x7f8cdc0c2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:03.836 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.835+0000 7f8ce1ade640 1 -- 172.21.15.157:0/1054130781 >> v1:172.21.15.157:6789/0 conn(0x7f8cdc0c1590 legacy=0x7f8cdc0c1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:03.836 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.835+0000 7f8ce0adc640 1 -- 172.21.15.157:0/1054130781 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 966310643 0 0) 0x7f8cdc153110 con 0x7f8cdc0c1590 2024-04-24T23:35:03.836 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.835+0000 7f8ce1ade640 1 -- 172.21.15.157:0/1054130781 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cdc1536b0 con 0x7f8cdc070860 2024-04-24T23:35:03.838 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.835+0000 7f8ce1ade640 1 --2- 172.21.15.157:0/1054130781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cdc070860 0x7f8cdc070c40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4009870 tx=0x7f8cd402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=680c922c82713cb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:03.838 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.835+0000 7f8ce0adc640 1 -- 172.21.15.157:0/1054130781 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd4037070 con 0x7f8cdc070860 2024-04-24T23:35:03.838 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.835+0000 7f8ce0adc640 1 -- 172.21.15.157:0/1054130781 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cd4032440 con 0x7f8cdc070860 2024-04-24T23:35:03.838 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.835+0000 7f8ce0adc640 1 -- 172.21.15.157:0/1054130781 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd4032810 con 0x7f8cdc070860 2024-04-24T23:35:03.838 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.836+0000 7f8ce3d42640 1 -- 172.21.15.157:0/1054130781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cdc070860 msgr2=0x7f8cdc070c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:03.838 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.836+0000 7f8ce3d42640 1 --2- 172.21.15.157:0/1054130781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cdc070860 0x7f8cdc070c40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4009870 tx=0x7f8cd402a1c0 comp rx=0 tx=0).stop 2024-04-24T23:35:03.838 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.836+0000 7f8ce3d42640 1 -- 172.21.15.157:0/1054130781 shutdown_connections 2024-04-24T23:35:03.838 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.836+0000 7f8ce3d42640 1 --2- 172.21.15.157:0/1054130781 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8cdc0a5c50 0x7f8cdc0c2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:03.838 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.836+0000 7f8ce3d42640 1 --2- 172.21.15.157:0/1054130781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cdc070860 0x7f8cdc070c40 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:03.838 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.836+0000 7f8ce3d42640 1 -- 172.21.15.157:0/1054130781 >> 172.21.15.157:0/1054130781 conn(0x7f8cdc0a8720 msgr2=0x7f8cdc0a8b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:03.838 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.836+0000 7f8ce3d42640 1 -- 172.21.15.157:0/1054130781 shutdown_connections 2024-04-24T23:35:03.838 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.836+0000 7f8ce3d42640 1 -- 172.21.15.157:0/1054130781 wait complete. 2024-04-24T23:35:03.838 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.837+0000 7f8ce3d42640 1 Processor -- start 2024-04-24T23:35:03.839 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.837+0000 7f8ce3d42640 1 -- start start 2024-04-24T23:35:03.841 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.838+0000 7f8ce3d42640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8cdc070860 0x7f8cdc06dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:03.841 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.838+0000 7f8ce3d42640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8cdc0a5c50 0x7f8cdc1515e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:03.841 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.838+0000 7f8ce3d42640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cdc148110 0x7f8cdc1ee350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:03.841 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.838+0000 7f8ce3d42640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8cdc150b40 con 0x7f8cdc148110 2024-04-24T23:35:03.841 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.838+0000 7f8ce3d42640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8cdc1509c0 con 0x7f8cdc0a5c50 2024-04-24T23:35:03.841 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.838+0000 7f8ce3d42640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8cdc0a62c0 con 0x7f8cdc070860 2024-04-24T23:35:03.841 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.838+0000 7f8ce22df640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cdc148110 0x7f8cdc1ee350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:03.841 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.838+0000 7f8ce22df640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cdc148110 0x7f8cdc1ee350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39598/0 (socket says 172.21.15.157:39598) 2024-04-24T23:35:03.841 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.838+0000 7f8ce22df640 1 -- 172.21.15.157:0/1840723750 learned_addr learned my addr 172.21.15.157:0/1840723750 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:03.842 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.838+0000 7f8ce12dd640 1 --2- 172.21.15.157:0/1840723750 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8cdc0a5c50 0x7f8cdc1515e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:03.842 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.838+0000 7f8ce1ade640 1 --2- 172.21.15.157:0/1840723750 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8cdc070860 0x7f8cdc06dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:03.842 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.838+0000 7f8ce22df640 1 -- 172.21.15.157:0/1840723750 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8cdc070860 msgr2=0x7f8cdc06dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:03.842 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.838+0000 7f8ce22df640 1 --2- 172.21.15.157:0/1840723750 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8cdc070860 0x7f8cdc06dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:03.842 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.838+0000 7f8ce22df640 1 -- 172.21.15.157:0/1840723750 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8cdc0a5c50 msgr2=0x7f8cdc1515e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:03.842 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.838+0000 7f8ce22df640 1 --2- 172.21.15.157:0/1840723750 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8cdc0a5c50 0x7f8cdc1515e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:03.842 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.838+0000 7f8ce22df640 1 -- 172.21.15.157:0/1840723750 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cdc06db10 con 0x7f8cdc148110 2024-04-24T23:35:03.842 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.838+0000 7f8ce1ade640 1 --2- 172.21.15.157:0/1840723750 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8cdc070860 0x7f8cdc06dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:35:03.842 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.838+0000 7f8ce22df640 1 --2- 172.21.15.157:0/1840723750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cdc148110 0x7f8cdc1ee350 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc00cda0 tx=0x7f8ccc00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:03.842 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.839+0000 7f8cda7fc640 1 -- 172.21.15.157:0/1840723750 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc003b90 con 0x7f8cdc148110 2024-04-24T23:35:03.842 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.839+0000 7f8cda7fc640 1 -- 172.21.15.157:0/1840723750 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ccc003d30 con 0x7f8cdc148110 2024-04-24T23:35:03.842 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.839+0000 7f8ce3d42640 1 -- 172.21.15.157:0/1840723750 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cdc151020 con 0x7f8cdc148110 2024-04-24T23:35:03.842 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.839+0000 7f8cda7fc640 1 -- 172.21.15.157:0/1840723750 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc004240 con 0x7f8cdc148110 2024-04-24T23:35:03.843 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.839+0000 7f8ce3d42640 1 -- 172.21.15.157:0/1840723750 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cdc06d230 con 0x7f8cdc148110 2024-04-24T23:35:03.843 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.839+0000 7f8ce3d42640 1 -- 172.21.15.157:0/1840723750 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cdc0681c0 con 0x7f8cdc148110 2024-04-24T23:35:03.843 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.840+0000 7f8cda7fc640 1 -- 172.21.15.157:0/1840723750 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8ccc0043e0 con 0x7f8cdc148110 2024-04-24T23:35:03.843 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.840+0000 7f8cda7fc640 1 --2- 172.21.15.157:0/1840723750 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f8cc806be30 0x7f8cc806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:03.843 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.841+0000 7f8cda7fc640 1 -- 172.21.15.157:0/1840723750 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f8ccc08bac0 con 0x7f8cdc148110 2024-04-24T23:35:03.843 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.841+0000 7f8ce1ade640 1 --2- 172.21.15.157:0/1840723750 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f8cc806be30 0x7f8cc806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:03.843 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.841+0000 7f8ce1ade640 1 --2- 172.21.15.157:0/1840723750 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f8cc806be30 0x7f8cc806e2f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4034000 tx=0x7f8cd402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:03.845 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:03.845+0000 7f8cda7fc640 1 -- 172.21.15.157:0/1840723750 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8ccc05a1c0 con 0x7f8cdc148110 2024-04-24T23:35:04.018 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.018+0000 7f8ce3d42640 1 -- 172.21.15.157:0/1840723750 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8cdc0c3cc0 con 0x7f8cdc148110 2024-04-24T23:35:04.019 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.019+0000 7f8cda7fc640 1 -- 172.21.15.157:0/1840723750 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18538 (secure 0 0 0) 0x7f8ccc059ac0 con 0x7f8cdc148110 2024-04-24T23:35:04.020 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:35:04.020 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":25,"fsid":"223685fb-5b64-46a5-9857-3791c9c07df2","created":"2024-04-24T23:34:37.504202+0000","modified":"2024-04-24T23:34:58.744500+0000","last_up_change":"2024-04-24T23:34:47.584963+0000","last_in_change":"2024-04-24T23:34:39.632954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:34:47.796655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:34:52.177208+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:34:52.667022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:34:56.395517+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"95d14ac8-fccf-4261-be8e-edafa746f629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6809","nonce":4003822160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6811","nonce":4003822160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6815","nonce":4003822160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6813","nonce":4003822160}]},"public_addr":"172.21.15.157:6809/4003822160","cluster_addr":"172.21.15.157:6811/4003822160","heartbeat_back_addr":"172.21.15.157:6815/4003822160","heartbeat_front_addr":"172.21.15.157:6813/4003822160","state":["exists","up"]},{"osd":1,"uuid":"7be79da9-8eec-4990-9178-71d602e39d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6817","nonce":1557575576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6819","nonce":1557575576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6823","nonce":1557575576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6821","nonce":1557575576}]},"public_addr":"172.21.15.157:6817/1557575576","cluster_addr":"172.21.15.157:6819/1557575576","heartbeat_back_addr":"172.21.15.157:6823/1557575576","heartbeat_front_addr":"172.21.15.157:6821/1557575576","state":["exists","up"]},{"osd":2,"uuid":"0127ed00-b7c3-466d-bfb0-01b0fd315f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6825","nonce":2793948092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6827","nonce":2793948092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6831","nonce":2793948092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6829","nonce":2793948092}]},"public_addr":"172.21.15.157:6825/2793948092","cluster_addr":"172.21.15.157:6827/2793948092","heartbeat_back_addr":"172.21.15.157:6831/2793948092","heartbeat_front_addr":"172.21.15.157:6829/2793948092","state":["exists","up"]},{"osd":3,"uuid":"f2331acc-aa37-433e-868e-3561b6039562","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6801","nonce":422758549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6803","nonce":422758549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6807","nonce":422758549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6805","nonce":422758549}]},"public_addr":"172.21.15.157:6801/422758549","cluster_addr":"172.21.15.157:6803/422758549","heartbeat_back_addr":"172.21.15.157:6807/422758549","heartbeat_front_addr":"172.21.15.157:6805/422758549","state":["exists","up"]},{"osd":4,"uuid":"1998854b-7398-43f5-ab7d-79fa0debcf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6801","nonce":321910099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6803","nonce":321910099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6807","nonce":321910099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6805","nonce":321910099}]},"public_addr":"172.21.15.169:6801/321910099","cluster_addr":"172.21.15.169:6803/321910099","heartbeat_back_addr":"172.21.15.169:6807/321910099","heartbeat_front_addr":"172.21.15.169:6805/321910099","state":["exists","up"]},{"osd":5,"uuid":"c37153ff-9b2b-48b9-a684-2c82b6f39f15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6825","nonce":243321711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6827","nonce":243321711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6831","nonce":243321711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6829","nonce":243321711}]},"public_addr":"172.21.15.169:6825/243321711","cluster_addr":"172.21.15.169:6827/243321711","heartbeat_back_addr":"172.21.15.169:6831/243321711","heartbeat_front_addr":"172.21.15.169:6829/243321711","state":["exists","up"]},{"osd":6,"uuid":"8db79e1c-f36b-49cb-8d8a-a20c2aec57c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6809","nonce":3441420785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6811","nonce":3441420785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6815","nonce":3441420785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6813","nonce":3441420785}]},"public_addr":"172.21.15.169:6809/3441420785","cluster_addr":"172.21.15.169:6811/3441420785","heartbeat_back_addr":"172.21.15.169:6815/3441420785","heartbeat_front_addr":"172.21.15.169:6813/3441420785","state":["exists","up"]},{"osd":7,"uuid":"928586e9-296c-4b8b-9b27-42f9f155dbdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6817","nonce":1663179020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6819","nonce":1663179020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6823","nonce":1663179020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6821","nonce":1663179020}]},"public_addr":"172.21.15.169:6817/1663179020","cluster_addr":"172.21.15.169:6819/1663179020","heartbeat_back_addr":"172.21.15.169:6823/1663179020","heartbeat_front_addr":"172.21.15.169:6821/1663179020","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.848039+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.826489+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.801120+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.727706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.824343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.791915+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.878073+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.810831+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:35:04.022 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.021+0000 7f8ce3d42640 1 -- 172.21.15.157:0/1840723750 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f8cc806be30 msgr2=0x7f8cc806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:04.022 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.021+0000 7f8ce3d42640 1 --2- 172.21.15.157:0/1840723750 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f8cc806be30 0x7f8cc806e2f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4034000 tx=0x7f8cd402e000 comp rx=0 tx=0).stop 2024-04-24T23:35:04.022 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.021+0000 7f8ce3d42640 1 -- 172.21.15.157:0/1840723750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cdc148110 msgr2=0x7f8cdc1ee350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:04.023 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.021+0000 7f8ce3d42640 1 --2- 172.21.15.157:0/1840723750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cdc148110 0x7f8cdc1ee350 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc00cda0 tx=0x7f8ccc00a660 comp rx=0 tx=0).stop 2024-04-24T23:35:04.023 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.021+0000 7f8ce3d42640 1 -- 172.21.15.157:0/1840723750 shutdown_connections 2024-04-24T23:35:04.023 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.021+0000 7f8ce3d42640 1 --2- 172.21.15.157:0/1840723750 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f8cc806be30 0x7f8cc806e2f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:04.023 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.021+0000 7f8ce3d42640 1 --2- 172.21.15.157:0/1840723750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cdc148110 0x7f8cdc1ee350 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:04.023 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.021+0000 7f8ce3d42640 1 --2- 172.21.15.157:0/1840723750 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8cdc0a5c50 0x7f8cdc1515e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:04.023 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.021+0000 7f8ce3d42640 1 --2- 172.21.15.157:0/1840723750 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8cdc070860 0x7f8cdc06dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:04.023 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.021+0000 7f8ce3d42640 1 -- 172.21.15.157:0/1840723750 >> 172.21.15.157:0/1840723750 conn(0x7f8cdc0a8720 msgr2=0x7f8cdc0a8b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:04.024 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.021+0000 7f8ce3d42640 1 -- 172.21.15.157:0/1840723750 shutdown_connections 2024-04-24T23:35:04.024 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.021+0000 7f8ce3d42640 1 -- 172.21.15.157:0/1840723750 wait complete. 2024-04-24T23:35:04.033 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:34:47.796655+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:34:52.177208+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:34:52.667022+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T23:34:56.395517+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T23:35:04.034 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:35:04.135 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.134+0000 7fbf515d5640 1 Processor -- start 2024-04-24T23:35:04.135 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.134+0000 7fbf515d5640 1 -- start start 2024-04-24T23:35:04.143 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.135+0000 7fbf515d5640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbf4c0b03e0 0x7fbf4c1567e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:04.143 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.135+0000 7fbf515d5640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fbf4c1592f0 0x7fbf4c155c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:04.143 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.135+0000 7fbf515d5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbf4c0c55d0 0x7fbf4c0c5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:04.143 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.135+0000 7fbf515d5640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fbf4c0b07c0 con 0x7fbf4c0c55d0 2024-04-24T23:35:04.143 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.135+0000 7fbf515d5640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fbf4c0cb330 con 0x7fbf4c1592f0 2024-04-24T23:35:04.143 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.135+0000 7fbf515d5640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbf4c0cb610 con 0x7fbf4c0b03e0 2024-04-24T23:35:04.144 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.135+0000 7fbf4b7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbf4c0c55d0 0x7fbf4c0c5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:04.144 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.135+0000 7fbf4b7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbf4c0c55d0 0x7fbf4c0c5a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39614/0 (socket says 172.21.15.157:39614) 2024-04-24T23:35:04.144 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.135+0000 7fbf4b7fe640 1 -- 172.21.15.157:0/3433737458 learned_addr learned my addr 172.21.15.157:0/3433737458 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:04.144 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.135+0000 7fbf4affd640 1 --2- 172.21.15.157:0/3433737458 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbf4c0b03e0 0x7fbf4c1567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:04.144 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.135+0000 7fbf4a7fc640 1 --2- 172.21.15.157:0/3433737458 >> v2:172.21.15.169:3300/0 conn(0x7fbf4c1592f0 0x7fbf4c155c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:04.144 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.135+0000 7fbf4b7fe640 1 -- 172.21.15.157:0/3433737458 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbf4c0b03e0 msgr2=0x7fbf4c1567e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:04.144 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.135+0000 7fbf4b7fe640 1 --2- 172.21.15.157:0/3433737458 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbf4c0b03e0 0x7fbf4c1567e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:04.144 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.135+0000 7fbf4b7fe640 1 -- 172.21.15.157:0/3433737458 >> v2:172.21.15.169:3300/0 conn(0x7fbf4c1592f0 msgr2=0x7fbf4c155c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:04.145 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.135+0000 7fbf4b7fe640 1 --2- 172.21.15.157:0/3433737458 >> v2:172.21.15.169:3300/0 conn(0x7fbf4c1592f0 0x7fbf4c155c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:04.145 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.135+0000 7fbf4b7fe640 1 -- 172.21.15.157:0/3433737458 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf4c0cb8f0 con 0x7fbf4c0c55d0 2024-04-24T23:35:04.145 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.135+0000 7fbf4affd640 1 --2- 172.21.15.157:0/3433737458 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbf4c0b03e0 0x7fbf4c1567e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:35:04.145 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.135+0000 7fbf4a7fc640 1 --2- 172.21.15.157:0/3433737458 >> v2:172.21.15.169:3300/0 conn(0x7fbf4c1592f0 0x7fbf4c155c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:35:04.145 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.136+0000 7fbf4b7fe640 1 --2- 172.21.15.157:0/3433737458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf4c0c55d0 0x7fbf4c0c5a40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c067e80 tx=0x7fbf4402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47725c8d2c6c37a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:04.145 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.136+0000 7fbf49ffb640 1 -- 172.21.15.157:0/3433737458 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf44036070 con 0x7fbf4c0c55d0 2024-04-24T23:35:04.145 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.136+0000 7fbf49ffb640 1 -- 172.21.15.157:0/3433737458 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf44031600 con 0x7fbf4c0c55d0 2024-04-24T23:35:04.145 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.136+0000 7fbf49ffb640 1 -- 172.21.15.157:0/3433737458 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf440319d0 con 0x7fbf4c0c55d0 2024-04-24T23:35:04.145 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.137+0000 7fbf515d5640 1 -- 172.21.15.157:0/3433737458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf4c0c55d0 msgr2=0x7fbf4c0c5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:04.146 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.137+0000 7fbf515d5640 1 --2- 172.21.15.157:0/3433737458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf4c0c55d0 0x7fbf4c0c5a40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c067e80 tx=0x7fbf4402a610 comp rx=0 tx=0).stop 2024-04-24T23:35:04.146 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.137+0000 7fbf515d5640 1 -- 172.21.15.157:0/3433737458 shutdown_connections 2024-04-24T23:35:04.146 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.137+0000 7fbf515d5640 1 --2- 172.21.15.157:0/3433737458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf4c0c55d0 0x7fbf4c0c5a40 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:04.146 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.137+0000 7fbf515d5640 1 --2- 172.21.15.157:0/3433737458 >> v2:172.21.15.169:3300/0 conn(0x7fbf4c1592f0 0x7fbf4c155c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:04.146 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.137+0000 7fbf515d5640 1 --2- 172.21.15.157:0/3433737458 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbf4c0b03e0 0x7fbf4c1567e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:04.146 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.137+0000 7fbf515d5640 1 -- 172.21.15.157:0/3433737458 >> 172.21.15.157:0/3433737458 conn(0x7fbf4c067080 msgr2=0x7fbf4c067490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:04.146 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.137+0000 7fbf515d5640 1 -- 172.21.15.157:0/3433737458 shutdown_connections 2024-04-24T23:35:04.146 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.137+0000 7fbf515d5640 1 -- 172.21.15.157:0/3433737458 wait complete. 2024-04-24T23:35:04.147 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.138+0000 7fbf515d5640 1 Processor -- start 2024-04-24T23:35:04.147 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.138+0000 7fbf515d5640 1 -- start start 2024-04-24T23:35:04.147 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.139+0000 7fbf515d5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf4c0b03e0 0x7fbf4c14d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:04.147 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.139+0000 7fbf515d5640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbf4c0c55d0 0x7fbf4c0c9d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:04.147 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.139+0000 7fbf515d5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf4c1592f0 0x7fbf4c1e7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:04.147 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.139+0000 7fbf515d5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbf4c0cafc0 con 0x7fbf4c0b03e0 2024-04-24T23:35:04.147 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.139+0000 7fbf515d5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbf4c1556e0 con 0x7fbf4c1592f0 2024-04-24T23:35:04.148 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.139+0000 7fbf515d5640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbf4c159750 con 0x7fbf4c0c55d0 2024-04-24T23:35:04.148 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.139+0000 7fbf4affd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf4c0b03e0 0x7fbf4c14d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:04.148 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.139+0000 7fbf4affd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf4c0b03e0 0x7fbf4c14d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39616/0 (socket says 172.21.15.157:39616) 2024-04-24T23:35:04.148 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.139+0000 7fbf4affd640 1 -- 172.21.15.157:0/3809708438 learned_addr learned my addr 172.21.15.157:0/3809708438 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:04.148 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.139+0000 7fbf4b7fe640 1 --2- 172.21.15.157:0/3809708438 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf4c1592f0 0x7fbf4c1e7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:04.148 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.139+0000 7fbf4affd640 1 -- 172.21.15.157:0/3809708438 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbf4c0c55d0 msgr2=0x7fbf4c0c9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:04.148 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.139+0000 7fbf4affd640 1 --2- 172.21.15.157:0/3809708438 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbf4c0c55d0 0x7fbf4c0c9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:04.148 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.139+0000 7fbf4affd640 1 -- 172.21.15.157:0/3809708438 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf4c1592f0 msgr2=0x7fbf4c1e7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:04.149 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.139+0000 7fbf4affd640 1 --2- 172.21.15.157:0/3809708438 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf4c1592f0 0x7fbf4c1e7e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:04.149 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.139+0000 7fbf4affd640 1 -- 172.21.15.157:0/3809708438 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf4c0c17d0 con 0x7fbf4c0b03e0 2024-04-24T23:35:04.149 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.139+0000 7fbf4affd640 1 --2- 172.21.15.157:0/3809708438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf4c0b03e0 0x7fbf4c14d8a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c00e730 tx=0x7fbf3c00ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:04.149 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.139+0000 7fbf337fe640 1 -- 172.21.15.157:0/3809708438 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c00f040 con 0x7fbf4c0b03e0 2024-04-24T23:35:04.149 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.139+0000 7fbf337fe640 1 -- 172.21.15.157:0/3809708438 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf3c004230 con 0x7fbf4c0b03e0 2024-04-24T23:35:04.149 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.139+0000 7fbf515d5640 1 -- 172.21.15.157:0/3809708438 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf4c1eb8b0 con 0x7fbf4c0b03e0 2024-04-24T23:35:04.149 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.139+0000 7fbf337fe640 1 -- 172.21.15.157:0/3809708438 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c00ee40 con 0x7fbf4c0b03e0 2024-04-24T23:35:04.149 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.139+0000 7fbf515d5640 1 -- 172.21.15.157:0/3809708438 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf4c1eadd0 con 0x7fbf4c0b03e0 2024-04-24T23:35:04.149 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.140+0000 7fbf515d5640 1 -- 172.21.15.157:0/3809708438 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf4c0681c0 con 0x7fbf4c0b03e0 2024-04-24T23:35:04.149 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.141+0000 7fbf337fe640 1 -- 172.21.15.157:0/3809708438 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbf3c00c6d0 con 0x7fbf4c0b03e0 2024-04-24T23:35:04.150 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.141+0000 7fbf337fe640 1 --2- 172.21.15.157:0/3809708438 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fbf3806be60 0x7fbf3806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:04.150 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.141+0000 7fbf337fe640 1 -- 172.21.15.157:0/3809708438 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fbf3c08aef0 con 0x7fbf4c0b03e0 2024-04-24T23:35:04.150 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.141+0000 7fbf4a7fc640 1 --2- 172.21.15.157:0/3809708438 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fbf3806be60 0x7fbf3806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:04.150 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.142+0000 7fbf4a7fc640 1 --2- 172.21.15.157:0/3809708438 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fbf3806be60 0x7fbf3806e320 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbf40006f20 tx=0x7fbf40008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:04.150 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.146+0000 7fbf337fe640 1 -- 172.21.15.157:0/3809708438 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbf3c017020 con 0x7fbf4c0b03e0 2024-04-24T23:35:04.320 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.319+0000 7fbf515d5640 1 -- 172.21.15.157:0/3809708438 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbf4c0c4ff0 con 0x7fbf4c0b03e0 2024-04-24T23:35:04.320 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.320+0000 7fbf337fe640 1 -- 172.21.15.157:0/3809708438 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7fbf3c0594e0 con 0x7fbf4c0b03e0 2024-04-24T23:35:04.321 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2024-04-24T23:35:04.323 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.322+0000 7fbf515d5640 1 -- 172.21.15.157:0/3809708438 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fbf3806be60 msgr2=0x7fbf3806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:04.323 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.322+0000 7fbf515d5640 1 --2- 172.21.15.157:0/3809708438 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fbf3806be60 0x7fbf3806e320 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbf40006f20 tx=0x7fbf40008040 comp rx=0 tx=0).stop 2024-04-24T23:35:04.323 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.322+0000 7fbf515d5640 1 -- 172.21.15.157:0/3809708438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf4c0b03e0 msgr2=0x7fbf4c14d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:04.323 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.322+0000 7fbf515d5640 1 --2- 172.21.15.157:0/3809708438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf4c0b03e0 0x7fbf4c14d8a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c00e730 tx=0x7fbf3c00ebf0 comp rx=0 tx=0).stop 2024-04-24T23:35:04.323 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.322+0000 7fbf515d5640 1 -- 172.21.15.157:0/3809708438 shutdown_connections 2024-04-24T23:35:04.323 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.322+0000 7fbf515d5640 1 --2- 172.21.15.157:0/3809708438 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fbf3806be60 0x7fbf3806e320 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:04.323 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.322+0000 7fbf515d5640 1 --2- 172.21.15.157:0/3809708438 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf4c1592f0 0x7fbf4c1e7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:04.323 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.322+0000 7fbf515d5640 1 --2- 172.21.15.157:0/3809708438 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbf4c0c55d0 0x7fbf4c0c9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:04.323 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.322+0000 7fbf515d5640 1 --2- 172.21.15.157:0/3809708438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf4c0b03e0 0x7fbf4c14d8a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:04.323 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.322+0000 7fbf515d5640 1 -- 172.21.15.157:0/3809708438 >> 172.21.15.157:0/3809708438 conn(0x7fbf4c067080 msgr2=0x7fbf4c0c63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:04.323 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.322+0000 7fbf515d5640 1 -- 172.21.15.157:0/3809708438 shutdown_connections 2024-04-24T23:35:04.323 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.322+0000 7fbf515d5640 1 -- 172.21.15.157:0/3809708438 wait complete. 2024-04-24T23:35:04.333 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:35:04.435 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.434+0000 7f5539cb3640 1 Processor -- start 2024-04-24T23:35:04.435 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.434+0000 7f5539cb3640 1 -- start start 2024-04-24T23:35:04.436 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.435+0000 7f5539cb3640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5534150720 0x7f5534150b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:04.436 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.435+0000 7f5539cb3640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5534153330 0x7f55341537a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:04.436 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.435+0000 7f5539cb3640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f553406e240 con 0x7f5534072c50 2024-04-24T23:35:04.436 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.435+0000 7f5539cb3640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f5534156a50 con 0x7f5534150720 2024-04-24T23:35:04.437 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.435+0000 7f5539cb3640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5534156cf0 con 0x7f5534153330 2024-04-24T23:35:04.437 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.435+0000 7f5532ffd640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5534150720 0x7f5534150b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:04.437 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.435+0000 7f5533fff640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5534153330 0x7f55341537a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:04.437 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.435+0000 7f55337fe640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f5534072c50 0x7f553406d830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.157:56576/0 (socket says 172.21.15.157:56576) 2024-04-24T23:35:04.437 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.435+0000 7f55337fe640 1 -- 172.21.15.157:0/3423506488 learned_addr learned my addr 172.21.15.157:0/3423506488 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-24T23:35:04.437 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.436+0000 7f5532ffd640 1 -- 172.21.15.157:0/3423506488 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5534153330 msgr2=0x7f55341537a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:04.437 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.436+0000 7f5532ffd640 1 --2- 172.21.15.157:0/3423506488 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5534153330 0x7f55341537a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:04.437 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.436+0000 7f5532ffd640 1 -- 172.21.15.157:0/3423506488 >> v1:172.21.15.169:6789/0 conn(0x7f5534072c50 legacy=0x7f553406d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:04.437 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.436+0000 7f5532ffd640 1 -- 172.21.15.157:0/3423506488 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5534156fd0 con 0x7f5534150720 2024-04-24T23:35:04.439 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.436+0000 7f5532ffd640 1 --2- 172.21.15.157:0/3423506488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5534150720 0x7f5534150b80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f55280097a0 tx=0x7f552802a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc3d853164f3a316 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:04.439 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.437+0000 7f55327fc640 1 -- 172.21.15.157:0/3423506488 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5528037070 con 0x7f5534150720 2024-04-24T23:35:04.439 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.437+0000 7f55327fc640 1 -- 172.21.15.157:0/3423506488 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f552802adb0 con 0x7f5534150720 2024-04-24T23:35:04.439 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.437+0000 7f55327fc640 1 -- 172.21.15.157:0/3423506488 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5528032660 con 0x7f5534150720 2024-04-24T23:35:04.439 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.437+0000 7f5539cb3640 1 -- 172.21.15.157:0/3423506488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5534150720 msgr2=0x7f5534150b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:04.439 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.437+0000 7f5539cb3640 1 --2- 172.21.15.157:0/3423506488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5534150720 0x7f5534150b80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f55280097a0 tx=0x7f552802a060 comp rx=0 tx=0).stop 2024-04-24T23:35:04.439 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.437+0000 7f5539cb3640 1 -- 172.21.15.157:0/3423506488 shutdown_connections 2024-04-24T23:35:04.439 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.437+0000 7f5539cb3640 1 --2- 172.21.15.157:0/3423506488 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5534153330 0x7f55341537a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:04.439 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.437+0000 7f5539cb3640 1 --2- 172.21.15.157:0/3423506488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5534150720 0x7f5534150b80 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:04.440 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.437+0000 7f5539cb3640 1 -- 172.21.15.157:0/3423506488 >> 172.21.15.157:0/3423506488 conn(0x7f55340c68e0 msgr2=0x7f55340c6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:04.440 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.437+0000 7f5539cb3640 1 -- 172.21.15.157:0/3423506488 shutdown_connections 2024-04-24T23:35:04.440 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.437+0000 7f5539cb3640 1 -- 172.21.15.157:0/3423506488 wait complete. 2024-04-24T23:35:04.440 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.438+0000 7f5539cb3640 1 Processor -- start 2024-04-24T23:35:04.440 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.438+0000 7f5539cb3640 1 -- start start 2024-04-24T23:35:04.444 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.439+0000 7f5539cb3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5534072c50 0x7f55340ac680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:04.444 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.439+0000 7f5539cb3640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5534150720 0x7f553414ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:04.444 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.439+0000 7f5539cb3640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5534153330 0x7f553414b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:04.444 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.439+0000 7f5539cb3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f55341577d0 con 0x7f5534072c50 2024-04-24T23:35:04.444 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.439+0000 7f5539cb3640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f553406df40 con 0x7f5534153330 2024-04-24T23:35:04.444 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.439+0000 7f5539cb3640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5534153ce0 con 0x7f5534150720 2024-04-24T23:35:04.445 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.439+0000 7f55337fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5534072c50 0x7f55340ac680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:04.445 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.439+0000 7f55337fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5534072c50 0x7f55340ac680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39628/0 (socket says 172.21.15.157:39628) 2024-04-24T23:35:04.445 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.439+0000 7f55337fe640 1 -- 172.21.15.157:0/791121642 learned_addr learned my addr 172.21.15.157:0/791121642 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:04.445 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.439+0000 7f5533fff640 1 --2- 172.21.15.157:0/791121642 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5534153330 0x7f553414b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:04.445 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.439+0000 7f5532ffd640 1 --2- 172.21.15.157:0/791121642 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5534150720 0x7f553414ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:04.445 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.440+0000 7f55337fe640 1 -- 172.21.15.157:0/791121642 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5534150720 msgr2=0x7f553414ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:04.445 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.440+0000 7f55337fe640 1 --2- 172.21.15.157:0/791121642 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5534150720 0x7f553414ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:04.445 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.440+0000 7f55337fe640 1 -- 172.21.15.157:0/791121642 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5534153330 msgr2=0x7f553414b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:04.445 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.440+0000 7f55337fe640 1 --2- 172.21.15.157:0/791121642 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5534153330 0x7f553414b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:04.445 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.440+0000 7f55337fe640 1 -- 172.21.15.157:0/791121642 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55340c10f0 con 0x7f5534072c50 2024-04-24T23:35:04.445 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.440+0000 7f55337fe640 1 --2- 172.21.15.157:0/791121642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5534072c50 0x7f55340ac680 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f552400c3f0 tx=0x7f552400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:04.446 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.440+0000 7f551bfff640 1 -- 172.21.15.157:0/791121642 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5524017070 con 0x7f5534072c50 2024-04-24T23:35:04.446 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.440+0000 7f551bfff640 1 -- 172.21.15.157:0/791121642 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5524005ce0 con 0x7f5534072c50 2024-04-24T23:35:04.446 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.440+0000 7f5539cb3640 1 -- 172.21.15.157:0/791121642 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5534155bc0 con 0x7f5534072c50 2024-04-24T23:35:04.446 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.440+0000 7f551bfff640 1 -- 172.21.15.157:0/791121642 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55240072f0 con 0x7f5534072c50 2024-04-24T23:35:04.446 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.440+0000 7f5539cb3640 1 -- 172.21.15.157:0/791121642 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55341551d0 con 0x7f5534072c50 2024-04-24T23:35:04.446 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.441+0000 7f5539cb3640 1 -- 172.21.15.157:0/791121642 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55340681c0 con 0x7f5534072c50 2024-04-24T23:35:04.446 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.442+0000 7f551bfff640 1 -- 172.21.15.157:0/791121642 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f552401a070 con 0x7f5534072c50 2024-04-24T23:35:04.446 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.442+0000 7f551bfff640 1 --2- 172.21.15.157:0/791121642 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f552006be30 0x7f552006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:04.446 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.442+0000 7f551bfff640 1 -- 172.21.15.157:0/791121642 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f552408a490 con 0x7f5534072c50 2024-04-24T23:35:04.446 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.442+0000 7f5532ffd640 1 --2- 172.21.15.157:0/791121642 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f552006be30 0x7f552006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:04.446 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.443+0000 7f5532ffd640 1 --2- 172.21.15.157:0/791121642 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f552006be30 0x7f552006e2f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5528034000 tx=0x7f552802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:04.447 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.446+0000 7f551bfff640 1 -- 172.21.15.157:0/791121642 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f55240584b0 con 0x7f5534072c50 2024-04-24T23:35:04.620 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.619+0000 7f5539cb3640 1 -- 172.21.15.157:0/791121642 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f55340c50f0 con 0x7f5534072c50 2024-04-24T23:35:04.620 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.619+0000 7f551bfff640 1 -- 172.21.15.157:0/791121642 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f552405c7b0 con 0x7f5534072c50 2024-04-24T23:35:04.620 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2024-04-24T23:35:04.622 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.621+0000 7f5539cb3640 1 -- 172.21.15.157:0/791121642 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f552006be30 msgr2=0x7f552006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:04.622 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.621+0000 7f5539cb3640 1 --2- 172.21.15.157:0/791121642 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f552006be30 0x7f552006e2f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5528034000 tx=0x7f552802e000 comp rx=0 tx=0).stop 2024-04-24T23:35:04.622 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.621+0000 7f5539cb3640 1 -- 172.21.15.157:0/791121642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5534072c50 msgr2=0x7f55340ac680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:04.623 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.621+0000 7f5539cb3640 1 --2- 172.21.15.157:0/791121642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5534072c50 0x7f55340ac680 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f552400c3f0 tx=0x7f552400c8b0 comp rx=0 tx=0).stop 2024-04-24T23:35:04.623 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.621+0000 7f5539cb3640 1 -- 172.21.15.157:0/791121642 shutdown_connections 2024-04-24T23:35:04.623 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.621+0000 7f5539cb3640 1 --2- 172.21.15.157:0/791121642 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f552006be30 0x7f552006e2f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:04.623 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.621+0000 7f5539cb3640 1 --2- 172.21.15.157:0/791121642 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5534153330 0x7f553414b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:04.623 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.621+0000 7f5539cb3640 1 --2- 172.21.15.157:0/791121642 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5534150720 0x7f553414ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:04.623 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.621+0000 7f5539cb3640 1 --2- 172.21.15.157:0/791121642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5534072c50 0x7f55340ac680 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:04.623 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.621+0000 7f5539cb3640 1 -- 172.21.15.157:0/791121642 >> 172.21.15.157:0/791121642 conn(0x7f55340c68e0 msgr2=0x7f55340c46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:04.623 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.622+0000 7f5539cb3640 1 -- 172.21.15.157:0/791121642 shutdown_connections 2024-04-24T23:35:04.623 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.622+0000 7f5539cb3640 1 -- 172.21.15.157:0/791121642 wait complete. 2024-04-24T23:35:04.634 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:35:04.735 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.734+0000 7fdc86293640 1 Processor -- start 2024-04-24T23:35:04.735 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.734+0000 7fdc86293640 1 -- start start 2024-04-24T23:35:04.744 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.735+0000 7fdc86293640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc800b03e0 0x7fdc801567e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:04.745 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.735+0000 7fdc86293640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fdc801592f0 0x7fdc80155c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:04.745 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.735+0000 7fdc86293640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc800cb330 con 0x7fdc800c55d0 2024-04-24T23:35:04.745 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.735+0000 7fdc86293640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fdc800b07c0 con 0x7fdc801592f0 2024-04-24T23:35:04.745 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.735+0000 7fdc86293640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fdc800cb610 con 0x7fdc800b03e0 2024-04-24T23:35:04.745 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.735+0000 7fdc84830640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fdc800c55d0 0x7fdc800c75f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:41126/0 (socket says 172.21.15.157:41126) 2024-04-24T23:35:04.745 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.735+0000 7fdc84830640 1 -- 172.21.15.157:0/929345148 learned_addr learned my addr 172.21.15.157:0/929345148 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-24T23:35:04.745 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.735+0000 7fdc7f7fe640 1 --2- 172.21.15.157:0/929345148 >> v2:172.21.15.169:3300/0 conn(0x7fdc801592f0 0x7fdc80155c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:04.745 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.735+0000 7fdc7ffff640 1 --2- 172.21.15.157:0/929345148 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc800b03e0 0x7fdc801567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:04.745 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.735+0000 7fdc7effd640 1 -- 172.21.15.157:0/929345148 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2983143756 0 0) 0x7fdc800cb330 con 0x7fdc800c55d0 2024-04-24T23:35:04.746 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.735+0000 7fdc7effd640 1 -- 172.21.15.157:0/929345148 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdc68003520 con 0x7fdc800c55d0 2024-04-24T23:35:04.746 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.736+0000 7fdc7effd640 1 -- 172.21.15.157:0/929345148 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4038911744 0 0) 0x7fdc68003520 con 0x7fdc800c55d0 2024-04-24T23:35:04.746 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.736+0000 7fdc7effd640 1 -- 172.21.15.157:0/929345148 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc800b03e0 msgr2=0x7fdc801567e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:04.746 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.736+0000 7fdc7effd640 1 --2- 172.21.15.157:0/929345148 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc800b03e0 0x7fdc801567e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:04.746 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.736+0000 7fdc7effd640 1 -- 172.21.15.157:0/929345148 >> v2:172.21.15.169:3300/0 conn(0x7fdc801592f0 msgr2=0x7fdc80155c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:04.746 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.736+0000 7fdc7effd640 1 --2- 172.21.15.157:0/929345148 >> v2:172.21.15.169:3300/0 conn(0x7fdc801592f0 0x7fdc80155c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:04.746 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.736+0000 7fdc7effd640 1 -- 172.21.15.157:0/929345148 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc800cb8f0 con 0x7fdc800c55d0 2024-04-24T23:35:04.746 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.736+0000 7fdc7effd640 1 -- 172.21.15.157:0/929345148 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1480525340 0 0) 0x7fdc780019f0 con 0x7fdc800c55d0 2024-04-24T23:35:04.746 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.736+0000 7fdc7effd640 1 -- 172.21.15.157:0/929345148 >> v1:172.21.15.157:6789/0 conn(0x7fdc800c55d0 legacy=0x7fdc800c75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:04.746 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.736+0000 7fdc7effd640 1 --2- 172.21.15.157:0/929345148 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc680039b0 0x7fdc68003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:04.746 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.736+0000 7fdc7effd640 1 --2- 172.21.15.157:0/929345148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc68004800 0x7fdc68004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:04.746 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.736+0000 7fdc7effd640 1 --2- 172.21.15.157:0/929345148 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc800b03e0 0x7fdc680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:04.746 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.736+0000 7fdc7effd640 1 -- 172.21.15.157:0/929345148 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdc800cb610 con 0x7fdc68004800 2024-04-24T23:35:04.747 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.736+0000 7fdc7effd640 1 -- 172.21.15.157:0/929345148 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdc6800b9c0 con 0x7fdc680039b0 2024-04-24T23:35:04.747 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.736+0000 7fdc7effd640 1 -- 172.21.15.157:0/929345148 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fdc6800bb40 con 0x7fdc800b03e0 2024-04-24T23:35:04.747 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.736+0000 7fdc7f7fe640 1 --2- 172.21.15.157:0/929345148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc68004800 0x7fdc68004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:04.747 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.736+0000 7fdc7ffff640 1 --2- 172.21.15.157:0/929345148 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc680039b0 0x7fdc68003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:04.747 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.736+0000 7fdc84830640 1 --2- 172.21.15.157:0/929345148 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc800b03e0 0x7fdc680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:04.747 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.736+0000 7fdc7f7fe640 1 -- 172.21.15.157:0/929345148 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc800b03e0 msgr2=0x7fdc680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:04.747 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.736+0000 7fdc7f7fe640 1 --2- 172.21.15.157:0/929345148 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc800b03e0 0x7fdc680051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:04.747 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.736+0000 7fdc7f7fe640 1 -- 172.21.15.157:0/929345148 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc680039b0 msgr2=0x7fdc68003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:04.747 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.736+0000 7fdc7f7fe640 1 --2- 172.21.15.157:0/929345148 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc680039b0 0x7fdc68003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:04.747 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.736+0000 7fdc7f7fe640 1 -- 172.21.15.157:0/929345148 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc68003520 con 0x7fdc68004800 2024-04-24T23:35:04.748 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.737+0000 7fdc7f7fe640 1 --2- 172.21.15.157:0/929345148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc68004800 0x7fdc68004c80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdc74009ae0 tx=0x7fdc68005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8f5313c2f15a006 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:04.748 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.737+0000 7fdc7effd640 1 -- 172.21.15.157:0/929345148 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc74036070 con 0x7fdc68004800 2024-04-24T23:35:04.748 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.737+0000 7fdc7effd640 1 -- 172.21.15.157:0/929345148 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc74005990 con 0x7fdc68004800 2024-04-24T23:35:04.748 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.737+0000 7fdc7effd640 1 -- 172.21.15.157:0/929345148 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc74009680 con 0x7fdc68004800 2024-04-24T23:35:04.748 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.738+0000 7fdc86293640 1 -- 172.21.15.157:0/929345148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc68004800 msgr2=0x7fdc68004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:04.748 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.738+0000 7fdc86293640 1 --2- 172.21.15.157:0/929345148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc68004800 0x7fdc68004c80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdc74009ae0 tx=0x7fdc68005830 comp rx=0 tx=0).stop 2024-04-24T23:35:04.748 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.738+0000 7fdc84830640 1 -- 172.21.15.157:0/929345148 reap_dead start 2024-04-24T23:35:04.748 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.738+0000 7fdc86293640 1 -- 172.21.15.157:0/929345148 shutdown_connections 2024-04-24T23:35:04.748 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.738+0000 7fdc86293640 1 -- 172.21.15.157:0/929345148 >> 172.21.15.157:0/929345148 conn(0x7fdc80067080 msgr2=0x7fdc80067490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:04.748 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.738+0000 7fdc86293640 1 -- 172.21.15.157:0/929345148 shutdown_connections 2024-04-24T23:35:04.749 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.738+0000 7fdc86293640 1 -- 172.21.15.157:0/929345148 wait complete. 2024-04-24T23:35:04.749 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.739+0000 7fdc86293640 1 Processor -- start 2024-04-24T23:35:04.749 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.739+0000 7fdc86293640 1 -- start start 2024-04-24T23:35:04.749 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.740+0000 7fdc86293640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc68004800 0x7fdc800aebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:04.749 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.740+0000 7fdc86293640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc800c49a0 0x7fdc800ade80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:04.749 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.740+0000 7fdc86293640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc800ae3c0 0x7fdc800c1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:04.749 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.740+0000 7fdc86293640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdc800cafc0 con 0x7fdc800ae3c0 2024-04-24T23:35:04.749 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.740+0000 7fdc86293640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdc801556e0 con 0x7fdc68004800 2024-04-24T23:35:04.749 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.740+0000 7fdc86293640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fdc80159750 con 0x7fdc800c49a0 2024-04-24T23:35:04.749 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.740+0000 7fdc84830640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc800ae3c0 0x7fdc800c1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:04.749 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.740+0000 7fdc84830640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc800ae3c0 0x7fdc800c1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39642/0 (socket says 172.21.15.157:39642) 2024-04-24T23:35:04.749 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.740+0000 7fdc84830640 1 -- 172.21.15.157:0/3529651845 learned_addr learned my addr 172.21.15.157:0/3529651845 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:04.749 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.740+0000 7fdc7f7fe640 1 --2- 172.21.15.157:0/3529651845 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc800c49a0 0x7fdc800ade80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:04.749 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.740+0000 7fdc7ffff640 1 --2- 172.21.15.157:0/3529651845 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc68004800 0x7fdc800aebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:04.749 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.740+0000 7fdc84830640 1 -- 172.21.15.157:0/3529651845 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc800c49a0 msgr2=0x7fdc800ade80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:04.750 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.740+0000 7fdc84830640 1 --2- 172.21.15.157:0/3529651845 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc800c49a0 0x7fdc800ade80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:04.750 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.740+0000 7fdc84830640 1 -- 172.21.15.157:0/3529651845 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc68004800 msgr2=0x7fdc800aebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:04.750 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.740+0000 7fdc84830640 1 --2- 172.21.15.157:0/3529651845 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc68004800 0x7fdc800aebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:04.750 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.740+0000 7fdc84830640 1 -- 172.21.15.157:0/3529651845 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc801eb670 con 0x7fdc800ae3c0 2024-04-24T23:35:04.750 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.740+0000 7fdc7f7fe640 1 --2- 172.21.15.157:0/3529651845 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc800c49a0 0x7fdc800ade80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:35:04.750 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.740+0000 7fdc84830640 1 --2- 172.21.15.157:0/3529651845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc800ae3c0 0x7fdc800c1d20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdc800c1cb0 tx=0x7fdc78007b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:04.750 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.740+0000 7fdc7cff9640 1 -- 172.21.15.157:0/3529651845 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc7800abc0 con 0x7fdc800ae3c0 2024-04-24T23:35:04.750 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.740+0000 7fdc7cff9640 1 -- 172.21.15.157:0/3529651845 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc78018070 con 0x7fdc800ae3c0 2024-04-24T23:35:04.750 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.740+0000 7fdc86293640 1 -- 172.21.15.157:0/3529651845 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc800af330 con 0x7fdc800ae3c0 2024-04-24T23:35:04.750 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.740+0000 7fdc7cff9640 1 -- 172.21.15.157:0/3529651845 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc780133f0 con 0x7fdc800ae3c0 2024-04-24T23:35:04.751 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.740+0000 7fdc86293640 1 -- 172.21.15.157:0/3529651845 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc801eab90 con 0x7fdc800ae3c0 2024-04-24T23:35:04.751 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.741+0000 7fdc86293640 1 -- 172.21.15.157:0/3529651845 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc800681c0 con 0x7fdc800ae3c0 2024-04-24T23:35:04.751 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.742+0000 7fdc7cff9640 1 -- 172.21.15.157:0/3529651845 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdc78013590 con 0x7fdc800ae3c0 2024-04-24T23:35:04.751 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.742+0000 7fdc7cff9640 1 --2- 172.21.15.157:0/3529651845 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fdc6c06be30 0x7fdc6c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:04.751 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.742+0000 7fdc7cff9640 1 -- 172.21.15.157:0/3529651845 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fdc7808b770 con 0x7fdc800ae3c0 2024-04-24T23:35:04.751 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.742+0000 7fdc7ffff640 1 --2- 172.21.15.157:0/3529651845 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fdc6c06be30 0x7fdc6c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:04.751 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.743+0000 7fdc7ffff640 1 --2- 172.21.15.157:0/3529651845 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fdc6c06be30 0x7fdc6c06e2f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdc70009a60 tx=0x7fdc70009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:04.752 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.746+0000 7fdc7cff9640 1 -- 172.21.15.157:0/3529651845 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdc78059d50 con 0x7fdc800ae3c0 2024-04-24T23:35:04.920 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.919+0000 7fdc86293640 1 -- 172.21.15.157:0/3529651845 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdc8006d1d0 con 0x7fdc800ae3c0 2024-04-24T23:35:04.920 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.919+0000 7fdc7cff9640 1 -- 172.21.15.157:0/3529651845 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7fdc78059650 con 0x7fdc800ae3c0 2024-04-24T23:35:04.920 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 64 2024-04-24T23:35:04.922 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.921+0000 7fdc86293640 1 -- 172.21.15.157:0/3529651845 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fdc6c06be30 msgr2=0x7fdc6c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:04.923 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.921+0000 7fdc86293640 1 --2- 172.21.15.157:0/3529651845 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fdc6c06be30 0x7fdc6c06e2f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdc70009a60 tx=0x7fdc70009190 comp rx=0 tx=0).stop 2024-04-24T23:35:04.923 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.921+0000 7fdc86293640 1 -- 172.21.15.157:0/3529651845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc800ae3c0 msgr2=0x7fdc800c1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:04.923 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.921+0000 7fdc86293640 1 --2- 172.21.15.157:0/3529651845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc800ae3c0 0x7fdc800c1d20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdc800c1cb0 tx=0x7fdc78007b10 comp rx=0 tx=0).stop 2024-04-24T23:35:04.923 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.921+0000 7fdc86293640 1 -- 172.21.15.157:0/3529651845 shutdown_connections 2024-04-24T23:35:04.923 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.921+0000 7fdc86293640 1 --2- 172.21.15.157:0/3529651845 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fdc6c06be30 0x7fdc6c06e2f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:04.923 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.921+0000 7fdc86293640 1 --2- 172.21.15.157:0/3529651845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc800ae3c0 0x7fdc800c1d20 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:04.923 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.921+0000 7fdc86293640 1 --2- 172.21.15.157:0/3529651845 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc800c49a0 0x7fdc800ade80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:04.923 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.921+0000 7fdc86293640 1 --2- 172.21.15.157:0/3529651845 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc68004800 0x7fdc800aebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:04.924 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.921+0000 7fdc86293640 1 -- 172.21.15.157:0/3529651845 >> 172.21.15.157:0/3529651845 conn(0x7fdc80067080 msgr2=0x7fdc80155a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:04.924 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.921+0000 7fdc86293640 1 -- 172.21.15.157:0/3529651845 shutdown_connections 2024-04-24T23:35:04.924 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:04.921+0000 7fdc86293640 1 -- 172.21.15.157:0/3529651845 wait complete. 2024-04-24T23:35:04.933 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T23:35:05.033 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.032+0000 7f75d0fce640 1 Processor -- start 2024-04-24T23:35:05.033 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.033+0000 7f75d0fce640 1 -- start start 2024-04-24T23:35:05.036 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.033+0000 7f75d0fce640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f75cc072c50 0x7f75cc06d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:05.036 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.033+0000 7f75d0fce640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75cc153330 0x7f75cc153780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:05.036 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.033+0000 7f75d0fce640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75cc06e240 con 0x7f75cc06dd70 2024-04-24T23:35:05.037 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.033+0000 7f75d0fce640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f75cc156a50 con 0x7f75cc072c50 2024-04-24T23:35:05.037 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.033+0000 7f75d0fce640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f75cc156cf0 con 0x7f75cc153330 2024-04-24T23:35:05.037 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.034+0000 7f75cad9d640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75cc153330 0x7f75cc153780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:05.037 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.034+0000 7f75ca59c640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f75cc072c50 0x7f75cc06d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:05.037 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.034+0000 7f75cad9d640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75cc153330 0x7f75cc153780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.157:50742/0 (socket says 172.21.15.157:50742) 2024-04-24T23:35:05.037 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.034+0000 7f75ca59c640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f75cc072c50 0x7f75cc06d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.157:36606/0 (socket says 172.21.15.157:36606) 2024-04-24T23:35:05.037 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.034+0000 7f75cad9d640 1 -- 172.21.15.157:0/1496243360 learned_addr learned my addr 172.21.15.157:0/1496243360 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:05.037 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.034+0000 7f75c9d9b640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f75cc06dd70 0x7f75cc150720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.157:56588/0 (socket says 172.21.15.157:56588) 2024-04-24T23:35:05.037 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.034+0000 7f75ca59c640 1 -- 172.21.15.157:0/1496243360 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75cc153330 msgr2=0x7f75cc153780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:05.037 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.034+0000 7f75ca59c640 1 --2- 172.21.15.157:0/1496243360 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75cc153330 0x7f75cc153780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:05.038 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.034+0000 7f75ca59c640 1 -- 172.21.15.157:0/1496243360 >> v1:172.21.15.169:6789/0 conn(0x7f75cc06dd70 legacy=0x7f75cc150720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:05.038 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.034+0000 7f75ca59c640 1 -- 172.21.15.157:0/1496243360 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75cc156fd0 con 0x7f75cc072c50 2024-04-24T23:35:05.038 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.034+0000 7f75cad9d640 1 --2- 172.21.15.157:0/1496243360 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75cc153330 0x7f75cc153780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:35:05.038 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.035+0000 7f75ca59c640 1 --2- 172.21.15.157:0/1496243360 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75cc072c50 0x7f75cc06d830 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f75c00097a0 tx=0x7f75c002a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=855e56dcd3ba4595 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:05.038 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.035+0000 7f75c959a640 1 -- 172.21.15.157:0/1496243360 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c0037070 con 0x7f75cc072c50 2024-04-24T23:35:05.038 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.035+0000 7f75c959a640 1 -- 172.21.15.157:0/1496243360 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75c002adb0 con 0x7f75cc072c50 2024-04-24T23:35:05.038 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.035+0000 7f75c959a640 1 -- 172.21.15.157:0/1496243360 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c0032660 con 0x7f75cc072c50 2024-04-24T23:35:05.044 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.036+0000 7f75d0fce640 1 -- 172.21.15.157:0/1496243360 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75cc072c50 msgr2=0x7f75cc06d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:05.044 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.036+0000 7f75d0fce640 1 --2- 172.21.15.157:0/1496243360 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75cc072c50 0x7f75cc06d830 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f75c00097a0 tx=0x7f75c002a060 comp rx=0 tx=0).stop 2024-04-24T23:35:05.044 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.036+0000 7f75d0fce640 1 -- 172.21.15.157:0/1496243360 shutdown_connections 2024-04-24T23:35:05.044 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.036+0000 7f75d0fce640 1 --2- 172.21.15.157:0/1496243360 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75cc153330 0x7f75cc153780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:05.044 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.036+0000 7f75d0fce640 1 --2- 172.21.15.157:0/1496243360 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75cc072c50 0x7f75cc06d830 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:05.045 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.036+0000 7f75d0fce640 1 -- 172.21.15.157:0/1496243360 >> 172.21.15.157:0/1496243360 conn(0x7f75cc0c68e0 msgr2=0x7f75cc0c6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:05.045 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.036+0000 7f75d0fce640 1 -- 172.21.15.157:0/1496243360 shutdown_connections 2024-04-24T23:35:05.045 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.036+0000 7f75d0fce640 1 -- 172.21.15.157:0/1496243360 wait complete. 2024-04-24T23:35:05.045 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.037+0000 7f75d0fce640 1 Processor -- start 2024-04-24T23:35:05.045 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.037+0000 7f75d0fce640 1 -- start start 2024-04-24T23:35:05.045 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.038+0000 7f75d0fce640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75cc06dd70 0x7f75cc0a83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:05.045 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.038+0000 7f75d0fce640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75cc072c50 0x7f75cc0a9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:05.045 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.038+0000 7f75d0fce640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75cc153330 0x7f75cc0a9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:05.045 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.038+0000 7f75d0fce640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f75cc153cc0 con 0x7f75cc06dd70 2024-04-24T23:35:05.045 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.038+0000 7f75d0fce640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f75cc1577d0 con 0x7f75cc072c50 2024-04-24T23:35:05.045 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.038+0000 7f75d0fce640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f75cc0c2fa0 con 0x7f75cc153330 2024-04-24T23:35:05.046 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.038+0000 7f75ca59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75cc06dd70 0x7f75cc0a83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:05.046 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.038+0000 7f75ca59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75cc06dd70 0x7f75cc0a83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39652/0 (socket says 172.21.15.157:39652) 2024-04-24T23:35:05.046 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.038+0000 7f75ca59c640 1 -- 172.21.15.157:0/4176341284 learned_addr learned my addr 172.21.15.157:0/4176341284 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:05.046 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.038+0000 7f75cad9d640 1 --2- 172.21.15.157:0/4176341284 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75cc153330 0x7f75cc0a9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:05.046 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.038+0000 7f75c9d9b640 1 --2- 172.21.15.157:0/4176341284 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75cc072c50 0x7f75cc0a9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:05.046 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.038+0000 7f75ca59c640 1 -- 172.21.15.157:0/4176341284 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75cc153330 msgr2=0x7f75cc0a9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:05.046 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.038+0000 7f75ca59c640 1 --2- 172.21.15.157:0/4176341284 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75cc153330 0x7f75cc0a9ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:05.047 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.038+0000 7f75ca59c640 1 -- 172.21.15.157:0/4176341284 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75cc072c50 msgr2=0x7f75cc0a9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:05.047 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.038+0000 7f75ca59c640 1 --2- 172.21.15.157:0/4176341284 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75cc072c50 0x7f75cc0a9990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:05.047 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.038+0000 7f75ca59c640 1 -- 172.21.15.157:0/4176341284 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75cc0a6bb0 con 0x7f75cc06dd70 2024-04-24T23:35:05.047 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.038+0000 7f75ca59c640 1 --2- 172.21.15.157:0/4176341284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75cc06dd70 0x7f75cc0a83d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f75c002a0a0 tx=0x7f75c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:05.047 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.038+0000 7f75baffd640 1 -- 172.21.15.157:0/4176341284 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c0041070 con 0x7f75cc06dd70 2024-04-24T23:35:05.047 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.038+0000 7f75baffd640 1 -- 172.21.15.157:0/4176341284 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75c003c670 con 0x7f75cc06dd70 2024-04-24T23:35:05.048 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.038+0000 7f75d0fce640 1 -- 172.21.15.157:0/4176341284 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75cc0a6e90 con 0x7f75cc06dd70 2024-04-24T23:35:05.048 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.038+0000 7f75baffd640 1 -- 172.21.15.157:0/4176341284 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c0037040 con 0x7f75cc06dd70 2024-04-24T23:35:05.048 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.038+0000 7f75d0fce640 1 -- 172.21.15.157:0/4176341284 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75cc0a7450 con 0x7f75cc06dd70 2024-04-24T23:35:05.048 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.039+0000 7f75d0fce640 1 -- 172.21.15.157:0/4176341284 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75cc0681c0 con 0x7f75cc06dd70 2024-04-24T23:35:05.048 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.040+0000 7f75baffd640 1 -- 172.21.15.157:0/4176341284 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f75c0044070 con 0x7f75cc06dd70 2024-04-24T23:35:05.048 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.040+0000 7f75baffd640 1 --2- 172.21.15.157:0/4176341284 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f75ac06be60 0x7f75ac06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:05.049 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.040+0000 7f75baffd640 1 -- 172.21.15.157:0/4176341284 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f75c00ab890 con 0x7f75cc06dd70 2024-04-24T23:35:05.049 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.041+0000 7f75c9d9b640 1 --2- 172.21.15.157:0/4176341284 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f75ac06be60 0x7f75ac06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:05.049 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.041+0000 7f75c9d9b640 1 --2- 172.21.15.157:0/4176341284 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f75ac06be60 0x7f75ac06e320 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f75cc1ea080 tx=0x7f75bc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:05.049 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.045+0000 7f75baffd640 1 -- 172.21.15.157:0/4176341284 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f75c0079e80 con 0x7f75cc06dd70 2024-04-24T23:35:05.219 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.219+0000 7f75d0fce640 1 -- 172.21.15.157:0/4176341284 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f75cc0a5e20 con 0x7f75cc06dd70 2024-04-24T23:35:05.220 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.219+0000 7f75baffd640 1 -- 172.21.15.157:0/4176341284 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7f75c0079780 con 0x7f75cc06dd70 2024-04-24T23:35:05.220 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2024-04-24T23:35:05.222 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.221+0000 7f75d0fce640 1 -- 172.21.15.157:0/4176341284 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f75ac06be60 msgr2=0x7f75ac06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:05.222 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.221+0000 7f75d0fce640 1 --2- 172.21.15.157:0/4176341284 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f75ac06be60 0x7f75ac06e320 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f75cc1ea080 tx=0x7f75bc009040 comp rx=0 tx=0).stop 2024-04-24T23:35:05.222 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.221+0000 7f75d0fce640 1 -- 172.21.15.157:0/4176341284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75cc06dd70 msgr2=0x7f75cc0a83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:05.222 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.221+0000 7f75d0fce640 1 --2- 172.21.15.157:0/4176341284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75cc06dd70 0x7f75cc0a83d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f75c002a0a0 tx=0x7f75c00046c0 comp rx=0 tx=0).stop 2024-04-24T23:35:05.222 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.221+0000 7f75d0fce640 1 -- 172.21.15.157:0/4176341284 shutdown_connections 2024-04-24T23:35:05.222 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.221+0000 7f75d0fce640 1 --2- 172.21.15.157:0/4176341284 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f75ac06be60 0x7f75ac06e320 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:05.223 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.221+0000 7f75d0fce640 1 --2- 172.21.15.157:0/4176341284 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75cc153330 0x7f75cc0a9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:05.223 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.221+0000 7f75d0fce640 1 --2- 172.21.15.157:0/4176341284 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75cc072c50 0x7f75cc0a9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:05.223 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.221+0000 7f75d0fce640 1 --2- 172.21.15.157:0/4176341284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75cc06dd70 0x7f75cc0a83d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:05.223 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.221+0000 7f75d0fce640 1 -- 172.21.15.157:0/4176341284 >> 172.21.15.157:0/4176341284 conn(0x7f75cc0c68e0 msgr2=0x7f75cc151150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:05.223 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.221+0000 7f75d0fce640 1 -- 172.21.15.157:0/4176341284 shutdown_connections 2024-04-24T23:35:05.223 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.221+0000 7f75d0fce640 1 -- 172.21.15.157:0/4176341284 wait complete. 2024-04-24T23:35:05.233 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T23:35:05.331 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.331+0000 7f22f5c5d640 1 Processor -- start 2024-04-24T23:35:05.332 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.331+0000 7f22f5c5d640 1 -- start start 2024-04-24T23:35:05.333 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.332+0000 7f22f5c5d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f22f0073100 0x7f22f00c2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:05.333 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.332+0000 7f22f5c5d640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22f0067270 0x7f22f015fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:05.334 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.332+0000 7f22f5c5d640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22f01553a0 con 0x7f22f00c14f0 2024-04-24T23:35:05.334 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.332+0000 7f22f5c5d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f22f00c3b00 con 0x7f22f0073100 2024-04-24T23:35:05.334 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.332+0000 7f22f5c5d640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f22f0155660 con 0x7f22f0067270 2024-04-24T23:35:05.334 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.332+0000 7f22ef7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f22f0073100 0x7f22f00c2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:05.334 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.332+0000 7f22ef7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f22f0073100 0x7f22f00c2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39660/0 (socket says 172.21.15.157:39660) 2024-04-24T23:35:05.334 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.332+0000 7f22ef7fe640 1 -- 172.21.15.157:0/2603202353 learned_addr learned my addr 172.21.15.157:0/2603202353 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:05.334 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.332+0000 7f22effff640 1 --2- 172.21.15.157:0/2603202353 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22f0067270 0x7f22f015fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:05.334 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.332+0000 7f22ef7fe640 1 -- 172.21.15.157:0/2603202353 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22f0067270 msgr2=0x7f22f015fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:05.334 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.332+0000 7f22ef7fe640 1 --2- 172.21.15.157:0/2603202353 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22f0067270 0x7f22f015fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:05.334 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.332+0000 7f22ef7fe640 1 -- 172.21.15.157:0/2603202353 >> v1:172.21.15.169:6789/0 conn(0x7f22f00c14f0 legacy=0x7f22f006b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:05.334 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.332+0000 7f22ef7fe640 1 -- 172.21.15.157:0/2603202353 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22f00c0730 con 0x7f22f0073100 2024-04-24T23:35:05.334 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.333+0000 7f22ef7fe640 1 --2- 172.21.15.157:0/2603202353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22f0073100 0x7f22f00c2770 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f22f00c82b0 tx=0x7f22e002a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7b4380932c786d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:05.335 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.333+0000 7f22ee7fc640 1 -- 172.21.15.157:0/2603202353 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e0037070 con 0x7f22f0073100 2024-04-24T23:35:05.335 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.333+0000 7f22ee7fc640 1 -- 172.21.15.157:0/2603202353 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22e002adc0 con 0x7f22f0073100 2024-04-24T23:35:05.335 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.333+0000 7f22ee7fc640 1 -- 172.21.15.157:0/2603202353 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e0032660 con 0x7f22f0073100 2024-04-24T23:35:05.337 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.333+0000 7f22f5c5d640 1 -- 172.21.15.157:0/2603202353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22f0073100 msgr2=0x7f22f00c2770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:05.337 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.333+0000 7f22f5c5d640 1 --2- 172.21.15.157:0/2603202353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22f0073100 0x7f22f00c2770 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f22f00c82b0 tx=0x7f22e002a0b0 comp rx=0 tx=0).stop 2024-04-24T23:35:05.337 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.334+0000 7f22f5c5d640 1 -- 172.21.15.157:0/2603202353 shutdown_connections 2024-04-24T23:35:05.337 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.334+0000 7f22f5c5d640 1 --2- 172.21.15.157:0/2603202353 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22f0067270 0x7f22f015fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:05.338 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.334+0000 7f22f5c5d640 1 --2- 172.21.15.157:0/2603202353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22f0073100 0x7f22f00c2770 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:05.338 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.334+0000 7f22f5c5d640 1 -- 172.21.15.157:0/2603202353 >> 172.21.15.157:0/2603202353 conn(0x7f22f00c3d60 msgr2=0x7f22f00c4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:05.338 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.334+0000 7f22f5c5d640 1 -- 172.21.15.157:0/2603202353 shutdown_connections 2024-04-24T23:35:05.338 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.334+0000 7f22f5c5d640 1 -- 172.21.15.157:0/2603202353 wait complete. 2024-04-24T23:35:05.338 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.335+0000 7f22f5c5d640 1 Processor -- start 2024-04-24T23:35:05.338 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.335+0000 7f22f5c5d640 1 -- start start 2024-04-24T23:35:05.338 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.335+0000 7f22f5c5d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22f0073100 0x7f22f006bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:05.338 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.335+0000 7f22f5c5d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22f00c14f0 0x7f22f0151ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:05.338 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.335+0000 7f22f5c5d640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22f0152420 0x7f22f01eb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:05.338 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.335+0000 7f22f5c5d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f22f00c0fd0 con 0x7f22f0073100 2024-04-24T23:35:05.338 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.335+0000 7f22f5c5d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f22f00c0ad0 con 0x7f22f00c14f0 2024-04-24T23:35:05.338 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.335+0000 7f22f5c5d640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f22f00711a0 con 0x7f22f0152420 2024-04-24T23:35:05.338 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.336+0000 7f22ef7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22f0073100 0x7f22f006bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:05.339 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.336+0000 7f22ef7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22f0073100 0x7f22f006bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39676/0 (socket says 172.21.15.157:39676) 2024-04-24T23:35:05.339 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.336+0000 7f22ef7fe640 1 -- 172.21.15.157:0/651327256 learned_addr learned my addr 172.21.15.157:0/651327256 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:05.339 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.336+0000 7f22eeffd640 1 --2- 172.21.15.157:0/651327256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22f00c14f0 0x7f22f0151ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:05.339 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.336+0000 7f22effff640 1 --2- 172.21.15.157:0/651327256 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22f0152420 0x7f22f01eb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:05.339 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.336+0000 7f22ef7fe640 1 -- 172.21.15.157:0/651327256 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22f0152420 msgr2=0x7f22f01eb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:05.339 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.336+0000 7f22ef7fe640 1 --2- 172.21.15.157:0/651327256 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22f0152420 0x7f22f01eb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:05.339 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.336+0000 7f22ef7fe640 1 -- 172.21.15.157:0/651327256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22f00c14f0 msgr2=0x7f22f0151ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:05.339 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.336+0000 7f22ef7fe640 1 --2- 172.21.15.157:0/651327256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22f00c14f0 0x7f22f0151ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:05.339 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.336+0000 7f22ef7fe640 1 -- 172.21.15.157:0/651327256 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22f01eeed0 con 0x7f22f0073100 2024-04-24T23:35:05.339 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.336+0000 7f22ef7fe640 1 --2- 172.21.15.157:0/651327256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22f0073100 0x7f22f006bb70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f22e0000c00 tx=0x7f22e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:05.339 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.336+0000 7f22e7fff640 1 -- 172.21.15.157:0/651327256 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e0037040 con 0x7f22f0073100 2024-04-24T23:35:05.339 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.336+0000 7f22e7fff640 1 -- 172.21.15.157:0/651327256 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22e0042070 con 0x7f22f0073100 2024-04-24T23:35:05.339 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.336+0000 7f22e7fff640 1 -- 172.21.15.157:0/651327256 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e003d880 con 0x7f22f0073100 2024-04-24T23:35:05.339 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.336+0000 7f22f5c5d640 1 -- 172.21.15.157:0/651327256 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22f01ef1b0 con 0x7f22f0073100 2024-04-24T23:35:05.340 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.336+0000 7f22f5c5d640 1 -- 172.21.15.157:0/651327256 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22f006c0b0 con 0x7f22f0073100 2024-04-24T23:35:05.340 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.337+0000 7f22f5c5d640 1 -- 172.21.15.157:0/651327256 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22f00681c0 con 0x7f22f0073100 2024-04-24T23:35:05.341 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.338+0000 7f22e7fff640 1 -- 172.21.15.157:0/651327256 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f22e002ac40 con 0x7f22f0073100 2024-04-24T23:35:05.341 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.338+0000 7f22e7fff640 1 --2- 172.21.15.157:0/651327256 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f22d006be30 0x7f22d006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:05.341 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.338+0000 7f22e7fff640 1 -- 172.21.15.157:0/651327256 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f22e00ab7e0 con 0x7f22f0073100 2024-04-24T23:35:05.341 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.338+0000 7f22eeffd640 1 --2- 172.21.15.157:0/651327256 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f22d006be30 0x7f22d006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:05.341 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.339+0000 7f22eeffd640 1 --2- 172.21.15.157:0/651327256 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f22d006be30 0x7f22d006e2f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f22f01ec8f0 tx=0x7f22d8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:05.343 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.342+0000 7f22e7fff640 1 -- 172.21.15.157:0/651327256 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f22e0079ee0 con 0x7f22f0073100 2024-04-24T23:35:05.528 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.527+0000 7f22f5c5d640 1 -- 172.21.15.157:0/651327256 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f22f006f0c0 con 0x7f22f0073100 2024-04-24T23:35:05.528 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.528+0000 7f22e7fff640 1 -- 172.21.15.157:0/651327256 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7f22e00797e0 con 0x7f22f0073100 2024-04-24T23:35:05.529 INFO:teuthology.orchestra.run.smithi157.stderr:listed 0 entries 2024-04-24T23:35:05.531 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.530+0000 7f22f5c5d640 1 -- 172.21.15.157:0/651327256 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f22d006be30 msgr2=0x7f22d006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:05.531 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.530+0000 7f22f5c5d640 1 --2- 172.21.15.157:0/651327256 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f22d006be30 0x7f22d006e2f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f22f01ec8f0 tx=0x7f22d8009040 comp rx=0 tx=0).stop 2024-04-24T23:35:05.531 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.530+0000 7f22f5c5d640 1 -- 172.21.15.157:0/651327256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22f0073100 msgr2=0x7f22f006bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:05.531 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.530+0000 7f22f5c5d640 1 --2- 172.21.15.157:0/651327256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22f0073100 0x7f22f006bb70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f22e0000c00 tx=0x7f22e00046c0 comp rx=0 tx=0).stop 2024-04-24T23:35:05.531 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.530+0000 7f22f5c5d640 1 -- 172.21.15.157:0/651327256 shutdown_connections 2024-04-24T23:35:05.531 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.530+0000 7f22f5c5d640 1 --2- 172.21.15.157:0/651327256 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f22d006be30 0x7f22d006e2f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:05.531 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.530+0000 7f22f5c5d640 1 --2- 172.21.15.157:0/651327256 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22f0152420 0x7f22f01eb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:05.531 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.530+0000 7f22f5c5d640 1 --2- 172.21.15.157:0/651327256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22f00c14f0 0x7f22f0151ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:05.531 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.530+0000 7f22f5c5d640 1 --2- 172.21.15.157:0/651327256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22f0073100 0x7f22f006bb70 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:05.531 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.530+0000 7f22f5c5d640 1 -- 172.21.15.157:0/651327256 >> 172.21.15.157:0/651327256 conn(0x7f22f00c3d60 msgr2=0x7f22f014d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:05.531 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.530+0000 7f22f5c5d640 1 -- 172.21.15.157:0/651327256 shutdown_connections 2024-04-24T23:35:05.531 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.530+0000 7f22f5c5d640 1 -- 172.21.15.157:0/651327256 wait complete. 2024-04-24T23:35:05.541 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-24T23:35:05.541 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:35:05.556 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:35:05.694 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.693+0000 7f12333dc640 1 Processor -- start 2024-04-24T23:35:05.694 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.694+0000 7f12333dc640 1 -- start start 2024-04-24T23:35:05.696 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.694+0000 7f12333dc640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f122c0be360 0x7f122c155350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:05.696 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.694+0000 7f12333dc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f122c155890 0x7f122c15fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:05.696 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.694+0000 7f12333dc640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f122c0c8bb0 con 0x7f122c152b40 2024-04-24T23:35:05.696 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.694+0000 7f12333dc640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f122c0c8e90 con 0x7f122c155890 2024-04-24T23:35:05.696 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.694+0000 7f12333dc640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f122c159480 con 0x7f122c0be360 2024-04-24T23:35:05.696 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.694+0000 7f1231979640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f122c155890 0x7f122c15fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:05.696 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.694+0000 7f1231979640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f122c155890 0x7f122c15fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39678/0 (socket says 172.21.15.157:39678) 2024-04-24T23:35:05.697 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.694+0000 7f1231979640 1 -- 172.21.15.157:0/719950415 learned_addr learned my addr 172.21.15.157:0/719950415 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:05.697 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.694+0000 7f1230977640 1 --2- 172.21.15.157:0/719950415 >> v2:172.21.15.169:3300/0 conn(0x7f122c0be360 0x7f122c155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:05.697 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.695+0000 7f1231979640 1 -- 172.21.15.157:0/719950415 >> v2:172.21.15.169:3300/0 conn(0x7f122c0be360 msgr2=0x7f122c155350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:05.697 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.695+0000 7f1231979640 1 --2- 172.21.15.157:0/719950415 >> v2:172.21.15.169:3300/0 conn(0x7f122c0be360 0x7f122c155350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:05.697 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.695+0000 7f1231979640 1 -- 172.21.15.157:0/719950415 >> v1:172.21.15.169:6789/0 conn(0x7f122c152b40 legacy=0x7f122c0c5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:05.697 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.695+0000 7f1231979640 1 -- 172.21.15.157:0/719950415 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f122c159760 con 0x7f122c155890 2024-04-24T23:35:05.697 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.695+0000 7f1230977640 1 --2- 172.21.15.157:0/719950415 >> v2:172.21.15.169:3300/0 conn(0x7f122c0be360 0x7f122c155350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:35:05.697 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.695+0000 7f1231979640 1 --2- 172.21.15.157:0/719950415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f122c155890 0x7f122c15fcd0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f122c06d120 tx=0x7f122402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89ca4f585b1791f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:05.697 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.695+0000 7f1223fff640 1 -- 172.21.15.157:0/719950415 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1224036070 con 0x7f122c155890 2024-04-24T23:35:05.697 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.696+0000 7f1223fff640 1 -- 172.21.15.157:0/719950415 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1224031600 con 0x7f122c155890 2024-04-24T23:35:05.697 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.696+0000 7f1223fff640 1 -- 172.21.15.157:0/719950415 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12240319d0 con 0x7f122c155890 2024-04-24T23:35:05.700 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.696+0000 7f12333dc640 1 -- 172.21.15.157:0/719950415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f122c155890 msgr2=0x7f122c15fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:05.700 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.696+0000 7f12333dc640 1 --2- 172.21.15.157:0/719950415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f122c155890 0x7f122c15fcd0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f122c06d120 tx=0x7f122402a610 comp rx=0 tx=0).stop 2024-04-24T23:35:05.701 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.696+0000 7f12333dc640 1 -- 172.21.15.157:0/719950415 shutdown_connections 2024-04-24T23:35:05.701 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.696+0000 7f12333dc640 1 --2- 172.21.15.157:0/719950415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f122c155890 0x7f122c15fcd0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:05.701 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.696+0000 7f12333dc640 1 --2- 172.21.15.157:0/719950415 >> v2:172.21.15.169:3300/0 conn(0x7f122c0be360 0x7f122c155350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:05.701 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.696+0000 7f12333dc640 1 -- 172.21.15.157:0/719950415 >> 172.21.15.157:0/719950415 conn(0x7f122c0672c0 msgr2=0x7f122c0676d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:05.701 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.696+0000 7f12333dc640 1 -- 172.21.15.157:0/719950415 shutdown_connections 2024-04-24T23:35:05.701 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.696+0000 7f12333dc640 1 -- 172.21.15.157:0/719950415 wait complete. 2024-04-24T23:35:05.701 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.697+0000 7f12333dc640 1 Processor -- start 2024-04-24T23:35:05.701 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.697+0000 7f12333dc640 1 -- start start 2024-04-24T23:35:05.701 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.698+0000 7f12333dc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f122c0be360 0x7f122c1eef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:05.701 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.698+0000 7f12333dc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f122c152b40 0x7f122c147720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:05.701 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.698+0000 7f12333dc640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f122c155890 0x7f122c1eaf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:05.701 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.698+0000 7f12333dc640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f122c0caa00 con 0x7f122c152b40 2024-04-24T23:35:05.701 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.698+0000 7f12333dc640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f122c0cb210 con 0x7f122c0be360 2024-04-24T23:35:05.702 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.698+0000 7f12333dc640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f122c155d00 con 0x7f122c155890 2024-04-24T23:35:05.702 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.698+0000 7f1230977640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f122c152b40 0x7f122c147720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:05.702 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.698+0000 7f1230977640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f122c152b40 0x7f122c147720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39686/0 (socket says 172.21.15.157:39686) 2024-04-24T23:35:05.702 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.698+0000 7f1230977640 1 -- 172.21.15.157:0/3772894600 learned_addr learned my addr 172.21.15.157:0/3772894600 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:05.702 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.698+0000 7f1230977640 1 -- 172.21.15.157:0/3772894600 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f122c155890 msgr2=0x7f122c1eaf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:05.702 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.698+0000 7f1230977640 1 --2- 172.21.15.157:0/3772894600 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f122c155890 0x7f122c1eaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:05.702 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.698+0000 7f1231178640 1 --2- 172.21.15.157:0/3772894600 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f122c0be360 0x7f122c1eef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:05.702 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.698+0000 7f1230977640 1 -- 172.21.15.157:0/3772894600 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f122c0be360 msgr2=0x7f122c1eef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:05.702 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.698+0000 7f1230977640 1 --2- 172.21.15.157:0/3772894600 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f122c0be360 0x7f122c1eef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:05.702 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.698+0000 7f1230977640 1 -- 172.21.15.157:0/3772894600 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f122c0ad130 con 0x7f122c152b40 2024-04-24T23:35:05.702 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.698+0000 7f1231178640 1 --2- 172.21.15.157:0/3772894600 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f122c0be360 0x7f122c1eef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:35:05.702 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.698+0000 7f1230977640 1 --2- 172.21.15.157:0/3772894600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f122c152b40 0x7f122c147720 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f121c00e810 tx=0x7f121c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:05.702 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.699+0000 7f1221ffb640 1 -- 172.21.15.157:0/3772894600 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f121c00f040 con 0x7f122c152b40 2024-04-24T23:35:05.703 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.699+0000 7f1221ffb640 1 -- 172.21.15.157:0/3772894600 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f121c004030 con 0x7f122c152b40 2024-04-24T23:35:05.703 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.699+0000 7f1221ffb640 1 -- 172.21.15.157:0/3772894600 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f121c004400 con 0x7f122c152b40 2024-04-24T23:35:05.703 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.699+0000 7f12333dc640 1 -- 172.21.15.157:0/3772894600 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f122c0ad410 con 0x7f122c152b40 2024-04-24T23:35:05.703 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.699+0000 7f12333dc640 1 -- 172.21.15.157:0/3772894600 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f122c147c60 con 0x7f122c152b40 2024-04-24T23:35:05.703 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.700+0000 7f12333dc640 1 -- 172.21.15.157:0/3772894600 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f122c0681c0 con 0x7f122c152b40 2024-04-24T23:35:05.703 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.700+0000 7f1221ffb640 1 -- 172.21.15.157:0/3772894600 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f121c00c6d0 con 0x7f122c152b40 2024-04-24T23:35:05.704 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.700+0000 7f1221ffb640 1 --2- 172.21.15.157:0/3772894600 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f121006be30 0x7f121006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:05.704 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.700+0000 7f1221ffb640 1 -- 172.21.15.157:0/3772894600 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f121c08af00 con 0x7f122c152b40 2024-04-24T23:35:05.704 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.701+0000 7f1231178640 1 --2- 172.21.15.157:0/3772894600 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f121006be30 0x7f121006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:05.704 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.701+0000 7f1231178640 1 --2- 172.21.15.157:0/3772894600 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f121006be30 0x7f121006e2f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f122c1588b0 tx=0x7f1228009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:05.705 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.705+0000 7f1221ffb640 1 -- 172.21.15.157:0/3772894600 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f121c017020 con 0x7f122c152b40 2024-04-24T23:35:05.879 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.879+0000 7f12333dc640 1 -- 172.21.15.157:0/3772894600 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f122c06d3f0 con 0x7f122c152b40 2024-04-24T23:35:05.881 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.880+0000 7f1221ffb640 1 -- 172.21.15.157:0/3772894600 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18538 (secure 0 0 0) 0x7f121c059600 con 0x7f122c152b40 2024-04-24T23:35:05.881 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:35:05.881 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":25,"fsid":"223685fb-5b64-46a5-9857-3791c9c07df2","created":"2024-04-24T23:34:37.504202+0000","modified":"2024-04-24T23:34:58.744500+0000","last_up_change":"2024-04-24T23:34:47.584963+0000","last_in_change":"2024-04-24T23:34:39.632954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:34:47.796655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:34:52.177208+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:34:52.667022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:34:56.395517+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"95d14ac8-fccf-4261-be8e-edafa746f629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6809","nonce":4003822160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6811","nonce":4003822160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6815","nonce":4003822160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6813","nonce":4003822160}]},"public_addr":"172.21.15.157:6809/4003822160","cluster_addr":"172.21.15.157:6811/4003822160","heartbeat_back_addr":"172.21.15.157:6815/4003822160","heartbeat_front_addr":"172.21.15.157:6813/4003822160","state":["exists","up"]},{"osd":1,"uuid":"7be79da9-8eec-4990-9178-71d602e39d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6817","nonce":1557575576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6819","nonce":1557575576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6823","nonce":1557575576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6821","nonce":1557575576}]},"public_addr":"172.21.15.157:6817/1557575576","cluster_addr":"172.21.15.157:6819/1557575576","heartbeat_back_addr":"172.21.15.157:6823/1557575576","heartbeat_front_addr":"172.21.15.157:6821/1557575576","state":["exists","up"]},{"osd":2,"uuid":"0127ed00-b7c3-466d-bfb0-01b0fd315f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6825","nonce":2793948092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6827","nonce":2793948092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6831","nonce":2793948092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6829","nonce":2793948092}]},"public_addr":"172.21.15.157:6825/2793948092","cluster_addr":"172.21.15.157:6827/2793948092","heartbeat_back_addr":"172.21.15.157:6831/2793948092","heartbeat_front_addr":"172.21.15.157:6829/2793948092","state":["exists","up"]},{"osd":3,"uuid":"f2331acc-aa37-433e-868e-3561b6039562","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6801","nonce":422758549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6803","nonce":422758549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6807","nonce":422758549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6805","nonce":422758549}]},"public_addr":"172.21.15.157:6801/422758549","cluster_addr":"172.21.15.157:6803/422758549","heartbeat_back_addr":"172.21.15.157:6807/422758549","heartbeat_front_addr":"172.21.15.157:6805/422758549","state":["exists","up"]},{"osd":4,"uuid":"1998854b-7398-43f5-ab7d-79fa0debcf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6801","nonce":321910099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6803","nonce":321910099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6807","nonce":321910099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6805","nonce":321910099}]},"public_addr":"172.21.15.169:6801/321910099","cluster_addr":"172.21.15.169:6803/321910099","heartbeat_back_addr":"172.21.15.169:6807/321910099","heartbeat_front_addr":"172.21.15.169:6805/321910099","state":["exists","up"]},{"osd":5,"uuid":"c37153ff-9b2b-48b9-a684-2c82b6f39f15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6825","nonce":243321711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6827","nonce":243321711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6831","nonce":243321711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6829","nonce":243321711}]},"public_addr":"172.21.15.169:6825/243321711","cluster_addr":"172.21.15.169:6827/243321711","heartbeat_back_addr":"172.21.15.169:6831/243321711","heartbeat_front_addr":"172.21.15.169:6829/243321711","state":["exists","up"]},{"osd":6,"uuid":"8db79e1c-f36b-49cb-8d8a-a20c2aec57c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6809","nonce":3441420785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6811","nonce":3441420785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6815","nonce":3441420785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6813","nonce":3441420785}]},"public_addr":"172.21.15.169:6809/3441420785","cluster_addr":"172.21.15.169:6811/3441420785","heartbeat_back_addr":"172.21.15.169:6815/3441420785","heartbeat_front_addr":"172.21.15.169:6813/3441420785","state":["exists","up"]},{"osd":7,"uuid":"928586e9-296c-4b8b-9b27-42f9f155dbdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6817","nonce":1663179020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6819","nonce":1663179020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6823","nonce":1663179020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6821","nonce":1663179020}]},"public_addr":"172.21.15.169:6817/1663179020","cluster_addr":"172.21.15.169:6819/1663179020","heartbeat_back_addr":"172.21.15.169:6823/1663179020","heartbeat_front_addr":"172.21.15.169:6821/1663179020","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.848039+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.826489+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.801120+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.727706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.824343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.791915+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.878073+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.810831+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:35:05.883 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.882+0000 7f12333dc640 1 -- 172.21.15.157:0/3772894600 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f121006be30 msgr2=0x7f121006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:05.883 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.882+0000 7f12333dc640 1 --2- 172.21.15.157:0/3772894600 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f121006be30 0x7f121006e2f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f122c1588b0 tx=0x7f1228009040 comp rx=0 tx=0).stop 2024-04-24T23:35:05.883 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.882+0000 7f12333dc640 1 -- 172.21.15.157:0/3772894600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f122c152b40 msgr2=0x7f122c147720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:05.883 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.882+0000 7f12333dc640 1 --2- 172.21.15.157:0/3772894600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f122c152b40 0x7f122c147720 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f121c00e810 tx=0x7f121c00ecd0 comp rx=0 tx=0).stop 2024-04-24T23:35:05.883 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.882+0000 7f12333dc640 1 -- 172.21.15.157:0/3772894600 shutdown_connections 2024-04-24T23:35:05.883 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.882+0000 7f12333dc640 1 --2- 172.21.15.157:0/3772894600 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f121006be30 0x7f121006e2f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:05.884 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.882+0000 7f12333dc640 1 --2- 172.21.15.157:0/3772894600 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f122c155890 0x7f122c1eaf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:05.884 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.882+0000 7f12333dc640 1 --2- 172.21.15.157:0/3772894600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f122c152b40 0x7f122c147720 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:05.884 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.882+0000 7f12333dc640 1 --2- 172.21.15.157:0/3772894600 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f122c0be360 0x7f122c1eef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:05.884 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.882+0000 7f12333dc640 1 -- 172.21.15.157:0/3772894600 >> 172.21.15.157:0/3772894600 conn(0x7f122c0672c0 msgr2=0x7f122c14aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:05.884 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.882+0000 7f12333dc640 1 -- 172.21.15.157:0/3772894600 shutdown_connections 2024-04-24T23:35:05.884 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.882+0000 7f12333dc640 1 -- 172.21.15.157:0/3772894600 wait complete. 2024-04-24T23:35:05.894 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:34:47.796655+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:34:52.177208+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:34:52.667022+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T23:34:56.395517+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T23:35:05.895 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:35:05.995 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.994+0000 7fcce979e640 1 Processor -- start 2024-04-24T23:35:05.995 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.994+0000 7fcce979e640 1 -- start start 2024-04-24T23:35:05.999 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.995+0000 7fcce979e640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcce4150720 0x7fcce4150b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:05.999 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.995+0000 7fcce979e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcce406e240 con 0x7fcce4072c50 2024-04-24T23:35:05.999 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.995+0000 7fcce979e640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcce4156a50 con 0x7fcce4153330 2024-04-24T23:35:05.999 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.995+0000 7fcce979e640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcce4156cf0 con 0x7fcce4150720 2024-04-24T23:35:05.999 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.995+0000 7fcce2ffd640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fcce4072c50 0x7fcce406d830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:41136/0 (socket says 172.21.15.157:41136) 2024-04-24T23:35:05.999 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.995+0000 7fcce2ffd640 1 -- 172.21.15.157:0/80049982 learned_addr learned my addr 172.21.15.157:0/80049982 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-24T23:35:05.999 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.995+0000 7fcce1ffb640 1 -- 172.21.15.157:0/80049982 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3805535317 0 0) 0x7fcce406e240 con 0x7fcce4072c50 2024-04-24T23:35:05.999 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.995+0000 7fcce1ffb640 1 -- 172.21.15.157:0/80049982 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcccc003520 con 0x7fcce4072c50 2024-04-24T23:35:05.999 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.995+0000 7fcce27fc640 1 --2- 172.21.15.157:0/80049982 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcce4150720 0x7fcce4150b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:05.999 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.995+0000 7fcce1ffb640 1 -- 172.21.15.157:0/80049982 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1634073654 0 0) 0x7fcccc003520 con 0x7fcce4072c50 2024-04-24T23:35:05.999 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.995+0000 7fcce1ffb640 1 -- 172.21.15.157:0/80049982 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcce4150720 msgr2=0x7fcce4150b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:06.000 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.995+0000 7fcce1ffb640 1 --2- 172.21.15.157:0/80049982 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcce4150720 0x7fcce4150b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:06.000 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.995+0000 7fcce1ffb640 1 -- 172.21.15.157:0/80049982 >> v1:172.21.15.169:6789/0 conn(0x7fcce4153330 legacy=0x7fcce41537c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:06.000 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.995+0000 7fcce1ffb640 1 -- 172.21.15.157:0/80049982 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcce4156fd0 con 0x7fcce4072c50 2024-04-24T23:35:06.000 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.995+0000 7fcce1ffb640 1 -- 172.21.15.157:0/80049982 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1480525340 0 0) 0x7fccd4001670 con 0x7fcce4072c50 2024-04-24T23:35:06.000 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.996+0000 7fcce1ffb640 1 -- 172.21.15.157:0/80049982 >> v1:172.21.15.157:6789/0 conn(0x7fcce4072c50 legacy=0x7fcce406d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:06.000 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.996+0000 7fcce1ffb640 1 --2- 172.21.15.157:0/80049982 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcce4150720 0x7fcccc0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:06.000 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.996+0000 7fcce1ffb640 1 --2- 172.21.15.157:0/80049982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcccc003f60 0x7fcccc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:06.000 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.996+0000 7fcce1ffb640 1 --2- 172.21.15.157:0/80049982 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcccc004d40 0x7fcccc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:06.000 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.996+0000 7fcce1ffb640 1 -- 172.21.15.157:0/80049982 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcce4156cf0 con 0x7fcccc003f60 2024-04-24T23:35:06.000 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.996+0000 7fcce1ffb640 1 -- 172.21.15.157:0/80049982 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcccc00bc00 con 0x7fcccc004d40 2024-04-24T23:35:06.000 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.996+0000 7fcce1ffb640 1 -- 172.21.15.157:0/80049982 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcccc00be00 con 0x7fcce4150720 2024-04-24T23:35:06.000 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.996+0000 7fcce27fc640 1 --2- 172.21.15.157:0/80049982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcccc003f60 0x7fcccc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:06.000 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.996+0000 7fcce2ffd640 1 --2- 172.21.15.157:0/80049982 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcce4150720 0x7fcccc0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:06.000 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.996+0000 7fcce37fe640 1 --2- 172.21.15.157:0/80049982 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcccc004d40 0x7fcccc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:06.001 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.996+0000 7fcce27fc640 1 -- 172.21.15.157:0/80049982 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcce4150720 msgr2=0x7fcccc0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:06.001 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.996+0000 7fcce27fc640 1 --2- 172.21.15.157:0/80049982 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcce4150720 0x7fcccc0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:06.001 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.996+0000 7fcce27fc640 1 -- 172.21.15.157:0/80049982 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcccc004d40 msgr2=0x7fcccc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:06.001 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.996+0000 7fcce27fc640 1 --2- 172.21.15.157:0/80049982 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcccc004d40 0x7fcccc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:06.001 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.996+0000 7fcce27fc640 1 -- 172.21.15.157:0/80049982 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcccc003520 con 0x7fcccc003f60 2024-04-24T23:35:06.001 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.996+0000 7fcce37fe640 1 --2- 172.21.15.157:0/80049982 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcccc004d40 0x7fcccc0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:35:06.001 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.997+0000 7fcce27fc640 1 --2- 172.21.15.157:0/80049982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcccc003f60 0x7fcccc004800 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fccd8009ac0 tx=0x7fcccc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ffcc73f19087fea server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:06.001 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.997+0000 7fcce1ffb640 1 -- 172.21.15.157:0/80049982 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd8036070 con 0x7fcccc003f60 2024-04-24T23:35:06.001 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.997+0000 7fcce1ffb640 1 -- 172.21.15.157:0/80049982 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccd8031400 con 0x7fcccc003f60 2024-04-24T23:35:06.001 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.997+0000 7fcce1ffb640 1 -- 172.21.15.157:0/80049982 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd8009690 con 0x7fcccc003f60 2024-04-24T23:35:06.001 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.997+0000 7fcce979e640 1 -- 172.21.15.157:0/80049982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcccc003f60 msgr2=0x7fcccc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:06.001 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.997+0000 7fcce979e640 1 --2- 172.21.15.157:0/80049982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcccc003f60 0x7fcccc004800 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fccd8009ac0 tx=0x7fcccc005830 comp rx=0 tx=0).stop 2024-04-24T23:35:06.001 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.997+0000 7fcce37fe640 1 -- 172.21.15.157:0/80049982 reap_dead start 2024-04-24T23:35:06.002 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.997+0000 7fcce979e640 1 -- 172.21.15.157:0/80049982 shutdown_connections 2024-04-24T23:35:06.002 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.997+0000 7fcce979e640 1 -- 172.21.15.157:0/80049982 >> 172.21.15.157:0/80049982 conn(0x7fcce40c68e0 msgr2=0x7fcce40c6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:06.002 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.997+0000 7fcce979e640 1 -- 172.21.15.157:0/80049982 shutdown_connections 2024-04-24T23:35:06.002 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.997+0000 7fcce979e640 1 -- 172.21.15.157:0/80049982 wait complete. 2024-04-24T23:35:06.005 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.999+0000 7fcce979e640 1 Processor -- start 2024-04-24T23:35:06.005 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.999+0000 7fcce979e640 1 -- start start 2024-04-24T23:35:06.005 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.999+0000 7fcce979e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcccc003f60 0x7fcce414e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:06.005 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.999+0000 7fcce979e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcce40711c0 0x7fcce41539b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:06.005 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.999+0000 7fcce979e640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcce4070d00 0x7fcce41eaf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:06.005 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.999+0000 7fcce979e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcce41577d0 con 0x7fcce40711c0 2024-04-24T23:35:06.005 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.999+0000 7fcce979e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcce406df40 con 0x7fcccc003f60 2024-04-24T23:35:06.005 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.999+0000 7fcce979e640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcce40c2fa0 con 0x7fcce4070d00 2024-04-24T23:35:06.005 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.999+0000 7fcce27fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcce40711c0 0x7fcce41539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:06.005 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.999+0000 7fcce2ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcccc003f60 0x7fcce414e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:06.005 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.999+0000 7fcce27fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcce40711c0 0x7fcce41539b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39712/0 (socket says 172.21.15.157:39712) 2024-04-24T23:35:06.006 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.999+0000 7fcce27fc640 1 -- 172.21.15.157:0/4095647909 learned_addr learned my addr 172.21.15.157:0/4095647909 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:06.006 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:05.999+0000 7fcce37fe640 1 --2- 172.21.15.157:0/4095647909 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcce4070d00 0x7fcce41eaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:06.006 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.000+0000 7fcce27fc640 1 -- 172.21.15.157:0/4095647909 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcce4070d00 msgr2=0x7fcce41eaf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:06.006 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.000+0000 7fcce27fc640 1 --2- 172.21.15.157:0/4095647909 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcce4070d00 0x7fcce41eaf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:06.006 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.000+0000 7fcce27fc640 1 -- 172.21.15.157:0/4095647909 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcccc003f60 msgr2=0x7fcce414e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:06.006 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.000+0000 7fcce27fc640 1 --2- 172.21.15.157:0/4095647909 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcccc003f60 0x7fcce414e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:06.006 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.000+0000 7fcce27fc640 1 -- 172.21.15.157:0/4095647909 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcce414c580 con 0x7fcce40711c0 2024-04-24T23:35:06.006 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.000+0000 7fcce27fc640 1 --2- 172.21.15.157:0/4095647909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcce40711c0 0x7fcce41539b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fccd8000c00 tx=0x7fccd8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:06.006 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.000+0000 7fcccb7fe640 1 -- 172.21.15.157:0/4095647909 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd8040020 con 0x7fcce40711c0 2024-04-24T23:35:06.006 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.000+0000 7fcccb7fe640 1 -- 172.21.15.157:0/4095647909 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccd8003c60 con 0x7fcce40711c0 2024-04-24T23:35:06.006 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.000+0000 7fcce979e640 1 -- 172.21.15.157:0/4095647909 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcce40c0f50 con 0x7fcce40711c0 2024-04-24T23:35:06.006 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.000+0000 7fcccb7fe640 1 -- 172.21.15.157:0/4095647909 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd8036040 con 0x7fcce40711c0 2024-04-24T23:35:06.006 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.000+0000 7fcce979e640 1 -- 172.21.15.157:0/4095647909 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcce4153ef0 con 0x7fcce40711c0 2024-04-24T23:35:06.007 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.001+0000 7fcce979e640 1 -- 172.21.15.157:0/4095647909 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcce4151eb0 con 0x7fcce40711c0 2024-04-24T23:35:06.007 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.001+0000 7fcccb7fe640 1 -- 172.21.15.157:0/4095647909 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fccd8043020 con 0x7fcce40711c0 2024-04-24T23:35:06.007 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.002+0000 7fcccb7fe640 1 --2- 172.21.15.157:0/4095647909 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fccd006be30 0x7fccd006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:06.007 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.002+0000 7fcccb7fe640 1 -- 172.21.15.157:0/4095647909 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fccd80ab120 con 0x7fcce40711c0 2024-04-24T23:35:06.007 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.002+0000 7fcce2ffd640 1 --2- 172.21.15.157:0/4095647909 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fccd006be30 0x7fccd006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:06.007 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.003+0000 7fcce2ffd640 1 --2- 172.21.15.157:0/4095647909 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fccd006be30 0x7fccd006e2f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcce414ea00 tx=0x7fccd4001c10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:06.007 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.006+0000 7fcccb7fe640 1 -- 172.21.15.157:0/4095647909 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcce4151eb0 con 0x7fcce40711c0 2024-04-24T23:35:06.179 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.178+0000 7fcce979e640 1 -- 172.21.15.157:0/4095647909 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcce4069470 con 0x7fcce40711c0 2024-04-24T23:35:06.179 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.178+0000 7fcccb7fe640 1 -- 172.21.15.157:0/4095647909 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7fcce4069470 con 0x7fcce40711c0 2024-04-24T23:35:06.179 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2024-04-24T23:35:06.181 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.180+0000 7fcce979e640 1 -- 172.21.15.157:0/4095647909 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fccd006be30 msgr2=0x7fccd006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:06.181 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.180+0000 7fcce979e640 1 --2- 172.21.15.157:0/4095647909 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fccd006be30 0x7fccd006e2f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcce414ea00 tx=0x7fccd4001c10 comp rx=0 tx=0).stop 2024-04-24T23:35:06.182 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.180+0000 7fcce979e640 1 -- 172.21.15.157:0/4095647909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcce40711c0 msgr2=0x7fcce41539b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:06.182 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.180+0000 7fcce979e640 1 --2- 172.21.15.157:0/4095647909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcce40711c0 0x7fcce41539b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fccd8000c00 tx=0x7fccd8004730 comp rx=0 tx=0).stop 2024-04-24T23:35:06.182 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.180+0000 7fcce979e640 1 -- 172.21.15.157:0/4095647909 shutdown_connections 2024-04-24T23:35:06.182 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.180+0000 7fcce979e640 1 --2- 172.21.15.157:0/4095647909 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fccd006be30 0x7fccd006e2f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:06.182 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.180+0000 7fcce979e640 1 --2- 172.21.15.157:0/4095647909 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcce4070d00 0x7fcce41eaf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:06.182 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.180+0000 7fcce979e640 1 --2- 172.21.15.157:0/4095647909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcce40711c0 0x7fcce41539b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:06.182 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.180+0000 7fcce979e640 1 --2- 172.21.15.157:0/4095647909 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcccc003f60 0x7fcce414e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:06.182 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.180+0000 7fcce979e640 1 -- 172.21.15.157:0/4095647909 >> 172.21.15.157:0/4095647909 conn(0x7fcce40c68e0 msgr2=0x7fcce4152a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:06.182 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.180+0000 7fcce979e640 1 -- 172.21.15.157:0/4095647909 shutdown_connections 2024-04-24T23:35:06.182 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.180+0000 7fcce979e640 1 -- 172.21.15.157:0/4095647909 wait complete. 2024-04-24T23:35:06.192 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:35:06.292 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.291+0000 7f0897570640 1 Processor -- start 2024-04-24T23:35:06.292 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.292+0000 7f0897570640 1 -- start start 2024-04-24T23:35:06.294 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.292+0000 7f0897570640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f08900b03e0 0x7f08901567e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:06.294 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.292+0000 7f0897570640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08901592f0 0x7f0890155c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:06.294 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.292+0000 7f0897570640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08900cb330 con 0x7f08900c55d0 2024-04-24T23:35:06.294 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.292+0000 7f0897570640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f08900b07c0 con 0x7f08900b03e0 2024-04-24T23:35:06.294 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.292+0000 7f0897570640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f08900cb610 con 0x7f08901592f0 2024-04-24T23:35:06.294 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.292+0000 7f089530c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f08900b03e0 0x7f08901567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:06.294 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.292+0000 7f089530c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f08900b03e0 0x7f08901567e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39728/0 (socket says 172.21.15.157:39728) 2024-04-24T23:35:06.294 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.292+0000 7f0895b0d640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f08900c55d0 0x7f08900c75f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:41140/0 (socket says 172.21.15.157:41140) 2024-04-24T23:35:06.294 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.292+0000 7f089530c640 1 -- 172.21.15.157:0/19458796 learned_addr learned my addr 172.21.15.157:0/19458796 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:06.294 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.293+0000 7f0894b0b640 1 --2- 172.21.15.157:0/19458796 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08901592f0 0x7f0890155c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:06.294 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.293+0000 7f089530c640 1 -- 172.21.15.157:0/19458796 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08901592f0 msgr2=0x7f0890155c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:06.294 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.293+0000 7f089530c640 1 --2- 172.21.15.157:0/19458796 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08901592f0 0x7f0890155c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:06.295 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.293+0000 7f089530c640 1 -- 172.21.15.157:0/19458796 >> v1:172.21.15.157:6789/0 conn(0x7f08900c55d0 legacy=0x7f08900c75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:06.295 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.293+0000 7f0887fff640 1 -- 172.21.15.157:0/19458796 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3562205952 0 0) 0x7f08900cb330 con 0x7f08900c55d0 2024-04-24T23:35:06.295 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.293+0000 7f089530c640 1 -- 172.21.15.157:0/19458796 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08900cb8f0 con 0x7f08900b03e0 2024-04-24T23:35:06.295 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.293+0000 7f0894b0b640 1 --2- 172.21.15.157:0/19458796 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08901592f0 0x7f0890155c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:35:06.296 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.293+0000 7f089530c640 1 --2- 172.21.15.157:0/19458796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08900b03e0 0x7f08901567e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f088c009e50 tx=0x7f088c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=477122ec0b942c19 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:06.296 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.294+0000 7f0887fff640 1 -- 172.21.15.157:0/19458796 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f088c036070 con 0x7f08900b03e0 2024-04-24T23:35:06.296 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.294+0000 7f0887fff640 1 -- 172.21.15.157:0/19458796 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f088c031600 con 0x7f08900b03e0 2024-04-24T23:35:06.296 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.294+0000 7f0887fff640 1 -- 172.21.15.157:0/19458796 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f088c0319d0 con 0x7f08900b03e0 2024-04-24T23:35:06.296 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.294+0000 7f0897570640 1 -- 172.21.15.157:0/19458796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08900b03e0 msgr2=0x7f08901567e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:06.296 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.294+0000 7f0897570640 1 --2- 172.21.15.157:0/19458796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08900b03e0 0x7f08901567e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f088c009e50 tx=0x7f088c02a680 comp rx=0 tx=0).stop 2024-04-24T23:35:06.297 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.294+0000 7f0897570640 1 -- 172.21.15.157:0/19458796 shutdown_connections 2024-04-24T23:35:06.297 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.294+0000 7f0897570640 1 --2- 172.21.15.157:0/19458796 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08901592f0 0x7f0890155c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:06.297 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.294+0000 7f0897570640 1 --2- 172.21.15.157:0/19458796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08900b03e0 0x7f08901567e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:06.297 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.294+0000 7f0897570640 1 -- 172.21.15.157:0/19458796 >> 172.21.15.157:0/19458796 conn(0x7f0890067080 msgr2=0x7f0890067490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:06.297 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.294+0000 7f0897570640 1 -- 172.21.15.157:0/19458796 shutdown_connections 2024-04-24T23:35:06.297 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.294+0000 7f0897570640 1 -- 172.21.15.157:0/19458796 wait complete. 2024-04-24T23:35:06.297 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.295+0000 7f0897570640 1 Processor -- start 2024-04-24T23:35:06.297 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.295+0000 7f0897570640 1 -- start start 2024-04-24T23:35:06.299 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.296+0000 7f0897570640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08900b03e0 0x7f08900af400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:06.299 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.296+0000 7f0897570640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08900c55d0 0x7f089014dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:06.300 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.296+0000 7f0897570640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08901592f0 0x7f089014e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:06.300 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.296+0000 7f0897570640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f08900cafc0 con 0x7f08900c55d0 2024-04-24T23:35:06.300 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.296+0000 7f0897570640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f08901556e0 con 0x7f08900b03e0 2024-04-24T23:35:06.300 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.296+0000 7f0897570640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0890159750 con 0x7f08901592f0 2024-04-24T23:35:06.300 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.296+0000 7f0894b0b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08900c55d0 0x7f089014dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:06.300 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.296+0000 7f0894b0b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08900c55d0 0x7f089014dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39742/0 (socket says 172.21.15.157:39742) 2024-04-24T23:35:06.300 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.296+0000 7f0894b0b640 1 -- 172.21.15.157:0/2288148341 learned_addr learned my addr 172.21.15.157:0/2288148341 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:06.300 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.296+0000 7f0895b0d640 1 --2- 172.21.15.157:0/2288148341 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08901592f0 0x7f089014e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:06.300 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.296+0000 7f089530c640 1 --2- 172.21.15.157:0/2288148341 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08900b03e0 0x7f08900af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:06.300 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.296+0000 7f0894b0b640 1 -- 172.21.15.157:0/2288148341 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08901592f0 msgr2=0x7f089014e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:06.300 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.296+0000 7f0894b0b640 1 --2- 172.21.15.157:0/2288148341 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08901592f0 0x7f089014e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:06.300 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.296+0000 7f0894b0b640 1 -- 172.21.15.157:0/2288148341 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08900b03e0 msgr2=0x7f08900af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:06.300 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.296+0000 7f0894b0b640 1 --2- 172.21.15.157:0/2288148341 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08900b03e0 0x7f08900af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:06.301 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.296+0000 7f0894b0b640 1 -- 172.21.15.157:0/2288148341 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08900ae780 con 0x7f08900c55d0 2024-04-24T23:35:06.301 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.296+0000 7f0894b0b640 1 --2- 172.21.15.157:0/2288148341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08900c55d0 0x7f089014dee0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f088000c820 tx=0x7f088000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:06.301 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.297+0000 7f0885ffb640 1 -- 172.21.15.157:0/2288148341 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0880004220 con 0x7f08900c55d0 2024-04-24T23:35:06.301 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.297+0000 7f0885ffb640 1 -- 172.21.15.157:0/2288148341 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08800043c0 con 0x7f08900c55d0 2024-04-24T23:35:06.301 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.297+0000 7f0897570640 1 -- 172.21.15.157:0/2288148341 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08900ab8a0 con 0x7f08900c55d0 2024-04-24T23:35:06.301 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.297+0000 7f0885ffb640 1 -- 172.21.15.157:0/2288148341 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0880005bb0 con 0x7f08900c55d0 2024-04-24T23:35:06.301 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.297+0000 7f0897570640 1 -- 172.21.15.157:0/2288148341 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08900abe60 con 0x7f08900c55d0 2024-04-24T23:35:06.301 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.298+0000 7f0897570640 1 -- 172.21.15.157:0/2288148341 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08900681c0 con 0x7f08900c55d0 2024-04-24T23:35:06.301 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.298+0000 7f0885ffb640 1 -- 172.21.15.157:0/2288148341 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0880007830 con 0x7f08900c55d0 2024-04-24T23:35:06.301 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.298+0000 7f0885ffb640 1 --2- 172.21.15.157:0/2288148341 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f087406be30 0x7f087406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:06.301 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.298+0000 7f0885ffb640 1 -- 172.21.15.157:0/2288148341 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f088008b3d0 con 0x7f08900c55d0 2024-04-24T23:35:06.301 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.299+0000 7f089530c640 1 --2- 172.21.15.157:0/2288148341 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f087406be30 0x7f087406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:06.301 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.299+0000 7f089530c640 1 --2- 172.21.15.157:0/2288148341 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f087406be30 0x7f087406e2f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f088c033000 tx=0x7f088c009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:06.303 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.303+0000 7f0885ffb640 1 -- 172.21.15.157:0/2288148341 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f088001e020 con 0x7f08900c55d0 2024-04-24T23:35:06.482 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.481+0000 7f0897570640 1 -- 172.21.15.157:0/2288148341 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f08901569a0 con 0x7f08900c55d0 2024-04-24T23:35:06.482 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.481+0000 7f0885ffb640 1 -- 172.21.15.157:0/2288148341 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f0880015030 con 0x7f08900c55d0 2024-04-24T23:35:06.482 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2024-04-24T23:35:06.484 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.483+0000 7f0897570640 1 -- 172.21.15.157:0/2288148341 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f087406be30 msgr2=0x7f087406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:06.484 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.483+0000 7f0897570640 1 --2- 172.21.15.157:0/2288148341 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f087406be30 0x7f087406e2f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f088c033000 tx=0x7f088c009000 comp rx=0 tx=0).stop 2024-04-24T23:35:06.484 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.483+0000 7f0897570640 1 -- 172.21.15.157:0/2288148341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08900c55d0 msgr2=0x7f089014dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:06.485 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.483+0000 7f0897570640 1 --2- 172.21.15.157:0/2288148341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08900c55d0 0x7f089014dee0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f088000c820 tx=0x7f088000cce0 comp rx=0 tx=0).stop 2024-04-24T23:35:06.485 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.483+0000 7f0897570640 1 -- 172.21.15.157:0/2288148341 shutdown_connections 2024-04-24T23:35:06.485 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.483+0000 7f0897570640 1 --2- 172.21.15.157:0/2288148341 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f087406be30 0x7f087406e2f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:06.485 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.483+0000 7f0897570640 1 --2- 172.21.15.157:0/2288148341 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08901592f0 0x7f089014e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:06.485 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.483+0000 7f0897570640 1 --2- 172.21.15.157:0/2288148341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08900c55d0 0x7f089014dee0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:06.485 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.483+0000 7f0897570640 1 --2- 172.21.15.157:0/2288148341 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08900b03e0 0x7f08900af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:06.485 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.483+0000 7f0897570640 1 -- 172.21.15.157:0/2288148341 >> 172.21.15.157:0/2288148341 conn(0x7f0890067080 msgr2=0x7f0890158870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:06.485 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.483+0000 7f0897570640 1 -- 172.21.15.157:0/2288148341 shutdown_connections 2024-04-24T23:35:06.485 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.483+0000 7f0897570640 1 -- 172.21.15.157:0/2288148341 wait complete. 2024-04-24T23:35:06.495 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:35:06.594 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.594+0000 7fc5a9f34640 1 Processor -- start 2024-04-24T23:35:06.594 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.594+0000 7fc5a9f34640 1 -- start start 2024-04-24T23:35:06.598 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.594+0000 7fc5a9f34640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5a4156390 0x7fc5a4152cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:06.598 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.594+0000 7fc5a9f34640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5a40736b0 con 0x7fc5a40c3370 2024-04-24T23:35:06.598 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.594+0000 7fc5a9f34640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5a40a60e0 con 0x7fc5a40a5c50 2024-04-24T23:35:06.598 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.594+0000 7fc5a9f34640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc5a4155f00 con 0x7fc5a4156390 2024-04-24T23:35:06.598 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.595+0000 7fc5a37fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fc5a40c3370 0x7fc5a40c12e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:41146/0 (socket says 172.21.15.157:41146) 2024-04-24T23:35:06.598 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.595+0000 7fc5a37fe640 1 -- 172.21.15.157:0/1151727137 learned_addr learned my addr 172.21.15.157:0/1151727137 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-24T23:35:06.598 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.595+0000 7fc5a27fc640 1 -- 172.21.15.157:0/1151727137 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2554730178 0 0) 0x7fc5a40736b0 con 0x7fc5a40c3370 2024-04-24T23:35:06.598 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.595+0000 7fc5a3fff640 1 --2- 172.21.15.157:0/1151727137 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5a4156390 0x7fc5a4152cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:06.598 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.595+0000 7fc5a27fc640 1 -- 172.21.15.157:0/1151727137 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc590003520 con 0x7fc5a40c3370 2024-04-24T23:35:06.599 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.595+0000 7fc5a27fc640 1 -- 172.21.15.157:0/1151727137 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1603888985 0 0) 0x7fc590003520 con 0x7fc5a40c3370 2024-04-24T23:35:06.599 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.595+0000 7fc5a27fc640 1 -- 172.21.15.157:0/1151727137 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5a4156390 msgr2=0x7fc5a4152cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:06.599 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.595+0000 7fc5a27fc640 1 --2- 172.21.15.157:0/1151727137 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5a4156390 0x7fc5a4152cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:06.599 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.595+0000 7fc5a27fc640 1 -- 172.21.15.157:0/1151727137 >> v1:172.21.15.169:6789/0 conn(0x7fc5a40a5c50 legacy=0x7fc5a415ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:06.599 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.595+0000 7fc5a27fc640 1 -- 172.21.15.157:0/1151727137 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5a40a63c0 con 0x7fc5a40c3370 2024-04-24T23:35:06.599 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.595+0000 7fc5a27fc640 1 -- 172.21.15.157:0/1151727137 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1480525340 0 0) 0x7fc59c0019f0 con 0x7fc5a40c3370 2024-04-24T23:35:06.599 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.595+0000 7fc5a27fc640 1 -- 172.21.15.157:0/1151727137 >> v1:172.21.15.157:6789/0 conn(0x7fc5a40c3370 legacy=0x7fc5a40c12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:06.599 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.595+0000 7fc5a27fc640 1 --2- 172.21.15.157:0/1151727137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5900039e0 0x7fc590003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:06.599 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.595+0000 7fc5a27fc640 1 --2- 172.21.15.157:0/1151727137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc590004800 0x7fc590004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:06.599 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.595+0000 7fc5a27fc640 1 --2- 172.21.15.157:0/1151727137 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5a4156390 0x7fc5900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:06.599 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.595+0000 7fc5a27fc640 1 -- 172.21.15.157:0/1151727137 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a4155f00 con 0x7fc590004800 2024-04-24T23:35:06.599 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.595+0000 7fc5a27fc640 1 -- 172.21.15.157:0/1151727137 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc59000ba70 con 0x7fc5900039e0 2024-04-24T23:35:06.599 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.595+0000 7fc5a27fc640 1 -- 172.21.15.157:0/1151727137 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc59000bc70 con 0x7fc5a4156390 2024-04-24T23:35:06.599 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.595+0000 7fc5a2ffd640 1 --2- 172.21.15.157:0/1151727137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc590004800 0x7fc590004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:06.600 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.595+0000 7fc5a37fe640 1 --2- 172.21.15.157:0/1151727137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5900039e0 0x7fc590003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:06.600 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.595+0000 7fc5a3fff640 1 --2- 172.21.15.157:0/1151727137 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5a4156390 0x7fc5900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:06.600 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.596+0000 7fc5a2ffd640 1 -- 172.21.15.157:0/1151727137 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5a4156390 msgr2=0x7fc5900051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:06.600 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.596+0000 7fc5a2ffd640 1 --2- 172.21.15.157:0/1151727137 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5a4156390 0x7fc5900051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:06.600 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.596+0000 7fc5a2ffd640 1 -- 172.21.15.157:0/1151727137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5900039e0 msgr2=0x7fc590003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:06.600 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.596+0000 7fc5a2ffd640 1 --2- 172.21.15.157:0/1151727137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5900039e0 0x7fc590003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:06.600 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.596+0000 7fc5a2ffd640 1 -- 172.21.15.157:0/1151727137 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc590003520 con 0x7fc590004800 2024-04-24T23:35:06.600 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.596+0000 7fc5a2ffd640 1 --2- 172.21.15.157:0/1151727137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc590004800 0x7fc590004c80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc58c001720 tx=0x7fc590005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=640f012899efbe16 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:06.600 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.596+0000 7fc5a27fc640 1 -- 172.21.15.157:0/1151727137 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc58c001370 con 0x7fc590004800 2024-04-24T23:35:06.600 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.596+0000 7fc5a27fc640 1 -- 172.21.15.157:0/1151727137 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc58c0052d0 con 0x7fc590004800 2024-04-24T23:35:06.600 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.596+0000 7fc5a27fc640 1 -- 172.21.15.157:0/1151727137 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc58c02ab90 con 0x7fc590004800 2024-04-24T23:35:06.600 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.597+0000 7fc5a9f34640 1 -- 172.21.15.157:0/1151727137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc590004800 msgr2=0x7fc590004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:06.600 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.597+0000 7fc5a9f34640 1 --2- 172.21.15.157:0/1151727137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc590004800 0x7fc590004c80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc58c001720 tx=0x7fc590005830 comp rx=0 tx=0).stop 2024-04-24T23:35:06.600 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.597+0000 7fc5a3fff640 1 -- 172.21.15.157:0/1151727137 reap_dead start 2024-04-24T23:35:06.601 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.597+0000 7fc5a9f34640 1 -- 172.21.15.157:0/1151727137 shutdown_connections 2024-04-24T23:35:06.601 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.597+0000 7fc5a9f34640 1 -- 172.21.15.157:0/1151727137 >> 172.21.15.157:0/1151727137 conn(0x7fc5a40c68e0 msgr2=0x7fc5a40c6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:06.601 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.597+0000 7fc5a9f34640 1 -- 172.21.15.157:0/1151727137 shutdown_connections 2024-04-24T23:35:06.601 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.597+0000 7fc5a9f34640 1 -- 172.21.15.157:0/1151727137 wait complete. 2024-04-24T23:35:06.603 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.598+0000 7fc5a9f34640 1 Processor -- start 2024-04-24T23:35:06.603 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.598+0000 7fc5a9f34640 1 -- start start 2024-04-24T23:35:06.604 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.599+0000 7fc5a9f34640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc590004800 0x7fc5a40a96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:06.604 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.599+0000 7fc5a9f34640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5a41ebc40 0x7fc5a40ab9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:06.604 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.599+0000 7fc5a9f34640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5a4151500 0x7fc5a41519b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:06.604 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.599+0000 7fc5a9f34640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a40c19f0 con 0x7fc590004800 2024-04-24T23:35:06.604 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.599+0000 7fc5a9f34640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a4155b90 con 0x7fc5a41ebc40 2024-04-24T23:35:06.604 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.599+0000 7fc5a9f34640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc5a40c5bc0 con 0x7fc5a4151500 2024-04-24T23:35:06.604 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.599+0000 7fc5a37fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc590004800 0x7fc5a40a96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:06.604 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.599+0000 7fc5a37fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc590004800 0x7fc5a40a96e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39760/0 (socket says 172.21.15.157:39760) 2024-04-24T23:35:06.604 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.599+0000 7fc5a37fe640 1 -- 172.21.15.157:0/3983955121 learned_addr learned my addr 172.21.15.157:0/3983955121 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:06.604 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.599+0000 7fc5a2ffd640 1 --2- 172.21.15.157:0/3983955121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5a41ebc40 0x7fc5a40ab9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:06.604 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.599+0000 7fc5a3fff640 1 --2- 172.21.15.157:0/3983955121 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5a4151500 0x7fc5a41519b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:06.604 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.599+0000 7fc5a37fe640 1 -- 172.21.15.157:0/3983955121 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5a4151500 msgr2=0x7fc5a41519b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:06.604 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.599+0000 7fc5a37fe640 1 --2- 172.21.15.157:0/3983955121 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5a4151500 0x7fc5a41519b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:06.605 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.599+0000 7fc5a37fe640 1 -- 172.21.15.157:0/3983955121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5a41ebc40 msgr2=0x7fc5a40ab9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:06.605 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.599+0000 7fc5a37fe640 1 --2- 172.21.15.157:0/3983955121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5a41ebc40 0x7fc5a40ab9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:06.605 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.599+0000 7fc5a37fe640 1 -- 172.21.15.157:0/3983955121 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5a41dbd40 con 0x7fc590004800 2024-04-24T23:35:06.605 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.599+0000 7fc5a37fe640 1 --2- 172.21.15.157:0/3983955121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc590004800 0x7fc5a40a96e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc59c0077a0 tx=0x7fc59c00a360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:06.605 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.600+0000 7fc59bfff640 1 -- 172.21.15.157:0/3983955121 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59c017020 con 0x7fc590004800 2024-04-24T23:35:06.605 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.600+0000 7fc59bfff640 1 -- 172.21.15.157:0/3983955121 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc59c00ace0 con 0x7fc590004800 2024-04-24T23:35:06.605 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.600+0000 7fc59bfff640 1 -- 172.21.15.157:0/3983955121 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59c013630 con 0x7fc590004800 2024-04-24T23:35:06.605 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.600+0000 7fc5a9f34640 1 -- 172.21.15.157:0/3983955121 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5a41dc020 con 0x7fc590004800 2024-04-24T23:35:06.605 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.600+0000 7fc5a9f34640 1 -- 172.21.15.157:0/3983955121 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5a41ee730 con 0x7fc590004800 2024-04-24T23:35:06.605 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.601+0000 7fc5a9f34640 1 -- 172.21.15.157:0/3983955121 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5a40681c0 con 0x7fc590004800 2024-04-24T23:35:06.605 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.601+0000 7fc59bfff640 1 -- 172.21.15.157:0/3983955121 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc59c01a020 con 0x7fc590004800 2024-04-24T23:35:06.605 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.601+0000 7fc59bfff640 1 --2- 172.21.15.157:0/3983955121 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fc58406be60 0x7fc58406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:06.605 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.601+0000 7fc59bfff640 1 -- 172.21.15.157:0/3983955121 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc59c08bb60 con 0x7fc590004800 2024-04-24T23:35:06.605 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.602+0000 7fc5a2ffd640 1 --2- 172.21.15.157:0/3983955121 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fc58406be60 0x7fc58406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:06.606 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.602+0000 7fc5a2ffd640 1 --2- 172.21.15.157:0/3983955121 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fc58406be60 0x7fc58406e320 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc58c032000 tx=0x7fc58c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:06.606 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.606+0000 7fc59bfff640 1 -- 172.21.15.157:0/3983955121 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc59c021020 con 0x7fc590004800 2024-04-24T23:35:06.782 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.782+0000 7fc5a9f34640 1 -- 172.21.15.157:0/3983955121 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc5a40c4240 con 0x7fc590004800 2024-04-24T23:35:06.783 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.782+0000 7fc59bfff640 1 -- 172.21.15.157:0/3983955121 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7fc59c05a260 con 0x7fc590004800 2024-04-24T23:35:06.783 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 64 2024-04-24T23:35:06.785 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.784+0000 7fc5a9f34640 1 -- 172.21.15.157:0/3983955121 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fc58406be60 msgr2=0x7fc58406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:06.785 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.784+0000 7fc5a9f34640 1 --2- 172.21.15.157:0/3983955121 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fc58406be60 0x7fc58406e320 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc58c032000 tx=0x7fc58c02e000 comp rx=0 tx=0).stop 2024-04-24T23:35:06.785 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.784+0000 7fc5a9f34640 1 -- 172.21.15.157:0/3983955121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc590004800 msgr2=0x7fc5a40a96e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:06.785 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.784+0000 7fc5a9f34640 1 --2- 172.21.15.157:0/3983955121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc590004800 0x7fc5a40a96e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc59c0077a0 tx=0x7fc59c00a360 comp rx=0 tx=0).stop 2024-04-24T23:35:06.786 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.784+0000 7fc5a9f34640 1 -- 172.21.15.157:0/3983955121 shutdown_connections 2024-04-24T23:35:06.786 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.784+0000 7fc5a9f34640 1 --2- 172.21.15.157:0/3983955121 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fc58406be60 0x7fc58406e320 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:06.786 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.784+0000 7fc5a9f34640 1 --2- 172.21.15.157:0/3983955121 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5a4151500 0x7fc5a41519b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:06.786 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.784+0000 7fc5a9f34640 1 --2- 172.21.15.157:0/3983955121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5a41ebc40 0x7fc5a40ab9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:06.786 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.784+0000 7fc5a9f34640 1 --2- 172.21.15.157:0/3983955121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc590004800 0x7fc5a40a96e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:06.786 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.784+0000 7fc5a9f34640 1 -- 172.21.15.157:0/3983955121 >> 172.21.15.157:0/3983955121 conn(0x7fc5a40c68e0 msgr2=0x7fc5a4154570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:06.786 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.785+0000 7fc5a9f34640 1 -- 172.21.15.157:0/3983955121 shutdown_connections 2024-04-24T23:35:06.786 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.785+0000 7fc5a9f34640 1 -- 172.21.15.157:0/3983955121 wait complete. 2024-04-24T23:35:06.796 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T23:35:06.896 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.896+0000 7f581f012640 1 Processor -- start 2024-04-24T23:35:06.896 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.896+0000 7f581f012640 1 -- start start 2024-04-24T23:35:06.898 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.896+0000 7f581f012640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f58180b03e0 0x7f58181567e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:06.898 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.896+0000 7f581f012640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58180cb330 con 0x7f58181592f0 2024-04-24T23:35:06.898 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.896+0000 7f581f012640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58180cb610 con 0x7f58180c55d0 2024-04-24T23:35:06.898 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.896+0000 7f581f012640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f58180b07c0 con 0x7f58180b03e0 2024-04-24T23:35:06.898 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.896+0000 7f5817fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f58181592f0 0x7f5818155c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:41160/0 (socket says 172.21.15.157:41160) 2024-04-24T23:35:06.898 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.896+0000 7f5817fff640 1 -- 172.21.15.157:0/3506263253 learned_addr learned my addr 172.21.15.157:0/3506263253 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-24T23:35:06.898 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.897+0000 7f581cdae640 1 --2- 172.21.15.157:0/3506263253 >> v2:172.21.15.169:3300/0 conn(0x7f58180b03e0 0x7f58181567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:06.898 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.897+0000 7f58177fe640 1 -- 172.21.15.157:0/3506263253 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1539871374 0 0) 0x7f58180cb330 con 0x7f58181592f0 2024-04-24T23:35:06.898 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.897+0000 7f58177fe640 1 -- 172.21.15.157:0/3506263253 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5808003520 con 0x7f58181592f0 2024-04-24T23:35:06.898 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.897+0000 7f58177fe640 1 -- 172.21.15.157:0/3506263253 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 849071527 0 0) 0x7f5808003520 con 0x7f58181592f0 2024-04-24T23:35:06.898 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.897+0000 7f581cdae640 1 -- 172.21.15.157:0/3506263253 >> v1:172.21.15.169:6789/0 conn(0x7f58180c55d0 legacy=0x7f581815fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:06.898 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.897+0000 7f581cdae640 1 -- 172.21.15.157:0/3506263253 >> v1:172.21.15.157:6789/0 conn(0x7f58181592f0 legacy=0x7f5818155c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:06.899 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.897+0000 7f581cdae640 1 -- 172.21.15.157:0/3506263253 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58180cb8f0 con 0x7f58180b03e0 2024-04-24T23:35:06.900 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.898+0000 7f581cdae640 1 --2- 172.21.15.157:0/3506263253 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58180b03e0 0x7f58181567e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f580c0097a0 tx=0x7f580c02a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83d289dd576a43a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:06.900 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.898+0000 7f58177fe640 1 -- 172.21.15.157:0/3506263253 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580c037070 con 0x7f58180b03e0 2024-04-24T23:35:06.900 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.898+0000 7f58177fe640 1 -- 172.21.15.157:0/3506263253 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f580c02abc0 con 0x7f58180b03e0 2024-04-24T23:35:06.900 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.898+0000 7f58177fe640 1 -- 172.21.15.157:0/3506263253 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58040017f0 con 0x7f58180b03e0 2024-04-24T23:35:06.900 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.898+0000 7f581f012640 1 -- 172.21.15.157:0/3506263253 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58180b03e0 msgr2=0x7f58181567e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:06.901 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.898+0000 7f581f012640 1 --2- 172.21.15.157:0/3506263253 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58180b03e0 0x7f58181567e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f580c0097a0 tx=0x7f580c02a130 comp rx=0 tx=0).stop 2024-04-24T23:35:06.901 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.898+0000 7f581f012640 1 -- 172.21.15.157:0/3506263253 shutdown_connections 2024-04-24T23:35:06.901 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.898+0000 7f581f012640 1 --2- 172.21.15.157:0/3506263253 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58180b03e0 0x7f58181567e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:06.901 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.898+0000 7f581f012640 1 -- 172.21.15.157:0/3506263253 >> 172.21.15.157:0/3506263253 conn(0x7f5818067080 msgr2=0x7f5818067490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:06.901 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.898+0000 7f581f012640 1 -- 172.21.15.157:0/3506263253 shutdown_connections 2024-04-24T23:35:06.901 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.898+0000 7f581f012640 1 -- 172.21.15.157:0/3506263253 wait complete. 2024-04-24T23:35:06.901 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.899+0000 7f581f012640 1 Processor -- start 2024-04-24T23:35:06.901 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.899+0000 7f581f012640 1 -- start start 2024-04-24T23:35:06.904 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.900+0000 7f581f012640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58180b03e0 0x7f581814f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:06.904 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.900+0000 7f581f012640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58180c55d0 0x7f58180c9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:06.904 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.900+0000 7f581f012640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58181592f0 0x7f581814ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:06.904 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.900+0000 7f581f012640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f58181556e0 con 0x7f58180c55d0 2024-04-24T23:35:06.904 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.900+0000 7f581f012640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5818156310 con 0x7f58181592f0 2024-04-24T23:35:06.904 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.900+0000 7f581f012640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5818159780 con 0x7f58180b03e0 2024-04-24T23:35:06.904 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.900+0000 7f5817fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58180c55d0 0x7f58180c9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:06.904 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.900+0000 7f5817fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58180c55d0 0x7f58180c9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39768/0 (socket says 172.21.15.157:39768) 2024-04-24T23:35:06.904 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.900+0000 7f5817fff640 1 -- 172.21.15.157:0/3162083045 learned_addr learned my addr 172.21.15.157:0/3162083045 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:06.904 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.901+0000 7f581cdae640 1 --2- 172.21.15.157:0/3162083045 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58180b03e0 0x7f581814f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:06.904 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.901+0000 7f581d5af640 1 --2- 172.21.15.157:0/3162083045 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58181592f0 0x7f581814ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:06.904 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.901+0000 7f5817fff640 1 -- 172.21.15.157:0/3162083045 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58180b03e0 msgr2=0x7f581814f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:06.905 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.901+0000 7f5817fff640 1 --2- 172.21.15.157:0/3162083045 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58180b03e0 0x7f581814f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:06.905 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.901+0000 7f5817fff640 1 -- 172.21.15.157:0/3162083045 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58181592f0 msgr2=0x7f581814ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:06.905 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.901+0000 7f5817fff640 1 --2- 172.21.15.157:0/3162083045 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58181592f0 0x7f581814ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:06.905 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.901+0000 7f5817fff640 1 -- 172.21.15.157:0/3162083045 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f581814fc50 con 0x7f58180c55d0 2024-04-24T23:35:06.905 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.901+0000 7f5817fff640 1 --2- 172.21.15.157:0/3162083045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58180c55d0 0x7f58180c9aa0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f580400c750 tx=0x7f580400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:06.905 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.901+0000 7f58157fa640 1 -- 172.21.15.157:0/3162083045 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5804017070 con 0x7f58180c55d0 2024-04-24T23:35:06.905 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.901+0000 7f58157fa640 1 -- 172.21.15.157:0/3162083045 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5804007270 con 0x7f58180c55d0 2024-04-24T23:35:06.905 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.901+0000 7f58157fa640 1 -- 172.21.15.157:0/3162083045 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58040057a0 con 0x7f58180c55d0 2024-04-24T23:35:06.905 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.901+0000 7f581f012640 1 -- 172.21.15.157:0/3162083045 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58181ee240 con 0x7f58180c55d0 2024-04-24T23:35:06.905 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.901+0000 7f581f012640 1 -- 172.21.15.157:0/3162083045 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58181eea80 con 0x7f58180c55d0 2024-04-24T23:35:06.905 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.902+0000 7f581f012640 1 -- 172.21.15.157:0/3162083045 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58180c1ad0 con 0x7f58180c55d0 2024-04-24T23:35:06.905 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.902+0000 7f58157fa640 1 -- 172.21.15.157:0/3162083045 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f580401a070 con 0x7f58180c55d0 2024-04-24T23:35:06.905 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.903+0000 7f58157fa640 1 --2- 172.21.15.157:0/3162083045 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f57fc06be60 0x7f57fc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:06.905 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.903+0000 7f58157fa640 1 -- 172.21.15.157:0/3162083045 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f580408a630 con 0x7f58180c55d0 2024-04-24T23:35:06.906 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.903+0000 7f581cdae640 1 --2- 172.21.15.157:0/3162083045 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f57fc06be60 0x7f57fc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:06.906 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.904+0000 7f581cdae640 1 --2- 172.21.15.157:0/3162083045 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f57fc06be60 0x7f57fc06e320 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f580c034000 tx=0x7f580c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:06.908 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:06.907+0000 7f58157fa640 1 -- 172.21.15.157:0/3162083045 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f58180c1ad0 con 0x7f58180c55d0 2024-04-24T23:35:07.083 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.083+0000 7f581f012640 1 -- 172.21.15.157:0/3162083045 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f58180681c0 con 0x7f58180c55d0 2024-04-24T23:35:07.084 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.083+0000 7f58157fa640 1 -- 172.21.15.157:0/3162083045 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7f58181ee240 con 0x7f58180c55d0 2024-04-24T23:35:07.084 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2024-04-24T23:35:07.086 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.085+0000 7f581f012640 1 -- 172.21.15.157:0/3162083045 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f57fc06be60 msgr2=0x7f57fc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:07.086 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.085+0000 7f581f012640 1 --2- 172.21.15.157:0/3162083045 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f57fc06be60 0x7f57fc06e320 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f580c034000 tx=0x7f580c02e000 comp rx=0 tx=0).stop 2024-04-24T23:35:07.086 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.085+0000 7f581f012640 1 -- 172.21.15.157:0/3162083045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58180c55d0 msgr2=0x7f58180c9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:07.086 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.085+0000 7f581f012640 1 --2- 172.21.15.157:0/3162083045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58180c55d0 0x7f58180c9aa0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f580400c750 tx=0x7f580400cc10 comp rx=0 tx=0).stop 2024-04-24T23:35:07.087 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.085+0000 7f581f012640 1 -- 172.21.15.157:0/3162083045 shutdown_connections 2024-04-24T23:35:07.087 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.085+0000 7f581f012640 1 --2- 172.21.15.157:0/3162083045 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f57fc06be60 0x7f57fc06e320 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:07.087 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.085+0000 7f581f012640 1 --2- 172.21.15.157:0/3162083045 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58181592f0 0x7f581814ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:07.087 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.085+0000 7f581f012640 1 --2- 172.21.15.157:0/3162083045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58180c55d0 0x7f58180c9aa0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:07.087 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.085+0000 7f581f012640 1 --2- 172.21.15.157:0/3162083045 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58180b03e0 0x7f581814f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:07.087 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.085+0000 7f581f012640 1 -- 172.21.15.157:0/3162083045 >> 172.21.15.157:0/3162083045 conn(0x7f5818067080 msgr2=0x7f58180c7440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:07.087 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.085+0000 7f581f012640 1 -- 172.21.15.157:0/3162083045 shutdown_connections 2024-04-24T23:35:07.087 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.086+0000 7f581f012640 1 -- 172.21.15.157:0/3162083045 wait complete. 2024-04-24T23:35:07.097 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T23:35:07.195 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.194+0000 7f8154770640 1 Processor -- start 2024-04-24T23:35:07.195 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.194+0000 7f8154770640 1 -- start start 2024-04-24T23:35:07.197 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.195+0000 7f8154770640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f814c152b10 0x7f814c155ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:07.197 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.195+0000 7f8154770640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f814c0c6940 0x7f814c0c3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:07.197 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.195+0000 7f8154770640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f814c152ef0 con 0x7f814c0c3940 2024-04-24T23:35:07.197 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.195+0000 7f8154770640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f814c0c2d90 con 0x7f814c0c6940 2024-04-24T23:35:07.197 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.195+0000 7f8154770640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f814c0c1fd0 con 0x7f814c152b10 2024-04-24T23:35:07.197 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.195+0000 7f8151d0b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f814c0c6940 0x7f814c0c3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:07.197 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.195+0000 7f8151d0b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f814c0c6940 0x7f814c0c3400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39778/0 (socket says 172.21.15.157:39778) 2024-04-24T23:35:07.197 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.195+0000 7f8151d0b640 1 -- 172.21.15.157:0/297560675 learned_addr learned my addr 172.21.15.157:0/297560675 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:07.198 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.195+0000 7f815250c640 1 --2- 172.21.15.157:0/297560675 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f814c152b10 0x7f814c155ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:07.198 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.195+0000 7f8151d0b640 1 -- 172.21.15.157:0/297560675 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f814c152b10 msgr2=0x7f814c155ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:07.198 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.195+0000 7f8151d0b640 1 --2- 172.21.15.157:0/297560675 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f814c152b10 0x7f814c155ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:07.198 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.195+0000 7f8151d0b640 1 -- 172.21.15.157:0/297560675 >> v1:172.21.15.169:6789/0 conn(0x7f814c0c3940 legacy=0x7f814c1568e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:07.198 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.196+0000 7f8151d0b640 1 -- 172.21.15.157:0/297560675 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f814c0c22b0 con 0x7f814c0c6940 2024-04-24T23:35:07.198 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.196+0000 7f815250c640 1 --2- 172.21.15.157:0/297560675 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f814c152b10 0x7f814c155ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:35:07.198 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.196+0000 7f8151d0b640 1 --2- 172.21.15.157:0/297560675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f814c0c6940 0x7f814c0c3400 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f813c009870 tx=0x7f813c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d16bce21b3a8217c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:07.198 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.196+0000 7f815150a640 1 -- 172.21.15.157:0/297560675 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813c037070 con 0x7f814c0c6940 2024-04-24T23:35:07.198 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.196+0000 7f815150a640 1 -- 172.21.15.157:0/297560675 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f813c032440 con 0x7f814c0c6940 2024-04-24T23:35:07.198 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.196+0000 7f815150a640 1 -- 172.21.15.157:0/297560675 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813c032810 con 0x7f814c0c6940 2024-04-24T23:35:07.201 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.197+0000 7f8154770640 1 -- 172.21.15.157:0/297560675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f814c0c6940 msgr2=0x7f814c0c3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:07.201 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.197+0000 7f8154770640 1 --2- 172.21.15.157:0/297560675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f814c0c6940 0x7f814c0c3400 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f813c009870 tx=0x7f813c02a210 comp rx=0 tx=0).stop 2024-04-24T23:35:07.201 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.197+0000 7f8154770640 1 -- 172.21.15.157:0/297560675 shutdown_connections 2024-04-24T23:35:07.201 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.197+0000 7f8154770640 1 --2- 172.21.15.157:0/297560675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f814c0c6940 0x7f814c0c3400 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:07.201 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.197+0000 7f8154770640 1 --2- 172.21.15.157:0/297560675 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f814c152b10 0x7f814c155ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:07.202 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.197+0000 7f8154770640 1 -- 172.21.15.157:0/297560675 >> 172.21.15.157:0/297560675 conn(0x7f814c0aa7b0 msgr2=0x7f814c0aab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:07.202 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.197+0000 7f8154770640 1 -- 172.21.15.157:0/297560675 shutdown_connections 2024-04-24T23:35:07.202 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.197+0000 7f8154770640 1 -- 172.21.15.157:0/297560675 wait complete. 2024-04-24T23:35:07.202 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.198+0000 7f8154770640 1 Processor -- start 2024-04-24T23:35:07.202 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.198+0000 7f8154770640 1 -- start start 2024-04-24T23:35:07.202 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.199+0000 7f8154770640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f814c0c3940 0x7f814c1ef150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:07.202 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.199+0000 7f8154770640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f814c0c6940 0x7f814c150f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:07.202 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.199+0000 7f8154770640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f814c152b10 0x7f814c14b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:07.202 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.199+0000 7f8154770640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f814c0c9900 con 0x7f814c0c6940 2024-04-24T23:35:07.202 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.199+0000 7f8154770640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f814c0c2870 con 0x7f814c0c3940 2024-04-24T23:35:07.202 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.199+0000 7f8154770640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f814c156ff0 con 0x7f814c152b10 2024-04-24T23:35:07.202 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.199+0000 7f8151d0b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f814c0c6940 0x7f814c150f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:07.202 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.199+0000 7f8151d0b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f814c0c6940 0x7f814c150f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39794/0 (socket says 172.21.15.157:39794) 2024-04-24T23:35:07.203 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.199+0000 7f8151d0b640 1 -- 172.21.15.157:0/2517526512 learned_addr learned my addr 172.21.15.157:0/2517526512 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:07.203 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.199+0000 7f815250c640 1 --2- 172.21.15.157:0/2517526512 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f814c0c3940 0x7f814c1ef150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:07.203 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.199+0000 7f8152d0d640 1 --2- 172.21.15.157:0/2517526512 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f814c152b10 0x7f814c14b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:07.203 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.199+0000 7f8151d0b640 1 -- 172.21.15.157:0/2517526512 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f814c152b10 msgr2=0x7f814c14b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:07.203 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.199+0000 7f8151d0b640 1 --2- 172.21.15.157:0/2517526512 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f814c152b10 0x7f814c14b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:07.203 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.199+0000 7f8151d0b640 1 -- 172.21.15.157:0/2517526512 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f814c0c3940 msgr2=0x7f814c1ef150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:07.203 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.199+0000 7f8151d0b640 1 --2- 172.21.15.157:0/2517526512 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f814c0c3940 0x7f814c1ef150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:07.203 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.199+0000 7f8151d0b640 1 -- 172.21.15.157:0/2517526512 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f814c0acd90 con 0x7f814c0c6940 2024-04-24T23:35:07.203 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.199+0000 7f815250c640 1 --2- 172.21.15.157:0/2517526512 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f814c0c3940 0x7f814c1ef150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:35:07.203 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.199+0000 7f8152d0d640 1 --2- 172.21.15.157:0/2517526512 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f814c152b10 0x7f814c14b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:35:07.203 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.199+0000 7f8151d0b640 1 --2- 172.21.15.157:0/2517526512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f814c0c6940 0x7f814c150f90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f813c009870 tx=0x7f813c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:07.203 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.199+0000 7f8142ffd640 1 -- 172.21.15.157:0/2517526512 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813c041070 con 0x7f814c0c6940 2024-04-24T23:35:07.203 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.199+0000 7f8142ffd640 1 -- 172.21.15.157:0/2517526512 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f813c03c870 con 0x7f814c0c6940 2024-04-24T23:35:07.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.199+0000 7f8154770640 1 -- 172.21.15.157:0/2517526512 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f814c0ad070 con 0x7f814c0c6940 2024-04-24T23:35:07.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.199+0000 7f8142ffd640 1 -- 172.21.15.157:0/2517526512 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813c037040 con 0x7f814c0c6940 2024-04-24T23:35:07.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.199+0000 7f8154770640 1 -- 172.21.15.157:0/2517526512 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f814c1514d0 con 0x7f814c0c6940 2024-04-24T23:35:07.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.200+0000 7f8154770640 1 -- 172.21.15.157:0/2517526512 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f814c0ad590 con 0x7f814c0c6940 2024-04-24T23:35:07.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.201+0000 7f8142ffd640 1 -- 172.21.15.157:0/2517526512 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f813c044070 con 0x7f814c0c6940 2024-04-24T23:35:07.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.201+0000 7f8142ffd640 1 --2- 172.21.15.157:0/2517526512 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f813006be30 0x7f813006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:07.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.201+0000 7f8142ffd640 1 -- 172.21.15.157:0/2517526512 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f813c0abcd0 con 0x7f814c0c6940 2024-04-24T23:35:07.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.201+0000 7f815250c640 1 --2- 172.21.15.157:0/2517526512 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f813006be30 0x7f813006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:07.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.202+0000 7f815250c640 1 --2- 172.21.15.157:0/2517526512 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f813006be30 0x7f813006e2f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f814800e3d0 tx=0x7f8148007a10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:07.206 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.205+0000 7f8142ffd640 1 -- 172.21.15.157:0/2517526512 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f814c0ad590 con 0x7f814c0c6940 2024-04-24T23:35:07.397 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.396+0000 7f8154770640 1 -- 172.21.15.157:0/2517526512 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f814c0681c0 con 0x7f814c0c6940 2024-04-24T23:35:07.397 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.396+0000 7f8142ffd640 1 -- 172.21.15.157:0/2517526512 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7f814c152ef0 con 0x7f814c0c6940 2024-04-24T23:35:07.397 INFO:teuthology.orchestra.run.smithi157.stderr:listed 0 entries 2024-04-24T23:35:07.399 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.398+0000 7f8154770640 1 -- 172.21.15.157:0/2517526512 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f813006be30 msgr2=0x7f813006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:07.400 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.398+0000 7f8154770640 1 --2- 172.21.15.157:0/2517526512 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f813006be30 0x7f813006e2f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f814800e3d0 tx=0x7f8148007a10 comp rx=0 tx=0).stop 2024-04-24T23:35:07.400 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.398+0000 7f8154770640 1 -- 172.21.15.157:0/2517526512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f814c0c6940 msgr2=0x7f814c150f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:07.400 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.398+0000 7f8154770640 1 --2- 172.21.15.157:0/2517526512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f814c0c6940 0x7f814c150f90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f813c009870 tx=0x7f813c0046c0 comp rx=0 tx=0).stop 2024-04-24T23:35:07.400 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.399+0000 7f8154770640 1 -- 172.21.15.157:0/2517526512 shutdown_connections 2024-04-24T23:35:07.400 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.399+0000 7f8154770640 1 --2- 172.21.15.157:0/2517526512 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f813006be30 0x7f813006e2f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:07.400 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.399+0000 7f8154770640 1 --2- 172.21.15.157:0/2517526512 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f814c152b10 0x7f814c14b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:07.400 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.399+0000 7f8154770640 1 --2- 172.21.15.157:0/2517526512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f814c0c6940 0x7f814c150f90 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:07.400 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.399+0000 7f8154770640 1 --2- 172.21.15.157:0/2517526512 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f814c0c3940 0x7f814c1ef150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:07.400 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.399+0000 7f8154770640 1 -- 172.21.15.157:0/2517526512 >> 172.21.15.157:0/2517526512 conn(0x7f814c0aa7b0 msgr2=0x7f814c0c7fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:07.400 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.399+0000 7f8154770640 1 -- 172.21.15.157:0/2517526512 shutdown_connections 2024-04-24T23:35:07.400 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:07.399+0000 7f8154770640 1 -- 172.21.15.157:0/2517526512 wait complete. 2024-04-24T23:35:07.410 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi157.front.sepia.ceph.com... 2024-04-24T23:35:07.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:35:07.410 DEBUG:teuthology.orchestra.run.smithi157:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-24T23:35:07.433 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi157.stderr:ceph-fuse[39497]: fuse finished with error 0 and tester_r 0 2024-04-24T23:35:07.435 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:35:13.536 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-04-24T23:35:13.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:35:13.537 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-24T23:35:13.575 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:35:13.723 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.723+0000 7eff22f16640 1 Processor -- start 2024-04-24T23:35:13.723 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.723+0000 7eff22f16640 1 -- start start 2024-04-24T23:35:13.724 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.723+0000 7eff22f16640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7eff1c14b950 0x7eff1c0c2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:13.724 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.723+0000 7eff22f16640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff1c1527d0 con 0x7eff1c154e60 2024-04-24T23:35:13.724 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.724+0000 7eff22f16640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff1c14bd30 con 0x7eff1c151ca0 2024-04-24T23:35:13.725 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.724+0000 7eff22f16640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7eff1c06c480 con 0x7eff1c14b950 2024-04-24T23:35:13.725 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.724+0000 7eff20cb2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7eff1c14b950 0x7eff1c0c2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:13.725 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.724+0000 7eff1bfff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7eff1c154e60 0x7eff1c0c5100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:47562/0 (socket says 172.21.15.157:47562) 2024-04-24T23:35:13.725 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.724+0000 7eff1bfff640 1 -- 172.21.15.157:0/3301902081 learned_addr learned my addr 172.21.15.157:0/3301902081 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-24T23:35:13.725 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.724+0000 7eff20cb2640 1 -- 172.21.15.157:0/3301902081 >> v1:172.21.15.169:6789/0 conn(0x7eff1c151ca0 legacy=0x7eff1c1520c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:13.725 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.724+0000 7eff20cb2640 1 -- 172.21.15.157:0/3301902081 >> v1:172.21.15.157:6789/0 conn(0x7eff1c154e60 legacy=0x7eff1c0c5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:13.725 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.724+0000 7eff20cb2640 1 -- 172.21.15.157:0/3301902081 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff1c1552f0 con 0x7eff1c14b950 2024-04-24T23:35:13.725 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.724+0000 7eff1b7fe640 1 -- 172.21.15.157:0/3301902081 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4264109963 0 0) 0x7eff1c1527d0 con 0x7eff1c154e60 2024-04-24T23:35:13.726 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.725+0000 7eff20cb2640 1 --2- 172.21.15.157:0/3301902081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff1c14b950 0x7eff1c0c2b30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7eff100097a0 tx=0x7eff1002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c53595eaed37f1b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:13.726 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.725+0000 7eff1b7fe640 1 -- 172.21.15.157:0/3301902081 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff10037070 con 0x7eff1c14b950 2024-04-24T23:35:13.727 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.725+0000 7eff1b7fe640 1 -- 172.21.15.157:0/3301902081 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff10032440 con 0x7eff1c14b950 2024-04-24T23:35:13.727 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.725+0000 7eff1b7fe640 1 -- 172.21.15.157:0/3301902081 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff10032810 con 0x7eff1c14b950 2024-04-24T23:35:13.727 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.725+0000 7eff22f16640 1 -- 172.21.15.157:0/3301902081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff1c14b950 msgr2=0x7eff1c0c2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:13.727 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.725+0000 7eff22f16640 1 --2- 172.21.15.157:0/3301902081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff1c14b950 0x7eff1c0c2b30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7eff100097a0 tx=0x7eff1002a1a0 comp rx=0 tx=0).stop 2024-04-24T23:35:13.727 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.725+0000 7eff22f16640 1 -- 172.21.15.157:0/3301902081 shutdown_connections 2024-04-24T23:35:13.727 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.725+0000 7eff22f16640 1 --2- 172.21.15.157:0/3301902081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff1c14b950 0x7eff1c0c2b30 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:13.727 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.725+0000 7eff22f16640 1 -- 172.21.15.157:0/3301902081 >> 172.21.15.157:0/3301902081 conn(0x7eff1c0a8720 msgr2=0x7eff1c0a8b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:13.727 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.726+0000 7eff22f16640 1 -- 172.21.15.157:0/3301902081 shutdown_connections 2024-04-24T23:35:13.727 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.726+0000 7eff22f16640 1 -- 172.21.15.157:0/3301902081 wait complete. 2024-04-24T23:35:13.729 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.727+0000 7eff22f16640 1 Processor -- start 2024-04-24T23:35:13.729 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.727+0000 7eff22f16640 1 -- start start 2024-04-24T23:35:13.730 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.727+0000 7eff22f16640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff1c14b950 0x7eff1c1ec490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:13.730 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.727+0000 7eff22f16640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff1c151ca0 0x7eff1c1ec9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:13.730 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.727+0000 7eff22f16640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff1c153490 0x7eff1c153940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:13.730 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.727+0000 7eff22f16640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c155b90 con 0x7eff1c151ca0 2024-04-24T23:35:13.730 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.727+0000 7eff22f16640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c1558b0 con 0x7eff1c153490 2024-04-24T23:35:13.730 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.727+0000 7eff22f16640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7eff1c0c5810 con 0x7eff1c14b950 2024-04-24T23:35:13.730 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.728+0000 7eff1bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff1c151ca0 0x7eff1c1ec9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:13.730 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.728+0000 7eff1bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff1c151ca0 0x7eff1c1ec9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42402/0 (socket says 172.21.15.157:42402) 2024-04-24T23:35:13.730 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.728+0000 7eff1bfff640 1 -- 172.21.15.157:0/3753875317 learned_addr learned my addr 172.21.15.157:0/3753875317 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:13.730 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.728+0000 7eff20cb2640 1 --2- 172.21.15.157:0/3753875317 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff1c14b950 0x7eff1c1ec490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:13.730 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.728+0000 7eff214b3640 1 --2- 172.21.15.157:0/3753875317 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff1c153490 0x7eff1c153940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:13.730 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.728+0000 7eff1bfff640 1 -- 172.21.15.157:0/3753875317 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff1c14b950 msgr2=0x7eff1c1ec490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:13.730 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.728+0000 7eff1bfff640 1 --2- 172.21.15.157:0/3753875317 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff1c14b950 0x7eff1c1ec490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:13.731 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.728+0000 7eff1bfff640 1 -- 172.21.15.157:0/3753875317 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff1c153490 msgr2=0x7eff1c153940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:13.731 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.728+0000 7eff1bfff640 1 --2- 172.21.15.157:0/3753875317 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff1c153490 0x7eff1c153940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:13.731 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.728+0000 7eff1bfff640 1 -- 172.21.15.157:0/3753875317 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff1c143e80 con 0x7eff1c151ca0 2024-04-24T23:35:13.731 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.728+0000 7eff214b3640 1 --2- 172.21.15.157:0/3753875317 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff1c153490 0x7eff1c153940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:35:13.731 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.728+0000 7eff1bfff640 1 --2- 172.21.15.157:0/3753875317 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff1c151ca0 0x7eff1c1ec9d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7eff0800c500 tx=0x7eff0800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:13.731 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.728+0000 7eff197fa640 1 -- 172.21.15.157:0/3753875317 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff08018070 con 0x7eff1c151ca0 2024-04-24T23:35:13.731 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.728+0000 7eff197fa640 1 -- 172.21.15.157:0/3753875317 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff08007580 con 0x7eff1c151ca0 2024-04-24T23:35:13.731 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.728+0000 7eff22f16640 1 -- 172.21.15.157:0/3753875317 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff1c143ba0 con 0x7eff1c151ca0 2024-04-24T23:35:13.731 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.728+0000 7eff197fa640 1 -- 172.21.15.157:0/3753875317 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff08004a20 con 0x7eff1c151ca0 2024-04-24T23:35:13.731 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.728+0000 7eff22f16640 1 -- 172.21.15.157:0/3753875317 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff1c151550 con 0x7eff1c151ca0 2024-04-24T23:35:13.732 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.730+0000 7eff22f16640 1 -- 172.21.15.157:0/3753875317 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff1c0681c0 con 0x7eff1c151ca0 2024-04-24T23:35:13.732 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.730+0000 7eff197fa640 1 -- 172.21.15.157:0/3753875317 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7eff0801b070 con 0x7eff1c151ca0 2024-04-24T23:35:13.732 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.730+0000 7eff197fa640 1 --2- 172.21.15.157:0/3753875317 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7eff0006be30 0x7eff0006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:13.732 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.730+0000 7eff197fa640 1 -- 172.21.15.157:0/3753875317 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7eff0808a360 con 0x7eff1c151ca0 2024-04-24T23:35:13.732 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.731+0000 7eff20cb2640 1 --2- 172.21.15.157:0/3753875317 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7eff0006be30 0x7eff0006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:13.733 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.731+0000 7eff20cb2640 1 --2- 172.21.15.157:0/3753875317 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7eff0006be30 0x7eff0006e2f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7eff10034000 tx=0x7eff1002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:13.735 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.735+0000 7eff197fa640 1 -- 172.21.15.157:0/3753875317 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7eff08058410 con 0x7eff1c151ca0 2024-04-24T23:35:13.915 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.914+0000 7eff22f16640 1 -- 172.21.15.157:0/3753875317 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7eff1c0c6620 con 0x7eff1c151ca0 2024-04-24T23:35:13.916 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.916+0000 7eff197fa640 1 -- 172.21.15.157:0/3753875317 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18538 (secure 0 0 0) 0x7eff0805c710 con 0x7eff1c151ca0 2024-04-24T23:35:13.917 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:35:13.917 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":25,"fsid":"223685fb-5b64-46a5-9857-3791c9c07df2","created":"2024-04-24T23:34:37.504202+0000","modified":"2024-04-24T23:34:58.744500+0000","last_up_change":"2024-04-24T23:34:47.584963+0000","last_in_change":"2024-04-24T23:34:39.632954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:34:47.796655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:34:52.177208+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:34:52.667022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:34:56.395517+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"95d14ac8-fccf-4261-be8e-edafa746f629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6809","nonce":4003822160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6811","nonce":4003822160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6815","nonce":4003822160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6813","nonce":4003822160}]},"public_addr":"172.21.15.157:6809/4003822160","cluster_addr":"172.21.15.157:6811/4003822160","heartbeat_back_addr":"172.21.15.157:6815/4003822160","heartbeat_front_addr":"172.21.15.157:6813/4003822160","state":["exists","up"]},{"osd":1,"uuid":"7be79da9-8eec-4990-9178-71d602e39d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6817","nonce":1557575576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6819","nonce":1557575576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6823","nonce":1557575576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6821","nonce":1557575576}]},"public_addr":"172.21.15.157:6817/1557575576","cluster_addr":"172.21.15.157:6819/1557575576","heartbeat_back_addr":"172.21.15.157:6823/1557575576","heartbeat_front_addr":"172.21.15.157:6821/1557575576","state":["exists","up"]},{"osd":2,"uuid":"0127ed00-b7c3-466d-bfb0-01b0fd315f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6825","nonce":2793948092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6827","nonce":2793948092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6831","nonce":2793948092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6829","nonce":2793948092}]},"public_addr":"172.21.15.157:6825/2793948092","cluster_addr":"172.21.15.157:6827/2793948092","heartbeat_back_addr":"172.21.15.157:6831/2793948092","heartbeat_front_addr":"172.21.15.157:6829/2793948092","state":["exists","up"]},{"osd":3,"uuid":"f2331acc-aa37-433e-868e-3561b6039562","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6801","nonce":422758549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6803","nonce":422758549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6807","nonce":422758549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6805","nonce":422758549}]},"public_addr":"172.21.15.157:6801/422758549","cluster_addr":"172.21.15.157:6803/422758549","heartbeat_back_addr":"172.21.15.157:6807/422758549","heartbeat_front_addr":"172.21.15.157:6805/422758549","state":["exists","up"]},{"osd":4,"uuid":"1998854b-7398-43f5-ab7d-79fa0debcf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6801","nonce":321910099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6803","nonce":321910099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6807","nonce":321910099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6805","nonce":321910099}]},"public_addr":"172.21.15.169:6801/321910099","cluster_addr":"172.21.15.169:6803/321910099","heartbeat_back_addr":"172.21.15.169:6807/321910099","heartbeat_front_addr":"172.21.15.169:6805/321910099","state":["exists","up"]},{"osd":5,"uuid":"c37153ff-9b2b-48b9-a684-2c82b6f39f15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6825","nonce":243321711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6827","nonce":243321711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6831","nonce":243321711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6829","nonce":243321711}]},"public_addr":"172.21.15.169:6825/243321711","cluster_addr":"172.21.15.169:6827/243321711","heartbeat_back_addr":"172.21.15.169:6831/243321711","heartbeat_front_addr":"172.21.15.169:6829/243321711","state":["exists","up"]},{"osd":6,"uuid":"8db79e1c-f36b-49cb-8d8a-a20c2aec57c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6809","nonce":3441420785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6811","nonce":3441420785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6815","nonce":3441420785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6813","nonce":3441420785}]},"public_addr":"172.21.15.169:6809/3441420785","cluster_addr":"172.21.15.169:6811/3441420785","heartbeat_back_addr":"172.21.15.169:6815/3441420785","heartbeat_front_addr":"172.21.15.169:6813/3441420785","state":["exists","up"]},{"osd":7,"uuid":"928586e9-296c-4b8b-9b27-42f9f155dbdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6817","nonce":1663179020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6819","nonce":1663179020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6823","nonce":1663179020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6821","nonce":1663179020}]},"public_addr":"172.21.15.169:6817/1663179020","cluster_addr":"172.21.15.169:6819/1663179020","heartbeat_back_addr":"172.21.15.169:6823/1663179020","heartbeat_front_addr":"172.21.15.169:6821/1663179020","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.848039+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.826489+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.801120+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.727706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.824343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.791915+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.878073+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.810831+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:35:13.919 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.918+0000 7eff22f16640 1 -- 172.21.15.157:0/3753875317 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7eff0006be30 msgr2=0x7eff0006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:13.919 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.918+0000 7eff22f16640 1 --2- 172.21.15.157:0/3753875317 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7eff0006be30 0x7eff0006e2f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7eff10034000 tx=0x7eff1002e000 comp rx=0 tx=0).stop 2024-04-24T23:35:13.919 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.918+0000 7eff22f16640 1 -- 172.21.15.157:0/3753875317 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff1c151ca0 msgr2=0x7eff1c1ec9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:13.919 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.918+0000 7eff22f16640 1 --2- 172.21.15.157:0/3753875317 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff1c151ca0 0x7eff1c1ec9d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7eff0800c500 tx=0x7eff0800c9c0 comp rx=0 tx=0).stop 2024-04-24T23:35:13.920 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.918+0000 7eff22f16640 1 -- 172.21.15.157:0/3753875317 shutdown_connections 2024-04-24T23:35:13.920 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.918+0000 7eff22f16640 1 --2- 172.21.15.157:0/3753875317 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7eff0006be30 0x7eff0006e2f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:13.920 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.918+0000 7eff22f16640 1 --2- 172.21.15.157:0/3753875317 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff1c153490 0x7eff1c153940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:13.920 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.918+0000 7eff22f16640 1 --2- 172.21.15.157:0/3753875317 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff1c151ca0 0x7eff1c1ec9d0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:13.920 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.918+0000 7eff22f16640 1 --2- 172.21.15.157:0/3753875317 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff1c14b950 0x7eff1c1ec490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:13.920 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.918+0000 7eff22f16640 1 -- 172.21.15.157:0/3753875317 >> 172.21.15.157:0/3753875317 conn(0x7eff1c0a8720 msgr2=0x7eff1c0c4df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:13.920 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.918+0000 7eff22f16640 1 -- 172.21.15.157:0/3753875317 shutdown_connections 2024-04-24T23:35:13.920 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:13.918+0000 7eff22f16640 1 -- 172.21.15.157:0/3753875317 wait complete. 2024-04-24T23:35:13.930 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:35:14.026 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.025+0000 7fa3b2db0640 1 Processor -- start 2024-04-24T23:35:14.026 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.025+0000 7fa3b2db0640 1 -- start start 2024-04-24T23:35:14.030 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.026+0000 7fa3b2db0640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa3ac06b160 0x7fa3ac06b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:14.030 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.026+0000 7fa3b2db0640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa3ac0c7470 0x7fa3ac158920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:14.030 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.026+0000 7fa3b2db0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3ac0a74d0 con 0x7fa3ac0c2380 2024-04-24T23:35:14.030 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.026+0000 7fa3b2db0640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fa3ac1562e0 con 0x7fa3ac06b160 2024-04-24T23:35:14.030 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.026+0000 7fa3b2db0640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa3ac0c4030 con 0x7fa3ac0c7470 2024-04-24T23:35:14.030 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.026+0000 7fa3b134d640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fa3ac0c2380 0x7fa3ac072910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:47568/0 (socket says 172.21.15.157:47568) 2024-04-24T23:35:14.030 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.026+0000 7fa3b134d640 1 -- 172.21.15.157:0/4213960437 learned_addr learned my addr 172.21.15.157:0/4213960437 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-24T23:35:14.030 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.027+0000 7fa3b0b4c640 1 --2- 172.21.15.157:0/4213960437 >> v2:172.21.15.169:3300/0 conn(0x7fa3ac06b160 0x7fa3ac06b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:14.031 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.027+0000 7fa3abfff640 1 --2- 172.21.15.157:0/4213960437 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa3ac0c7470 0x7fa3ac158920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:14.031 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.027+0000 7fa3ab7fe640 1 -- 172.21.15.157:0/4213960437 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2180857161 0 0) 0x7fa3ac0a74d0 con 0x7fa3ac0c2380 2024-04-24T23:35:14.031 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.027+0000 7fa3ab7fe640 1 -- 172.21.15.157:0/4213960437 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa394003520 con 0x7fa3ac0c2380 2024-04-24T23:35:14.031 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.027+0000 7fa3ab7fe640 1 -- 172.21.15.157:0/4213960437 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4104123185 0 0) 0x7fa394003520 con 0x7fa3ac0c2380 2024-04-24T23:35:14.031 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.027+0000 7fa3ab7fe640 1 -- 172.21.15.157:0/4213960437 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa3ac0c7470 msgr2=0x7fa3ac158920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:14.031 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.027+0000 7fa3ab7fe640 1 --2- 172.21.15.157:0/4213960437 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa3ac0c7470 0x7fa3ac158920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:14.031 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.027+0000 7fa3ab7fe640 1 -- 172.21.15.157:0/4213960437 >> v2:172.21.15.169:3300/0 conn(0x7fa3ac06b160 msgr2=0x7fa3ac06b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:14.031 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.027+0000 7fa3ab7fe640 1 --2- 172.21.15.157:0/4213960437 >> v2:172.21.15.169:3300/0 conn(0x7fa3ac06b160 0x7fa3ac06b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:14.031 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.027+0000 7fa3ab7fe640 1 -- 172.21.15.157:0/4213960437 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3ac156000 con 0x7fa3ac0c2380 2024-04-24T23:35:14.031 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.027+0000 7fa3ab7fe640 1 -- 172.21.15.157:0/4213960437 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1480525340 0 0) 0x7fa3a40019f0 con 0x7fa3ac0c2380 2024-04-24T23:35:14.031 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.027+0000 7fa3ab7fe640 1 -- 172.21.15.157:0/4213960437 >> v1:172.21.15.157:6789/0 conn(0x7fa3ac0c2380 legacy=0x7fa3ac072910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:14.032 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.027+0000 7fa3ab7fe640 1 --2- 172.21.15.157:0/4213960437 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3940039b0 0x7fa394003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:14.032 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.027+0000 7fa3ab7fe640 1 --2- 172.21.15.157:0/4213960437 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa3ac0c7470 0x7fa394004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:14.032 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.027+0000 7fa3ab7fe640 1 --2- 172.21.15.157:0/4213960437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa394004d40 0x7fa3940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:14.032 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.027+0000 7fa3ab7fe640 1 -- 172.21.15.157:0/4213960437 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ac1562e0 con 0x7fa394004d40 2024-04-24T23:35:14.032 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.027+0000 7fa3ab7fe640 1 -- 172.21.15.157:0/4213960437 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ac0c4030 con 0x7fa3940039b0 2024-04-24T23:35:14.032 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.027+0000 7fa3ab7fe640 1 -- 172.21.15.157:0/4213960437 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa39400b9c0 con 0x7fa3ac0c7470 2024-04-24T23:35:14.032 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.027+0000 7fa3b134d640 1 --2- 172.21.15.157:0/4213960437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa394004d40 0x7fa3940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:14.032 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.027+0000 7fa3b0b4c640 1 --2- 172.21.15.157:0/4213960437 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3940039b0 0x7fa394003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:14.032 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.027+0000 7fa3abfff640 1 --2- 172.21.15.157:0/4213960437 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa3ac0c7470 0x7fa394004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:14.032 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.027+0000 7fa3b134d640 1 -- 172.21.15.157:0/4213960437 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa3ac0c7470 msgr2=0x7fa394004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:14.032 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.027+0000 7fa3b134d640 1 --2- 172.21.15.157:0/4213960437 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa3ac0c7470 0x7fa394004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:14.032 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.027+0000 7fa3b134d640 1 -- 172.21.15.157:0/4213960437 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3940039b0 msgr2=0x7fa394003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:14.032 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.027+0000 7fa3b134d640 1 --2- 172.21.15.157:0/4213960437 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3940039b0 0x7fa394003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:14.033 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.027+0000 7fa3b134d640 1 -- 172.21.15.157:0/4213960437 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa394003520 con 0x7fa394004d40 2024-04-24T23:35:14.033 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.028+0000 7fa3b134d640 1 --2- 172.21.15.157:0/4213960437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa394004d40 0x7fa3940051c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa3a400a870 tx=0x7fa394005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6a859c1529a6ced server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:14.033 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.028+0000 7fa3ab7fe640 1 -- 172.21.15.157:0/4213960437 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a400a4c0 con 0x7fa394004d40 2024-04-24T23:35:14.033 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.028+0000 7fa3ab7fe640 1 -- 172.21.15.157:0/4213960437 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3a402f070 con 0x7fa394004d40 2024-04-24T23:35:14.033 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.028+0000 7fa3ab7fe640 1 -- 172.21.15.157:0/4213960437 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a402bcc0 con 0x7fa394004d40 2024-04-24T23:35:14.033 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.029+0000 7fa3b2db0640 1 -- 172.21.15.157:0/4213960437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa394004d40 msgr2=0x7fa3940051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:14.033 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.029+0000 7fa3b2db0640 1 --2- 172.21.15.157:0/4213960437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa394004d40 0x7fa3940051c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa3a400a870 tx=0x7fa394005830 comp rx=0 tx=0).stop 2024-04-24T23:35:14.033 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.029+0000 7fa3b134d640 1 -- 172.21.15.157:0/4213960437 reap_dead start 2024-04-24T23:35:14.033 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.029+0000 7fa3b2db0640 1 -- 172.21.15.157:0/4213960437 shutdown_connections 2024-04-24T23:35:14.033 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.029+0000 7fa3b2db0640 1 -- 172.21.15.157:0/4213960437 >> 172.21.15.157:0/4213960437 conn(0x7fa3ac0a9730 msgr2=0x7fa3ac0a9b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:14.033 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.029+0000 7fa3b2db0640 1 -- 172.21.15.157:0/4213960437 shutdown_connections 2024-04-24T23:35:14.033 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.029+0000 7fa3b2db0640 1 -- 172.21.15.157:0/4213960437 wait complete. 2024-04-24T23:35:14.036 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.030+0000 7fa3b2db0640 1 Processor -- start 2024-04-24T23:35:14.036 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.030+0000 7fa3b2db0640 1 -- start start 2024-04-24T23:35:14.036 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.031+0000 7fa3b2db0640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa3ac156ac0 0x7fa3ac152590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:14.036 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.031+0000 7fa3b2db0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3ac1593b0 0x7fa3ac1eb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:14.036 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.031+0000 7fa3b2db0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3ac156ea0 0x7fa3ac153620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:14.036 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.031+0000 7fa3b2db0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ac155a60 con 0x7fa3ac1593b0 2024-04-24T23:35:14.036 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.031+0000 7fa3b2db0640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ac0c1c80 con 0x7fa3ac156ea0 2024-04-24T23:35:14.037 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.031+0000 7fa3b2db0640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa3ac155be0 con 0x7fa3ac156ac0 2024-04-24T23:35:14.037 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.031+0000 7fa3abfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3ac1593b0 0x7fa3ac1eb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:14.037 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.031+0000 7fa3abfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3ac1593b0 0x7fa3ac1eb8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42430/0 (socket says 172.21.15.157:42430) 2024-04-24T23:35:14.037 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.031+0000 7fa3abfff640 1 -- 172.21.15.157:0/3493575501 learned_addr learned my addr 172.21.15.157:0/3493575501 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:14.037 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.031+0000 7fa3abfff640 1 -- 172.21.15.157:0/3493575501 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa3ac156ac0 msgr2=0x7fa3ac152590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:14.037 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.031+0000 7fa3abfff640 1 --2- 172.21.15.157:0/3493575501 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa3ac156ac0 0x7fa3ac152590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:14.037 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.031+0000 7fa3abfff640 1 -- 172.21.15.157:0/3493575501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3ac156ea0 msgr2=0x7fa3ac153620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:14.037 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.031+0000 7fa3abfff640 1 --2- 172.21.15.157:0/3493575501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3ac156ea0 0x7fa3ac153620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:14.037 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.031+0000 7fa3abfff640 1 -- 172.21.15.157:0/3493575501 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3ac1ebe10 con 0x7fa3ac1593b0 2024-04-24T23:35:14.037 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.031+0000 7fa3abfff640 1 --2- 172.21.15.157:0/3493575501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3ac1593b0 0x7fa3ac1eb8d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0011940 tx=0x7fa3a0011e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:14.037 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.031+0000 7fa3a97fa640 1 -- 172.21.15.157:0/3493575501 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a0019070 con 0x7fa3ac1593b0 2024-04-24T23:35:14.037 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.031+0000 7fa3a97fa640 1 -- 172.21.15.157:0/3493575501 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3a0004240 con 0x7fa3ac1593b0 2024-04-24T23:35:14.038 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.031+0000 7fa3b2db0640 1 -- 172.21.15.157:0/3493575501 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3ac1550d0 con 0x7fa3ac1593b0 2024-04-24T23:35:14.038 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.031+0000 7fa3a97fa640 1 -- 172.21.15.157:0/3493575501 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a000f860 con 0x7fa3ac1593b0 2024-04-24T23:35:14.038 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.031+0000 7fa3b2db0640 1 -- 172.21.15.157:0/3493575501 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3ac1eb0d0 con 0x7fa3ac1593b0 2024-04-24T23:35:14.038 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.032+0000 7fa3b2db0640 1 -- 172.21.15.157:0/3493575501 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3ac1eaee0 con 0x7fa3ac1593b0 2024-04-24T23:35:14.038 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.033+0000 7fa3a97fa640 1 -- 172.21.15.157:0/3493575501 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa3a001c070 con 0x7fa3ac1593b0 2024-04-24T23:35:14.038 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.033+0000 7fa3a97fa640 1 --2- 172.21.15.157:0/3493575501 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fa39806be60 0x7fa39806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:14.038 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.033+0000 7fa3a97fa640 1 -- 172.21.15.157:0/3493575501 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa3a008b5b0 con 0x7fa3ac1593b0 2024-04-24T23:35:14.038 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.033+0000 7fa3b0b4c640 1 --2- 172.21.15.157:0/3493575501 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fa39806be60 0x7fa39806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:14.038 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.034+0000 7fa3b0b4c640 1 --2- 172.21.15.157:0/3493575501 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fa39806be60 0x7fa39806e320 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa39c006b10 tx=0x7fa39c006aa0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:14.039 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.038+0000 7fa3a97fa640 1 -- 172.21.15.157:0/3493575501 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa3a0059c10 con 0x7fa3ac1593b0 2024-04-24T23:35:14.213 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.213+0000 7fa3b2db0640 1 -- 172.21.15.157:0/3493575501 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa3ac0688d0 con 0x7fa3ac1593b0 2024-04-24T23:35:14.215 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.214+0000 7fa3a97fa640 1 -- 172.21.15.157:0/3493575501 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18538 (secure 0 0 0) 0x7fa3a0059510 con 0x7fa3ac1593b0 2024-04-24T23:35:14.215 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:35:14.215 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":25,"fsid":"223685fb-5b64-46a5-9857-3791c9c07df2","created":"2024-04-24T23:34:37.504202+0000","modified":"2024-04-24T23:34:58.744500+0000","last_up_change":"2024-04-24T23:34:47.584963+0000","last_in_change":"2024-04-24T23:34:39.632954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:34:47.796655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:34:52.177208+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:34:52.667022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:34:56.395517+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"95d14ac8-fccf-4261-be8e-edafa746f629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6809","nonce":4003822160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6811","nonce":4003822160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6815","nonce":4003822160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6813","nonce":4003822160}]},"public_addr":"172.21.15.157:6809/4003822160","cluster_addr":"172.21.15.157:6811/4003822160","heartbeat_back_addr":"172.21.15.157:6815/4003822160","heartbeat_front_addr":"172.21.15.157:6813/4003822160","state":["exists","up"]},{"osd":1,"uuid":"7be79da9-8eec-4990-9178-71d602e39d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6817","nonce":1557575576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6819","nonce":1557575576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6823","nonce":1557575576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6821","nonce":1557575576}]},"public_addr":"172.21.15.157:6817/1557575576","cluster_addr":"172.21.15.157:6819/1557575576","heartbeat_back_addr":"172.21.15.157:6823/1557575576","heartbeat_front_addr":"172.21.15.157:6821/1557575576","state":["exists","up"]},{"osd":2,"uuid":"0127ed00-b7c3-466d-bfb0-01b0fd315f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6825","nonce":2793948092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6827","nonce":2793948092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6831","nonce":2793948092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6829","nonce":2793948092}]},"public_addr":"172.21.15.157:6825/2793948092","cluster_addr":"172.21.15.157:6827/2793948092","heartbeat_back_addr":"172.21.15.157:6831/2793948092","heartbeat_front_addr":"172.21.15.157:6829/2793948092","state":["exists","up"]},{"osd":3,"uuid":"f2331acc-aa37-433e-868e-3561b6039562","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6801","nonce":422758549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6803","nonce":422758549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6807","nonce":422758549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6805","nonce":422758549}]},"public_addr":"172.21.15.157:6801/422758549","cluster_addr":"172.21.15.157:6803/422758549","heartbeat_back_addr":"172.21.15.157:6807/422758549","heartbeat_front_addr":"172.21.15.157:6805/422758549","state":["exists","up"]},{"osd":4,"uuid":"1998854b-7398-43f5-ab7d-79fa0debcf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6801","nonce":321910099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6803","nonce":321910099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6807","nonce":321910099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6805","nonce":321910099}]},"public_addr":"172.21.15.169:6801/321910099","cluster_addr":"172.21.15.169:6803/321910099","heartbeat_back_addr":"172.21.15.169:6807/321910099","heartbeat_front_addr":"172.21.15.169:6805/321910099","state":["exists","up"]},{"osd":5,"uuid":"c37153ff-9b2b-48b9-a684-2c82b6f39f15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6825","nonce":243321711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6827","nonce":243321711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6831","nonce":243321711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6829","nonce":243321711}]},"public_addr":"172.21.15.169:6825/243321711","cluster_addr":"172.21.15.169:6827/243321711","heartbeat_back_addr":"172.21.15.169:6831/243321711","heartbeat_front_addr":"172.21.15.169:6829/243321711","state":["exists","up"]},{"osd":6,"uuid":"8db79e1c-f36b-49cb-8d8a-a20c2aec57c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6809","nonce":3441420785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6811","nonce":3441420785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6815","nonce":3441420785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6813","nonce":3441420785}]},"public_addr":"172.21.15.169:6809/3441420785","cluster_addr":"172.21.15.169:6811/3441420785","heartbeat_back_addr":"172.21.15.169:6815/3441420785","heartbeat_front_addr":"172.21.15.169:6813/3441420785","state":["exists","up"]},{"osd":7,"uuid":"928586e9-296c-4b8b-9b27-42f9f155dbdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6817","nonce":1663179020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6819","nonce":1663179020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6823","nonce":1663179020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6821","nonce":1663179020}]},"public_addr":"172.21.15.169:6817/1663179020","cluster_addr":"172.21.15.169:6819/1663179020","heartbeat_back_addr":"172.21.15.169:6823/1663179020","heartbeat_front_addr":"172.21.15.169:6821/1663179020","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.848039+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.826489+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.801120+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.727706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.824343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.791915+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.878073+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.810831+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:35:14.217 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.216+0000 7fa3b2db0640 1 -- 172.21.15.157:0/3493575501 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fa39806be60 msgr2=0x7fa39806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:14.217 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.216+0000 7fa3b2db0640 1 --2- 172.21.15.157:0/3493575501 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fa39806be60 0x7fa39806e320 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa39c006b10 tx=0x7fa39c006aa0 comp rx=0 tx=0).stop 2024-04-24T23:35:14.217 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.216+0000 7fa3b2db0640 1 -- 172.21.15.157:0/3493575501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3ac1593b0 msgr2=0x7fa3ac1eb8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:14.217 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.216+0000 7fa3b2db0640 1 --2- 172.21.15.157:0/3493575501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3ac1593b0 0x7fa3ac1eb8d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0011940 tx=0x7fa3a0011e00 comp rx=0 tx=0).stop 2024-04-24T23:35:14.218 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.216+0000 7fa3b2db0640 1 -- 172.21.15.157:0/3493575501 shutdown_connections 2024-04-24T23:35:14.218 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.216+0000 7fa3b2db0640 1 --2- 172.21.15.157:0/3493575501 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fa39806be60 0x7fa39806e320 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:14.218 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.216+0000 7fa3b2db0640 1 --2- 172.21.15.157:0/3493575501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3ac156ea0 0x7fa3ac153620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:14.218 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.216+0000 7fa3b2db0640 1 --2- 172.21.15.157:0/3493575501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3ac1593b0 0x7fa3ac1eb8d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:14.218 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.216+0000 7fa3b2db0640 1 --2- 172.21.15.157:0/3493575501 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa3ac156ac0 0x7fa3ac152590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:14.218 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.216+0000 7fa3b2db0640 1 -- 172.21.15.157:0/3493575501 >> 172.21.15.157:0/3493575501 conn(0x7fa3ac0a9730 msgr2=0x7fa3ac0c57a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:14.218 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.216+0000 7fa3b2db0640 1 -- 172.21.15.157:0/3493575501 shutdown_connections 2024-04-24T23:35:14.218 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.216+0000 7fa3b2db0640 1 -- 172.21.15.157:0/3493575501 wait complete. 2024-04-24T23:35:14.228 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-24T23:35:14.328 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.328+0000 7fa08ecda640 1 Processor -- start 2024-04-24T23:35:14.328 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.328+0000 7fa08ecda640 1 -- start start 2024-04-24T23:35:14.329 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.328+0000 7fa08ecda640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa0880b03e0 0x7fa0881567e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:14.330 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.328+0000 7fa08ecda640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0881592f0 0x7fa088155c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:14.330 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.328+0000 7fa08ecda640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa0880c55d0 0x7fa0880c5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:14.330 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.328+0000 7fa08ecda640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fa0880b07c0 con 0x7fa0880b03e0 2024-04-24T23:35:14.330 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.328+0000 7fa08ecda640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fa0880cb330 con 0x7fa0880c55d0 2024-04-24T23:35:14.330 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.328+0000 7fa08ecda640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa0880cb610 con 0x7fa0881592f0 2024-04-24T23:35:14.331 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.328+0000 7fa08ca76640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa0880b03e0 0x7fa0881567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:14.331 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.328+0000 7fa08ca76640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa0880b03e0 0x7fa0881567e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42432/0 (socket says 172.21.15.157:42432) 2024-04-24T23:35:14.331 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.329+0000 7fa08ca76640 1 -- 172.21.15.157:0/2995674565 learned_addr learned my addr 172.21.15.157:0/2995674565 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:14.331 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.329+0000 7fa08d277640 1 --2- 172.21.15.157:0/2995674565 >> v2:172.21.15.169:3300/0 conn(0x7fa0880c55d0 0x7fa0880c5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:14.331 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.329+0000 7fa087fff640 1 --2- 172.21.15.157:0/2995674565 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0881592f0 0x7fa088155c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:14.331 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.329+0000 7fa08ca76640 1 -- 172.21.15.157:0/2995674565 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0881592f0 msgr2=0x7fa088155c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:14.331 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.329+0000 7fa08ca76640 1 --2- 172.21.15.157:0/2995674565 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0881592f0 0x7fa088155c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:14.331 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.329+0000 7fa08ca76640 1 -- 172.21.15.157:0/2995674565 >> v2:172.21.15.169:3300/0 conn(0x7fa0880c55d0 msgr2=0x7fa0880c5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:14.331 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.329+0000 7fa08ca76640 1 --2- 172.21.15.157:0/2995674565 >> v2:172.21.15.169:3300/0 conn(0x7fa0880c55d0 0x7fa0880c5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:14.331 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.329+0000 7fa08ca76640 1 -- 172.21.15.157:0/2995674565 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0880cb8f0 con 0x7fa0880b03e0 2024-04-24T23:35:14.331 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.329+0000 7fa087fff640 1 --2- 172.21.15.157:0/2995674565 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0881592f0 0x7fa088155c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:35:14.334 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.329+0000 7fa08ca76640 1 --2- 172.21.15.157:0/2995674565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0880b03e0 0x7fa0881567e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa08806cee0 tx=0x7fa07c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53bafa7ae23ab50a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:14.335 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.330+0000 7fa0877fe640 1 -- 172.21.15.157:0/2995674565 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07c037070 con 0x7fa0880b03e0 2024-04-24T23:35:14.335 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.330+0000 7fa0877fe640 1 -- 172.21.15.157:0/2995674565 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa07c032440 con 0x7fa0880b03e0 2024-04-24T23:35:14.335 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.330+0000 7fa0877fe640 1 -- 172.21.15.157:0/2995674565 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07c032810 con 0x7fa0880b03e0 2024-04-24T23:35:14.335 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.330+0000 7fa08ecda640 1 -- 172.21.15.157:0/2995674565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0880b03e0 msgr2=0x7fa0881567e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:14.335 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.330+0000 7fa08ecda640 1 --2- 172.21.15.157:0/2995674565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0880b03e0 0x7fa0881567e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa08806cee0 tx=0x7fa07c02a140 comp rx=0 tx=0).stop 2024-04-24T23:35:14.335 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.330+0000 7fa08ecda640 1 -- 172.21.15.157:0/2995674565 shutdown_connections 2024-04-24T23:35:14.335 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.330+0000 7fa08ecda640 1 --2- 172.21.15.157:0/2995674565 >> v2:172.21.15.169:3300/0 conn(0x7fa0880c55d0 0x7fa0880c5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:14.335 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.330+0000 7fa08ecda640 1 --2- 172.21.15.157:0/2995674565 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0881592f0 0x7fa088155c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:14.335 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.330+0000 7fa08ecda640 1 --2- 172.21.15.157:0/2995674565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0880b03e0 0x7fa0881567e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:14.335 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.330+0000 7fa08ecda640 1 -- 172.21.15.157:0/2995674565 >> 172.21.15.157:0/2995674565 conn(0x7fa088067080 msgr2=0x7fa088067490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:14.335 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.330+0000 7fa08ecda640 1 -- 172.21.15.157:0/2995674565 shutdown_connections 2024-04-24T23:35:14.335 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.330+0000 7fa08ecda640 1 -- 172.21.15.157:0/2995674565 wait complete. 2024-04-24T23:35:14.336 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.331+0000 7fa08ecda640 1 Processor -- start 2024-04-24T23:35:14.336 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.331+0000 7fa08ecda640 1 -- start start 2024-04-24T23:35:14.336 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.332+0000 7fa08ecda640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0880b03e0 0x7fa0880c1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:14.336 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.332+0000 7fa08ecda640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0880c55d0 0x7fa0880ada90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:14.336 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.332+0000 7fa08ecda640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0881592f0 0x7fa088150e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:14.336 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.332+0000 7fa08ecda640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa0880cafc0 con 0x7fa0881592f0 2024-04-24T23:35:14.336 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.332+0000 7fa08ecda640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa0881556e0 con 0x7fa0880c55d0 2024-04-24T23:35:14.336 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.332+0000 7fa08ecda640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa088159750 con 0x7fa0880b03e0 2024-04-24T23:35:14.336 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.332+0000 7fa08d277640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0881592f0 0x7fa088150e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:14.336 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.332+0000 7fa08d277640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0881592f0 0x7fa088150e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42442/0 (socket says 172.21.15.157:42442) 2024-04-24T23:35:14.336 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.332+0000 7fa08d277640 1 -- 172.21.15.157:0/3460616314 learned_addr learned my addr 172.21.15.157:0/3460616314 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:14.336 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.332+0000 7fa08ca76640 1 --2- 172.21.15.157:0/3460616314 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0880b03e0 0x7fa0880c1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:14.336 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.332+0000 7fa087fff640 1 --2- 172.21.15.157:0/3460616314 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0880c55d0 0x7fa0880ada90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:14.337 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.333+0000 7fa08d277640 1 -- 172.21.15.157:0/3460616314 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0880b03e0 msgr2=0x7fa0880c1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:14.337 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.333+0000 7fa08d277640 1 --2- 172.21.15.157:0/3460616314 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0880b03e0 0x7fa0880c1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:14.337 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.333+0000 7fa08d277640 1 -- 172.21.15.157:0/3460616314 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0880c55d0 msgr2=0x7fa0880ada90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:14.337 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.333+0000 7fa08d277640 1 --2- 172.21.15.157:0/3460616314 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0880c55d0 0x7fa0880ada90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:14.337 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.333+0000 7fa08d277640 1 -- 172.21.15.157:0/3460616314 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa08814cbe0 con 0x7fa0881592f0 2024-04-24T23:35:14.337 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.333+0000 7fa08d277640 1 --2- 172.21.15.157:0/3460616314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0881592f0 0x7fa088150e50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa08000efc0 tx=0x7fa08000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:14.337 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.333+0000 7fa0857fa640 1 -- 172.21.15.157:0/3460616314 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa080018020 con 0x7fa0881592f0 2024-04-24T23:35:14.337 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.333+0000 7fa0857fa640 1 -- 172.21.15.157:0/3460616314 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa080014480 con 0x7fa0881592f0 2024-04-24T23:35:14.337 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.333+0000 7fa0857fa640 1 -- 172.21.15.157:0/3460616314 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa080014870 con 0x7fa0881592f0 2024-04-24T23:35:14.337 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.333+0000 7fa08ecda640 1 -- 172.21.15.157:0/3460616314 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa08814ce50 con 0x7fa0881592f0 2024-04-24T23:35:14.337 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.333+0000 7fa08ecda640 1 -- 172.21.15.157:0/3460616314 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0880af8e0 con 0x7fa0881592f0 2024-04-24T23:35:14.337 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.334+0000 7fa08ecda640 1 -- 172.21.15.157:0/3460616314 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0880681c0 con 0x7fa0881592f0 2024-04-24T23:35:14.337 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.334+0000 7fa0857fa640 1 -- 172.21.15.157:0/3460616314 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa080005d10 con 0x7fa0881592f0 2024-04-24T23:35:14.338 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.335+0000 7fa0857fa640 1 --2- 172.21.15.157:0/3460616314 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fa06c06be30 0x7fa06c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:14.338 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.335+0000 7fa0857fa640 1 -- 172.21.15.157:0/3460616314 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa08008bee0 con 0x7fa0881592f0 2024-04-24T23:35:14.338 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.335+0000 7fa08ca76640 1 --2- 172.21.15.157:0/3460616314 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fa06c06be30 0x7fa06c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:14.338 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.335+0000 7fa08ca76640 1 --2- 172.21.15.157:0/3460616314 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fa06c06be30 0x7fa06c06e2f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa07c034000 tx=0x7fa07c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:14.339 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.339+0000 7fa0857fa640 1 -- 172.21.15.157:0/3460616314 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa080022030 con 0x7fa0881592f0 2024-04-24T23:35:14.555 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:14.554+0000 7fa08ecda640 1 -- 172.21.15.157:0/3460616314 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fa08814d020 con 0x7fa0881592f0 2024-04-24T23:35:15.448 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.448+0000 7fa0857fa640 1 -- 172.21.15.157:0/3460616314 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) ==== 118+0+0 (secure 0 0 0) 0x7fa080010040 con 0x7fa0881592f0 2024-04-24T23:35:15.452 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.451+0000 7fa08ecda640 1 -- 172.21.15.157:0/3460616314 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fa06c06be30 msgr2=0x7fa06c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:15.452 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.451+0000 7fa08ecda640 1 --2- 172.21.15.157:0/3460616314 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fa06c06be30 0x7fa06c06e2f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa07c034000 tx=0x7fa07c02e000 comp rx=0 tx=0).stop 2024-04-24T23:35:15.452 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.451+0000 7fa08ecda640 1 -- 172.21.15.157:0/3460616314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0881592f0 msgr2=0x7fa088150e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:15.452 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.451+0000 7fa08ecda640 1 --2- 172.21.15.157:0/3460616314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0881592f0 0x7fa088150e50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa08000efc0 tx=0x7fa08000c490 comp rx=0 tx=0).stop 2024-04-24T23:35:15.452 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.451+0000 7fa08ecda640 1 -- 172.21.15.157:0/3460616314 shutdown_connections 2024-04-24T23:35:15.452 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.451+0000 7fa08ecda640 1 --2- 172.21.15.157:0/3460616314 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fa06c06be30 0x7fa06c06e2f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:15.452 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.451+0000 7fa08ecda640 1 --2- 172.21.15.157:0/3460616314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0881592f0 0x7fa088150e50 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:15.452 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.451+0000 7fa08ecda640 1 --2- 172.21.15.157:0/3460616314 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0880c55d0 0x7fa0880ada90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:15.452 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.451+0000 7fa08ecda640 1 --2- 172.21.15.157:0/3460616314 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0880b03e0 0x7fa0880c1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:15.453 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.451+0000 7fa08ecda640 1 -- 172.21.15.157:0/3460616314 >> 172.21.15.157:0/3460616314 conn(0x7fa088067080 msgr2=0x7fa0880c63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:15.453 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.451+0000 7fa08ecda640 1 -- 172.21.15.157:0/3460616314 shutdown_connections 2024-04-24T23:35:15.453 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.451+0000 7fa08ecda640 1 -- 172.21.15.157:0/3460616314 wait complete. 2024-04-24T23:35:15.464 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-04-24T23:35:15.464 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-24T23:35:15.566 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.565+0000 7fb70efee640 1 Processor -- start 2024-04-24T23:35:15.566 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.565+0000 7fb70efee640 1 -- start start 2024-04-24T23:35:15.567 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.566+0000 7fb70efee640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb7081572e0 0x7fb708153120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:15.567 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.566+0000 7fb70efee640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7080c40f0 con 0x7fb708153660 2024-04-24T23:35:15.567 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.566+0000 7fb70efee640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7080c77a0 con 0x7fb7080c3d10 2024-04-24T23:35:15.567 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.566+0000 7fb70efee640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fb708150ba0 con 0x7fb7081572e0 2024-04-24T23:35:15.568 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.566+0000 7fb707fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb7081572e0 0x7fb708153120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:15.568 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.566+0000 7fb70d58b640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fb708153660 0x7fb70806f7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:47582/0 (socket says 172.21.15.157:47582) 2024-04-24T23:35:15.568 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.566+0000 7fb70d58b640 1 -- 172.21.15.157:0/899162619 learned_addr learned my addr 172.21.15.157:0/899162619 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-24T23:35:15.568 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.567+0000 7fb707fff640 1 -- 172.21.15.157:0/899162619 >> v1:172.21.15.169:6789/0 conn(0x7fb7080c3d10 legacy=0x7fb7080c6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:15.568 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.567+0000 7fb707fff640 1 -- 172.21.15.157:0/899162619 >> v1:172.21.15.157:6789/0 conn(0x7fb708153660 legacy=0x7fb70806f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:15.568 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.567+0000 7fb707fff640 1 -- 172.21.15.157:0/899162619 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7080c7230 con 0x7fb7081572e0 2024-04-24T23:35:15.568 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.567+0000 7fb7077fe640 1 -- 172.21.15.157:0/899162619 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1965234130 0 0) 0x7fb700001490 con 0x7fb708153660 2024-04-24T23:35:15.569 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.567+0000 7fb707fff640 1 --2- 172.21.15.157:0/899162619 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7081572e0 0x7fb708153120 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc0097a0 tx=0x7fb6fc02a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90c6fc6c2cb29b44 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:15.570 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.567+0000 7fb7077fe640 1 -- 172.21.15.157:0/899162619 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc037070 con 0x7fb7081572e0 2024-04-24T23:35:15.570 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.567+0000 7fb7077fe640 1 -- 172.21.15.157:0/899162619 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6fc02adc0 con 0x7fb7081572e0 2024-04-24T23:35:15.570 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.567+0000 7fb7077fe640 1 -- 172.21.15.157:0/899162619 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc032660 con 0x7fb7081572e0 2024-04-24T23:35:15.570 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.568+0000 7fb70efee640 1 -- 172.21.15.157:0/899162619 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7081572e0 msgr2=0x7fb708153120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:15.570 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.568+0000 7fb70efee640 1 --2- 172.21.15.157:0/899162619 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7081572e0 0x7fb708153120 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc0097a0 tx=0x7fb6fc02a0b0 comp rx=0 tx=0).stop 2024-04-24T23:35:15.570 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.568+0000 7fb70efee640 1 -- 172.21.15.157:0/899162619 shutdown_connections 2024-04-24T23:35:15.570 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.568+0000 7fb70efee640 1 --2- 172.21.15.157:0/899162619 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7081572e0 0x7fb708153120 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:15.570 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.568+0000 7fb70efee640 1 -- 172.21.15.157:0/899162619 >> 172.21.15.157:0/899162619 conn(0x7fb7080c7990 msgr2=0x7fb7080c7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:15.570 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.568+0000 7fb70efee640 1 -- 172.21.15.157:0/899162619 shutdown_connections 2024-04-24T23:35:15.570 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.568+0000 7fb70efee640 1 -- 172.21.15.157:0/899162619 wait complete. 2024-04-24T23:35:15.570 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.569+0000 7fb70efee640 1 Processor -- start 2024-04-24T23:35:15.570 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.569+0000 7fb70efee640 1 -- start start 2024-04-24T23:35:15.573 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.570+0000 7fb70efee640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb7080c3d10 0x7fb7081dcb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:15.573 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.570+0000 7fb70efee640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb708153660 0x7fb7080ad090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:15.573 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.570+0000 7fb70efee640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7081572e0 0x7fb708151140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:15.573 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.570+0000 7fb70efee640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb708156ae0 con 0x7fb7081572e0 2024-04-24T23:35:15.573 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.570+0000 7fb70efee640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb708156840 con 0x7fb708153660 2024-04-24T23:35:15.573 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.570+0000 7fb70efee640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb708156dc0 con 0x7fb7080c3d10 2024-04-24T23:35:15.573 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.570+0000 7fb70d58b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7081572e0 0x7fb708151140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:15.574 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.570+0000 7fb70d58b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7081572e0 0x7fb708151140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42452/0 (socket says 172.21.15.157:42452) 2024-04-24T23:35:15.574 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.570+0000 7fb70d58b640 1 -- 172.21.15.157:0/3462507539 learned_addr learned my addr 172.21.15.157:0/3462507539 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:15.574 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.570+0000 7fb707fff640 1 --2- 172.21.15.157:0/3462507539 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb708153660 0x7fb7080ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:15.574 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.570+0000 7fb70cd8a640 1 --2- 172.21.15.157:0/3462507539 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb7080c3d10 0x7fb7081dcb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:15.574 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.570+0000 7fb70d58b640 1 -- 172.21.15.157:0/3462507539 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb7080c3d10 msgr2=0x7fb7081dcb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:15.574 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.570+0000 7fb70d58b640 1 --2- 172.21.15.157:0/3462507539 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb7080c3d10 0x7fb7081dcb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:15.574 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.570+0000 7fb70d58b640 1 -- 172.21.15.157:0/3462507539 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb708153660 msgr2=0x7fb7080ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:15.574 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.570+0000 7fb70d58b640 1 --2- 172.21.15.157:0/3462507539 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb708153660 0x7fb7080ad090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:15.574 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.570+0000 7fb70d58b640 1 -- 172.21.15.157:0/3462507539 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7081dd760 con 0x7fb7081572e0 2024-04-24T23:35:15.574 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.570+0000 7fb70d58b640 1 --2- 172.21.15.157:0/3462507539 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7081572e0 0x7fb708151140 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb70000c7d0 tx=0x7fb70000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:15.574 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.571+0000 7fb7057fa640 1 -- 172.21.15.157:0/3462507539 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb700018070 con 0x7fb7081572e0 2024-04-24T23:35:15.574 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.571+0000 7fb7057fa640 1 -- 172.21.15.157:0/3462507539 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb70000a640 con 0x7fb7081572e0 2024-04-24T23:35:15.575 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.571+0000 7fb70efee640 1 -- 172.21.15.157:0/3462507539 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7081eb960 con 0x7fb7081572e0 2024-04-24T23:35:15.575 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.571+0000 7fb7057fa640 1 -- 172.21.15.157:0/3462507539 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70000aa30 con 0x7fb7081572e0 2024-04-24T23:35:15.575 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.571+0000 7fb70efee640 1 -- 172.21.15.157:0/3462507539 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7081dd050 con 0x7fb7081572e0 2024-04-24T23:35:15.575 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.572+0000 7fb70efee640 1 -- 172.21.15.157:0/3462507539 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7080688d0 con 0x7fb7081572e0 2024-04-24T23:35:15.575 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.572+0000 7fb7057fa640 1 -- 172.21.15.157:0/3462507539 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb70001b070 con 0x7fb7081572e0 2024-04-24T23:35:15.575 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.572+0000 7fb7057fa640 1 --2- 172.21.15.157:0/3462507539 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fb6f406be60 0x7fb6f406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:15.575 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.572+0000 7fb7057fa640 1 -- 172.21.15.157:0/3462507539 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fb70008a710 con 0x7fb7081572e0 2024-04-24T23:35:15.575 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.573+0000 7fb70cd8a640 1 --2- 172.21.15.157:0/3462507539 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fb6f406be60 0x7fb6f406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:15.575 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.573+0000 7fb70cd8a640 1 --2- 172.21.15.157:0/3462507539 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fb6f406be60 0x7fb6f406e320 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb6f8003d90 tx=0x7fb6f8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:15.577 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.577+0000 7fb7057fa640 1 -- 172.21.15.157:0/3462507539 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb7080c40f0 con 0x7fb7081572e0 2024-04-24T23:35:15.787 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:15.787+0000 7fb70efee640 1 -- 172.21.15.157:0/3462507539 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fb7080716f0 con 0x7fb7081572e0 2024-04-24T23:35:16.453 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.452+0000 7fb7057fa640 1 -- 172.21.15.157:0/3462507539 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) ==== 122+0+0 (secure 0 0 0) 0x7fb700058e10 con 0x7fb7081572e0 2024-04-24T23:35:16.456 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.455+0000 7fb70efee640 1 -- 172.21.15.157:0/3462507539 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fb6f406be60 msgr2=0x7fb6f406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:16.456 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.455+0000 7fb70efee640 1 --2- 172.21.15.157:0/3462507539 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fb6f406be60 0x7fb6f406e320 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb6f8003d90 tx=0x7fb6f8009040 comp rx=0 tx=0).stop 2024-04-24T23:35:16.456 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.455+0000 7fb70efee640 1 -- 172.21.15.157:0/3462507539 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7081572e0 msgr2=0x7fb708151140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:16.456 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.455+0000 7fb70efee640 1 --2- 172.21.15.157:0/3462507539 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7081572e0 0x7fb708151140 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb70000c7d0 tx=0x7fb70000cc90 comp rx=0 tx=0).stop 2024-04-24T23:35:16.456 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.455+0000 7fb70efee640 1 -- 172.21.15.157:0/3462507539 shutdown_connections 2024-04-24T23:35:16.456 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.455+0000 7fb70efee640 1 --2- 172.21.15.157:0/3462507539 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fb6f406be60 0x7fb6f406e320 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:16.456 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.455+0000 7fb70efee640 1 --2- 172.21.15.157:0/3462507539 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7081572e0 0x7fb708151140 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:16.456 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.455+0000 7fb70efee640 1 --2- 172.21.15.157:0/3462507539 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb708153660 0x7fb7080ad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:16.456 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.455+0000 7fb70efee640 1 --2- 172.21.15.157:0/3462507539 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb7080c3d10 0x7fb7081dcb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:16.457 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.455+0000 7fb70efee640 1 -- 172.21.15.157:0/3462507539 >> 172.21.15.157:0/3462507539 conn(0x7fb7080c7990 msgr2=0x7fb7080c6b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:16.457 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.455+0000 7fb70efee640 1 -- 172.21.15.157:0/3462507539 shutdown_connections 2024-04-24T23:35:16.457 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.455+0000 7fb70efee640 1 -- 172.21.15.157:0/3462507539 wait complete. 2024-04-24T23:35:16.468 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-24T23:35:16.468 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:35:16.568 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.568+0000 7f7e977c8640 1 Processor -- start 2024-04-24T23:35:16.569 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.568+0000 7f7e977c8640 1 -- start start 2024-04-24T23:35:16.572 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.569+0000 7f7e977c8640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e900c2110 0x7f7e900ab5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:16.572 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.569+0000 7f7e977c8640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e90155d60 con 0x7f7e9006b3d0 2024-04-24T23:35:16.573 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.569+0000 7f7e977c8640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e90156040 con 0x7f7e900c3820 2024-04-24T23:35:16.573 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.569+0000 7f7e977c8640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7e9006e030 con 0x7f7e900c2110 2024-04-24T23:35:16.573 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.569+0000 7f7e95564640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f7e9006b3d0 0x7f7e9006b7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:47586/0 (socket says 172.21.15.157:47586) 2024-04-24T23:35:16.573 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.569+0000 7f7e95564640 1 -- 172.21.15.157:0/3048753092 learned_addr learned my addr 172.21.15.157:0/3048753092 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-24T23:35:16.573 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.569+0000 7f7e94d63640 1 --2- 172.21.15.157:0/3048753092 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e900c2110 0x7f7e900ab5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:16.573 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.569+0000 7f7e87fff640 1 -- 172.21.15.157:0/3048753092 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 696527569 0 0) 0x7f7e90155d60 con 0x7f7e9006b3d0 2024-04-24T23:35:16.573 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.569+0000 7f7e87fff640 1 -- 172.21.15.157:0/3048753092 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e7c003520 con 0x7f7e9006b3d0 2024-04-24T23:35:16.573 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.569+0000 7f7e87fff640 1 -- 172.21.15.157:0/3048753092 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4130289624 0 0) 0x7f7e90156040 con 0x7f7e900c3820 2024-04-24T23:35:16.573 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.569+0000 7f7e87fff640 1 -- 172.21.15.157:0/3048753092 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e90155d60 con 0x7f7e900c3820 2024-04-24T23:35:16.573 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.569+0000 7f7e87fff640 1 -- 172.21.15.157:0/3048753092 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 361668825 0 0) 0x7f7e7c003520 con 0x7f7e9006b3d0 2024-04-24T23:35:16.573 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.569+0000 7f7e87fff640 1 -- 172.21.15.157:0/3048753092 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e900c2110 msgr2=0x7f7e900ab5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:16.573 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.569+0000 7f7e87fff640 1 --2- 172.21.15.157:0/3048753092 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e900c2110 0x7f7e900ab5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:16.574 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.569+0000 7f7e87fff640 1 -- 172.21.15.157:0/3048753092 >> v1:172.21.15.169:6789/0 conn(0x7f7e900c3820 legacy=0x7f7e900c0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:16.574 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.569+0000 7f7e87fff640 1 -- 172.21.15.157:0/3048753092 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e900ac3d0 con 0x7f7e9006b3d0 2024-04-24T23:35:16.574 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.569+0000 7f7e87fff640 1 -- 172.21.15.157:0/3048753092 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1480525340 0 0) 0x7f7e8c001670 con 0x7f7e9006b3d0 2024-04-24T23:35:16.574 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.569+0000 7f7e94d63640 1 --2- 172.21.15.157:0/3048753092 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e900c2110 0x7f7e900ab5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:35:16.574 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.569+0000 7f7e87fff640 1 -- 172.21.15.157:0/3048753092 >> v1:172.21.15.157:6789/0 conn(0x7f7e9006b3d0 legacy=0x7f7e9006b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:16.574 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.569+0000 7f7e87fff640 1 --2- 172.21.15.157:0/3048753092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e7c003b70 0x7f7e7c003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:16.574 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.570+0000 7f7e87fff640 1 --2- 172.21.15.157:0/3048753092 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e7c004800 0x7f7e7c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:16.574 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.570+0000 7f7e87fff640 1 --2- 172.21.15.157:0/3048753092 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e900c2110 0x7f7e7c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:16.574 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.570+0000 7f7e87fff640 1 -- 172.21.15.157:0/3048753092 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7e7c00b9f0 con 0x7f7e7c003b70 2024-04-24T23:35:16.574 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.570+0000 7f7e87fff640 1 -- 172.21.15.157:0/3048753092 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7e7c00bba0 con 0x7f7e7c004800 2024-04-24T23:35:16.574 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.570+0000 7f7e87fff640 1 -- 172.21.15.157:0/3048753092 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7e7c00bda0 con 0x7f7e900c2110 2024-04-24T23:35:16.574 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.570+0000 7f7e95564640 1 --2- 172.21.15.157:0/3048753092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e7c003b70 0x7f7e7c003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:16.574 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.570+0000 7f7e94d63640 1 --2- 172.21.15.157:0/3048753092 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e7c004800 0x7f7e7c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:16.575 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.570+0000 7f7e95d65640 1 --2- 172.21.15.157:0/3048753092 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e900c2110 0x7f7e7c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:16.575 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.570+0000 7f7e95564640 1 -- 172.21.15.157:0/3048753092 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e900c2110 msgr2=0x7f7e7c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:16.575 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.570+0000 7f7e95564640 1 --2- 172.21.15.157:0/3048753092 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e900c2110 0x7f7e7c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:16.575 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.570+0000 7f7e95564640 1 -- 172.21.15.157:0/3048753092 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e7c004800 msgr2=0x7f7e7c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:16.575 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.570+0000 7f7e95564640 1 --2- 172.21.15.157:0/3048753092 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e7c004800 0x7f7e7c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:16.575 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.570+0000 7f7e95564640 1 -- 172.21.15.157:0/3048753092 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e7c003520 con 0x7f7e7c003b70 2024-04-24T23:35:16.575 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.571+0000 7f7e95564640 1 --2- 172.21.15.157:0/3048753092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e7c003b70 0x7f7e7c003f70 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c001eb0 tx=0x7f7e7c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8073635e2d362279 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:16.575 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.571+0000 7f7e87fff640 1 -- 172.21.15.157:0/3048753092 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c007ce0 con 0x7f7e7c003b70 2024-04-24T23:35:16.575 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.571+0000 7f7e87fff640 1 -- 172.21.15.157:0/3048753092 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e8c005ad0 con 0x7f7e7c003b70 2024-04-24T23:35:16.575 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.571+0000 7f7e87fff640 1 -- 172.21.15.157:0/3048753092 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c007070 con 0x7f7e7c003b70 2024-04-24T23:35:16.575 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.571+0000 7f7e977c8640 1 -- 172.21.15.157:0/3048753092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e7c003b70 msgr2=0x7f7e7c003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:16.575 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.571+0000 7f7e977c8640 1 --2- 172.21.15.157:0/3048753092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e7c003b70 0x7f7e7c003f70 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c001eb0 tx=0x7f7e7c005830 comp rx=0 tx=0).stop 2024-04-24T23:35:16.575 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.571+0000 7f7e95d65640 1 -- 172.21.15.157:0/3048753092 reap_dead start 2024-04-24T23:35:16.575 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.571+0000 7f7e977c8640 1 -- 172.21.15.157:0/3048753092 shutdown_connections 2024-04-24T23:35:16.575 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.571+0000 7f7e977c8640 1 -- 172.21.15.157:0/3048753092 >> 172.21.15.157:0/3048753092 conn(0x7f7e900c71f0 msgr2=0x7f7e900c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:16.575 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.571+0000 7f7e977c8640 1 -- 172.21.15.157:0/3048753092 shutdown_connections 2024-04-24T23:35:16.576 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.571+0000 7f7e977c8640 1 -- 172.21.15.157:0/3048753092 wait complete. 2024-04-24T23:35:16.578 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.572+0000 7f7e977c8640 1 Processor -- start 2024-04-24T23:35:16.578 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.572+0000 7f7e977c8640 1 -- start start 2024-04-24T23:35:16.578 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.573+0000 7f7e977c8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e7c003b70 0x7f7e9014a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:16.579 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.573+0000 7f7e977c8640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e90144890 0x7f7e90143b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:16.579 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.573+0000 7f7e977c8640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e90144dc0 0x7f7e901440b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:16.579 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.573+0000 7f7e977c8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9006bec0 con 0x7f7e7c003b70 2024-04-24T23:35:16.579 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.573+0000 7f7e977c8640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9014f7f0 con 0x7f7e90144dc0 2024-04-24T23:35:16.579 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.573+0000 7f7e977c8640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7e900c1bf0 con 0x7f7e90144890 2024-04-24T23:35:16.579 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.573+0000 7f7e95564640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e7c003b70 0x7f7e9014a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:16.579 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.573+0000 7f7e95564640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e7c003b70 0x7f7e9014a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42464/0 (socket says 172.21.15.157:42464) 2024-04-24T23:35:16.579 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.573+0000 7f7e95564640 1 -- 172.21.15.157:0/1707870708 learned_addr learned my addr 172.21.15.157:0/1707870708 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:16.579 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.573+0000 7f7e94d63640 1 --2- 172.21.15.157:0/1707870708 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e90144890 0x7f7e90143b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:16.579 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.573+0000 7f7e95d65640 1 --2- 172.21.15.157:0/1707870708 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e90144dc0 0x7f7e901440b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:16.579 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.573+0000 7f7e95564640 1 -- 172.21.15.157:0/1707870708 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e90144890 msgr2=0x7f7e90143b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:16.579 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.573+0000 7f7e95564640 1 --2- 172.21.15.157:0/1707870708 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e90144890 0x7f7e90143b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:16.579 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.573+0000 7f7e95564640 1 -- 172.21.15.157:0/1707870708 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e90144dc0 msgr2=0x7f7e901440b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:16.579 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.573+0000 7f7e95564640 1 --2- 172.21.15.157:0/1707870708 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e90144dc0 0x7f7e901440b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:16.580 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.573+0000 7f7e95564640 1 -- 172.21.15.157:0/1707870708 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e901ee2f0 con 0x7f7e7c003b70 2024-04-24T23:35:16.580 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.573+0000 7f7e95d65640 1 --2- 172.21.15.157:0/1707870708 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e90144dc0 0x7f7e901440b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:35:16.580 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.574+0000 7f7e95564640 1 --2- 172.21.15.157:0/1707870708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e7c003b70 0x7f7e9014a250 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c0018f0 tx=0x7f7e900ac5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:16.580 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.574+0000 7f7e85ffb640 1 -- 172.21.15.157:0/1707870708 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c041030 con 0x7f7e7c003b70 2024-04-24T23:35:16.580 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.574+0000 7f7e85ffb640 1 -- 172.21.15.157:0/1707870708 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e8c038040 con 0x7f7e7c003b70 2024-04-24T23:35:16.580 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.574+0000 7f7e977c8640 1 -- 172.21.15.157:0/1707870708 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e901ee5d0 con 0x7f7e7c003b70 2024-04-24T23:35:16.580 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.574+0000 7f7e85ffb640 1 -- 172.21.15.157:0/1707870708 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c001670 con 0x7f7e7c003b70 2024-04-24T23:35:16.580 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.574+0000 7f7e977c8640 1 -- 172.21.15.157:0/1707870708 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e901eeb90 con 0x7f7e7c003b70 2024-04-24T23:35:16.580 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.575+0000 7f7e977c8640 1 -- 172.21.15.157:0/1707870708 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e900681c0 con 0x7f7e7c003b70 2024-04-24T23:35:16.580 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.575+0000 7f7e85ffb640 1 -- 172.21.15.157:0/1707870708 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7e8c044070 con 0x7f7e7c003b70 2024-04-24T23:35:16.580 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.576+0000 7f7e85ffb640 1 --2- 172.21.15.157:0/1707870708 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f7e7406be30 0x7f7e7406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:16.580 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.576+0000 7f7e85ffb640 1 -- 172.21.15.157:0/1707870708 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f7e8c0aada0 con 0x7f7e7c003b70 2024-04-24T23:35:16.580 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.576+0000 7f7e94d63640 1 --2- 172.21.15.157:0/1707870708 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f7e7406be30 0x7f7e7406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:16.580 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.577+0000 7f7e94d63640 1 --2- 172.21.15.157:0/1707870708 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f7e7406be30 0x7f7e7406e2f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7e80009560 tx=0x7f7e80008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:16.581 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.580+0000 7f7e85ffb640 1 -- 172.21.15.157:0/1707870708 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7e8c005aa0 con 0x7f7e7c003b70 2024-04-24T23:35:16.788 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.788+0000 7f7e977c8640 1 -- 172.21.15.157:0/1707870708 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7e90152c90 con 0x7f7e7c003b70 2024-04-24T23:35:16.789 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.788+0000 7f7e85ffb640 1 -- 172.21.15.157:0/1707870708 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5189 (secure 0 0 0) 0x7f7e8c0794a0 con 0x7f7e7c003b70 2024-04-24T23:35:16.789 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:35:16.789 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1276438765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1276438765},{"type":"v1","addr":"172.21.15.169:6837","nonce":1276438765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1398599086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":1398599086},{"type":"v1","addr":"172.21.15.157:6835","nonce":1398599086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4258,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3795840339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3795840339},{"type":"v1","addr":"172.21.15.157:6834","nonce":3795840339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:34:53.701085+0000","modified":"2024-04-24T23:35:15.789166+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6836/3782746080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6835","nonce":3782746080},{"type":"v1","addr":"172.21.15.169:6836","nonce":3782746080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4270,"qdb_cluster":[4270]},"id":1}]} 2024-04-24T23:35:16.789 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 9 2024-04-24T23:35:16.791 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.790+0000 7f7e977c8640 1 -- 172.21.15.157:0/1707870708 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f7e7406be30 msgr2=0x7f7e7406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:16.791 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.790+0000 7f7e977c8640 1 --2- 172.21.15.157:0/1707870708 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f7e7406be30 0x7f7e7406e2f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7e80009560 tx=0x7f7e80008040 comp rx=0 tx=0).stop 2024-04-24T23:35:16.791 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.790+0000 7f7e977c8640 1 -- 172.21.15.157:0/1707870708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e7c003b70 msgr2=0x7f7e9014a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:16.791 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.790+0000 7f7e977c8640 1 --2- 172.21.15.157:0/1707870708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e7c003b70 0x7f7e9014a250 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c0018f0 tx=0x7f7e900ac5a0 comp rx=0 tx=0).stop 2024-04-24T23:35:16.791 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.790+0000 7f7e977c8640 1 -- 172.21.15.157:0/1707870708 shutdown_connections 2024-04-24T23:35:16.791 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.790+0000 7f7e977c8640 1 --2- 172.21.15.157:0/1707870708 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f7e7406be30 0x7f7e7406e2f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:16.791 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.790+0000 7f7e977c8640 1 --2- 172.21.15.157:0/1707870708 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e90144dc0 0x7f7e901440b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:16.792 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.790+0000 7f7e977c8640 1 --2- 172.21.15.157:0/1707870708 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e90144890 0x7f7e90143b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:16.792 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.790+0000 7f7e977c8640 1 --2- 172.21.15.157:0/1707870708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e7c003b70 0x7f7e9014a250 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:16.792 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.790+0000 7f7e977c8640 1 -- 172.21.15.157:0/1707870708 >> 172.21.15.157:0/1707870708 conn(0x7f7e900c71f0 msgr2=0x7f7e90153150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:16.792 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.790+0000 7f7e977c8640 1 -- 172.21.15.157:0/1707870708 shutdown_connections 2024-04-24T23:35:16.792 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.790+0000 7f7e977c8640 1 -- 172.21.15.157:0/1707870708 wait complete. 2024-04-24T23:35:16.802 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:35:16.900 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.899+0000 7f7532fb6640 1 Processor -- start 2024-04-24T23:35:16.900 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.899+0000 7f7532fb6640 1 -- start start 2024-04-24T23:35:16.901 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.900+0000 7f7532fb6640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f752c0a7c70 0x7f752c0a8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:16.901 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.900+0000 7f7532fb6640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f752c0c3490 0x7f752c0c6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:16.901 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.900+0000 7f7532fb6640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f752c1554d0 con 0x7f752c0c1e30 2024-04-24T23:35:16.901 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.900+0000 7f7532fb6640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f752c156080 con 0x7f752c0c3490 2024-04-24T23:35:16.902 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.900+0000 7f7532fb6640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f752c155750 con 0x7f752c0a7c70 2024-04-24T23:35:16.902 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.900+0000 7f752bfff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f752c0c3490 0x7f752c0c6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:16.902 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.900+0000 7f752bfff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f752c0c3490 0x7f752c0c6b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42470/0 (socket says 172.21.15.157:42470) 2024-04-24T23:35:16.902 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.900+0000 7f7531553640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f752c0c1e30 0x7f752c0c2260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:47600/0 (socket says 172.21.15.157:47600) 2024-04-24T23:35:16.902 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.900+0000 7f752bfff640 1 -- 172.21.15.157:0/2719715516 learned_addr learned my addr 172.21.15.157:0/2719715516 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:16.902 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.900+0000 7f7530d52640 1 --2- 172.21.15.157:0/2719715516 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f752c0a7c70 0x7f752c0a8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:16.902 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.901+0000 7f752bfff640 1 -- 172.21.15.157:0/2719715516 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f752c0a7c70 msgr2=0x7f752c0a8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:16.902 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.901+0000 7f752bfff640 1 --2- 172.21.15.157:0/2719715516 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f752c0a7c70 0x7f752c0a8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:16.902 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.901+0000 7f752bfff640 1 -- 172.21.15.157:0/2719715516 >> v1:172.21.15.157:6789/0 conn(0x7f752c0c1e30 legacy=0x7f752c0c2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:16.902 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.901+0000 7f752bfff640 1 -- 172.21.15.157:0/2719715516 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f752c155a30 con 0x7f752c0c3490 2024-04-24T23:35:16.903 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.901+0000 7f752bfff640 1 --2- 172.21.15.157:0/2719715516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f752c0c3490 0x7f752c0c6b20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7520009870 tx=0x7f752002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f899a810fd6f190d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:16.904 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.901+0000 7f752b7fe640 1 -- 172.21.15.157:0/2719715516 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7520037070 con 0x7f752c0c3490 2024-04-24T23:35:16.904 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.901+0000 7f752b7fe640 1 -- 172.21.15.157:0/2719715516 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f752002ae50 con 0x7f752c0c3490 2024-04-24T23:35:16.904 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.901+0000 7f752b7fe640 1 -- 172.21.15.157:0/2719715516 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7520032860 con 0x7f752c0c3490 2024-04-24T23:35:16.904 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.902+0000 7f7532fb6640 1 -- 172.21.15.157:0/2719715516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f752c0c3490 msgr2=0x7f752c0c6b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:16.904 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.902+0000 7f7532fb6640 1 --2- 172.21.15.157:0/2719715516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f752c0c3490 0x7f752c0c6b20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7520009870 tx=0x7f752002a300 comp rx=0 tx=0).stop 2024-04-24T23:35:16.904 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.902+0000 7f7532fb6640 1 -- 172.21.15.157:0/2719715516 shutdown_connections 2024-04-24T23:35:16.904 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.902+0000 7f7532fb6640 1 --2- 172.21.15.157:0/2719715516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f752c0c3490 0x7f752c0c6b20 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:16.904 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.902+0000 7f7532fb6640 1 --2- 172.21.15.157:0/2719715516 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f752c0a7c70 0x7f752c0a8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:16.904 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.902+0000 7f7532fb6640 1 -- 172.21.15.157:0/2719715516 >> 172.21.15.157:0/2719715516 conn(0x7f752c0aa7b0 msgr2=0x7f752c0aab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:16.904 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.902+0000 7f7532fb6640 1 -- 172.21.15.157:0/2719715516 shutdown_connections 2024-04-24T23:35:16.904 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.902+0000 7f7532fb6640 1 -- 172.21.15.157:0/2719715516 wait complete. 2024-04-24T23:35:16.904 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.903+0000 7f7532fb6640 1 Processor -- start 2024-04-24T23:35:16.904 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.903+0000 7f7532fb6640 1 -- start start 2024-04-24T23:35:16.909 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.904+0000 7f7532fb6640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f752c0a7c70 0x7f752c156a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:16.909 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.904+0000 7f7532fb6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f752c0c1e30 0x7f752c0a90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:16.909 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.904+0000 7f7532fb6640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f752c0c3490 0x7f752c0a95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:16.909 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.904+0000 7f7532fb6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f752c152d60 con 0x7f752c0c1e30 2024-04-24T23:35:16.909 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.904+0000 7f7532fb6640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f752c0c2970 con 0x7f752c0a7c70 2024-04-24T23:35:16.909 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.904+0000 7f7532fb6640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f752c152ee0 con 0x7f752c0c3490 2024-04-24T23:35:16.909 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.904+0000 7f752bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f752c0c1e30 0x7f752c0a90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:16.909 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.904+0000 7f752bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f752c0c1e30 0x7f752c0a90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42486/0 (socket says 172.21.15.157:42486) 2024-04-24T23:35:16.909 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.904+0000 7f752bfff640 1 -- 172.21.15.157:0/980914253 learned_addr learned my addr 172.21.15.157:0/980914253 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:16.909 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.904+0000 7f7530d52640 1 --2- 172.21.15.157:0/980914253 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f752c0a7c70 0x7f752c156a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:16.909 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.904+0000 7f7531553640 1 --2- 172.21.15.157:0/980914253 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f752c0c3490 0x7f752c0a95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:16.910 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.904+0000 7f752bfff640 1 -- 172.21.15.157:0/980914253 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f752c0c3490 msgr2=0x7f752c0a95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:16.910 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.904+0000 7f752bfff640 1 --2- 172.21.15.157:0/980914253 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f752c0c3490 0x7f752c0a95e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:16.910 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.904+0000 7f752bfff640 1 -- 172.21.15.157:0/980914253 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f752c0a7c70 msgr2=0x7f752c156a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:16.910 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.904+0000 7f752bfff640 1 --2- 172.21.15.157:0/980914253 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f752c0a7c70 0x7f752c156a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:16.910 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.904+0000 7f752bfff640 1 -- 172.21.15.157:0/980914253 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f752c1573a0 con 0x7f752c0c1e30 2024-04-24T23:35:16.910 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.904+0000 7f752bfff640 1 --2- 172.21.15.157:0/980914253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f752c0c1e30 0x7f752c0a90a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f75200090b0 tx=0x7f75200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:16.910 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.904+0000 7f75297fa640 1 -- 172.21.15.157:0/980914253 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7520041020 con 0x7f752c0c1e30 2024-04-24T23:35:16.910 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.905+0000 7f75297fa640 1 -- 172.21.15.157:0/980914253 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f752003c870 con 0x7f752c0c1e30 2024-04-24T23:35:16.910 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.905+0000 7f7532fb6640 1 -- 172.21.15.157:0/980914253 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f752c1debf0 con 0x7f752c0c1e30 2024-04-24T23:35:16.910 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.905+0000 7f75297fa640 1 -- 172.21.15.157:0/980914253 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7520037040 con 0x7f752c0c1e30 2024-04-24T23:35:16.910 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.905+0000 7f7532fb6640 1 -- 172.21.15.157:0/980914253 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f752c1549b0 con 0x7f752c0c1e30 2024-04-24T23:35:16.910 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.905+0000 7f7532fb6640 1 -- 172.21.15.157:0/980914253 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f752c1eb7e0 con 0x7f752c0c1e30 2024-04-24T23:35:16.910 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.906+0000 7f75297fa640 1 -- 172.21.15.157:0/980914253 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7520044020 con 0x7f752c0c1e30 2024-04-24T23:35:16.911 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.906+0000 7f75297fa640 1 --2- 172.21.15.157:0/980914253 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f751806be30 0x7f751806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:16.911 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.907+0000 7f75297fa640 1 -- 172.21.15.157:0/980914253 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f75200abaf0 con 0x7f752c0c1e30 2024-04-24T23:35:16.911 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.907+0000 7f7530d52640 1 --2- 172.21.15.157:0/980914253 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f751806be30 0x7f751806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:16.911 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.907+0000 7f7530d52640 1 --2- 172.21.15.157:0/980914253 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f751806be30 0x7f751806e2f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f751c006fd0 tx=0x7f751c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:16.911 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:16.911+0000 7f75297fa640 1 -- 172.21.15.157:0/980914253 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f752004b030 con 0x7f752c0c1e30 2024-04-24T23:35:17.088 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.087+0000 7f7532fb6640 1 -- 172.21.15.157:0/980914253 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f752c1eba00 con 0x7f752c0c1e30 2024-04-24T23:35:17.089 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.088+0000 7f75297fa640 1 -- 172.21.15.157:0/980914253 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18538 (secure 0 0 0) 0x7f752007a1f0 con 0x7f752c0c1e30 2024-04-24T23:35:17.089 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:35:17.090 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":25,"fsid":"223685fb-5b64-46a5-9857-3791c9c07df2","created":"2024-04-24T23:34:37.504202+0000","modified":"2024-04-24T23:34:58.744500+0000","last_up_change":"2024-04-24T23:34:47.584963+0000","last_in_change":"2024-04-24T23:34:39.632954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:34:47.796655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:34:52.177208+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:34:52.667022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:34:56.395517+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"95d14ac8-fccf-4261-be8e-edafa746f629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6809","nonce":4003822160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6811","nonce":4003822160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6815","nonce":4003822160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6813","nonce":4003822160}]},"public_addr":"172.21.15.157:6809/4003822160","cluster_addr":"172.21.15.157:6811/4003822160","heartbeat_back_addr":"172.21.15.157:6815/4003822160","heartbeat_front_addr":"172.21.15.157:6813/4003822160","state":["exists","up"]},{"osd":1,"uuid":"7be79da9-8eec-4990-9178-71d602e39d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6817","nonce":1557575576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6819","nonce":1557575576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6823","nonce":1557575576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6821","nonce":1557575576}]},"public_addr":"172.21.15.157:6817/1557575576","cluster_addr":"172.21.15.157:6819/1557575576","heartbeat_back_addr":"172.21.15.157:6823/1557575576","heartbeat_front_addr":"172.21.15.157:6821/1557575576","state":["exists","up"]},{"osd":2,"uuid":"0127ed00-b7c3-466d-bfb0-01b0fd315f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6825","nonce":2793948092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6827","nonce":2793948092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6831","nonce":2793948092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6829","nonce":2793948092}]},"public_addr":"172.21.15.157:6825/2793948092","cluster_addr":"172.21.15.157:6827/2793948092","heartbeat_back_addr":"172.21.15.157:6831/2793948092","heartbeat_front_addr":"172.21.15.157:6829/2793948092","state":["exists","up"]},{"osd":3,"uuid":"f2331acc-aa37-433e-868e-3561b6039562","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6801","nonce":422758549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6803","nonce":422758549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6807","nonce":422758549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6805","nonce":422758549}]},"public_addr":"172.21.15.157:6801/422758549","cluster_addr":"172.21.15.157:6803/422758549","heartbeat_back_addr":"172.21.15.157:6807/422758549","heartbeat_front_addr":"172.21.15.157:6805/422758549","state":["exists","up"]},{"osd":4,"uuid":"1998854b-7398-43f5-ab7d-79fa0debcf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6801","nonce":321910099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6803","nonce":321910099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6807","nonce":321910099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6805","nonce":321910099}]},"public_addr":"172.21.15.169:6801/321910099","cluster_addr":"172.21.15.169:6803/321910099","heartbeat_back_addr":"172.21.15.169:6807/321910099","heartbeat_front_addr":"172.21.15.169:6805/321910099","state":["exists","up"]},{"osd":5,"uuid":"c37153ff-9b2b-48b9-a684-2c82b6f39f15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6825","nonce":243321711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6827","nonce":243321711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6831","nonce":243321711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6829","nonce":243321711}]},"public_addr":"172.21.15.169:6825/243321711","cluster_addr":"172.21.15.169:6827/243321711","heartbeat_back_addr":"172.21.15.169:6831/243321711","heartbeat_front_addr":"172.21.15.169:6829/243321711","state":["exists","up"]},{"osd":6,"uuid":"8db79e1c-f36b-49cb-8d8a-a20c2aec57c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6809","nonce":3441420785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6811","nonce":3441420785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6815","nonce":3441420785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6813","nonce":3441420785}]},"public_addr":"172.21.15.169:6809/3441420785","cluster_addr":"172.21.15.169:6811/3441420785","heartbeat_back_addr":"172.21.15.169:6815/3441420785","heartbeat_front_addr":"172.21.15.169:6813/3441420785","state":["exists","up"]},{"osd":7,"uuid":"928586e9-296c-4b8b-9b27-42f9f155dbdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6817","nonce":1663179020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6819","nonce":1663179020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6823","nonce":1663179020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6821","nonce":1663179020}]},"public_addr":"172.21.15.169:6817/1663179020","cluster_addr":"172.21.15.169:6819/1663179020","heartbeat_back_addr":"172.21.15.169:6823/1663179020","heartbeat_front_addr":"172.21.15.169:6821/1663179020","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.848039+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.826489+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.801120+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.727706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.824343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.791915+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.878073+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.810831+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:35:17.091 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.091+0000 7f7532fb6640 1 -- 172.21.15.157:0/980914253 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f751806be30 msgr2=0x7f751806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:17.091 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.091+0000 7f7532fb6640 1 --2- 172.21.15.157:0/980914253 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f751806be30 0x7f751806e2f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f751c006fd0 tx=0x7f751c008040 comp rx=0 tx=0).stop 2024-04-24T23:35:17.092 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.091+0000 7f7532fb6640 1 -- 172.21.15.157:0/980914253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f752c0c1e30 msgr2=0x7f752c0a90a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:17.092 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.091+0000 7f7532fb6640 1 --2- 172.21.15.157:0/980914253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f752c0c1e30 0x7f752c0a90a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f75200090b0 tx=0x7f75200046c0 comp rx=0 tx=0).stop 2024-04-24T23:35:17.092 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.091+0000 7f7532fb6640 1 -- 172.21.15.157:0/980914253 shutdown_connections 2024-04-24T23:35:17.092 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.091+0000 7f7532fb6640 1 --2- 172.21.15.157:0/980914253 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f751806be30 0x7f751806e2f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:17.092 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.091+0000 7f7532fb6640 1 --2- 172.21.15.157:0/980914253 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f752c0c3490 0x7f752c0a95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:17.092 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.091+0000 7f7532fb6640 1 --2- 172.21.15.157:0/980914253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f752c0c1e30 0x7f752c0a90a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:17.092 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.091+0000 7f7532fb6640 1 --2- 172.21.15.157:0/980914253 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f752c0a7c70 0x7f752c156a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:17.092 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.091+0000 7f7532fb6640 1 -- 172.21.15.157:0/980914253 >> 172.21.15.157:0/980914253 conn(0x7f752c0aa7b0 msgr2=0x7f752c0c3af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:17.092 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.091+0000 7f7532fb6640 1 -- 172.21.15.157:0/980914253 shutdown_connections 2024-04-24T23:35:17.092 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.091+0000 7f7532fb6640 1 -- 172.21.15.157:0/980914253 wait complete. 2024-04-24T23:35:17.103 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-24T23:35:17.103 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:35:17.199 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.198+0000 7f08eb09e640 1 Processor -- start 2024-04-24T23:35:17.199 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.198+0000 7f08eb09e640 1 -- start start 2024-04-24T23:35:17.200 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.199+0000 7f08eb09e640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08e4152b40 0x7f08e40c5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:17.201 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.199+0000 7f08eb09e640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f08e40c5b70 0x7f08e4155a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:17.201 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.199+0000 7f08eb09e640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08e40c8bb0 con 0x7f08e40be360 2024-04-24T23:35:17.201 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.199+0000 7f08eb09e640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f08e40c8e90 con 0x7f08e40c5b70 2024-04-24T23:35:17.201 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.199+0000 7f08eb09e640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f08e4159480 con 0x7f08e4152b40 2024-04-24T23:35:17.201 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.199+0000 7f08e963b640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f08e40c5b70 0x7f08e4155a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:17.201 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.199+0000 7f08e8e3a640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08e4152b40 0x7f08e40c5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:17.201 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.200+0000 7f08e3fff640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f08e40be360 0x7f08e4155350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.157:42484/0 (socket says 172.21.15.157:42484) 2024-04-24T23:35:17.201 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.200+0000 7f08e8e3a640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08e4152b40 0x7f08e40c5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.157:52582/0 (socket says 172.21.15.157:52582) 2024-04-24T23:35:17.201 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.200+0000 7f08e3fff640 1 -- 172.21.15.157:0/3987591657 learned_addr learned my addr 172.21.15.157:0/3987591657 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-24T23:35:17.201 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.200+0000 7f08e963b640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f08e40c5b70 0x7f08e4155a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.157:44772/0 (socket says 172.21.15.157:44772) 2024-04-24T23:35:17.201 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.200+0000 7f08e963b640 1 -- 172.21.15.157:0/3987591657 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08e4152b40 msgr2=0x7f08e40c5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:17.201 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.200+0000 7f08e963b640 1 --2- 172.21.15.157:0/3987591657 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08e4152b40 0x7f08e40c5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:17.201 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.200+0000 7f08e963b640 1 -- 172.21.15.157:0/3987591657 >> v1:172.21.15.169:6789/0 conn(0x7f08e40be360 legacy=0x7f08e4155350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:17.201 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.200+0000 7f08e963b640 1 -- 172.21.15.157:0/3987591657 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08e4159760 con 0x7f08e40c5b70 2024-04-24T23:35:17.202 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.200+0000 7f08e8e3a640 1 --2- 172.21.15.157:0/3987591657 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08e4152b40 0x7f08e40c5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:35:17.202 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.200+0000 7f08e37fe640 1 -- 172.21.15.157:0/3987591657 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3563253349 0 0) 0x7f08e40c8bb0 con 0x7f08e40be360 2024-04-24T23:35:17.203 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.201+0000 7f08e963b640 1 --2- 172.21.15.157:0/3987591657 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08e40c5b70 0x7f08e4155a60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f08dc009d90 tx=0x7f08dc02a530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0389aa12781824 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:17.203 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.201+0000 7f08e37fe640 1 -- 172.21.15.157:0/3987591657 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08dc007740 con 0x7f08e40c5b70 2024-04-24T23:35:17.203 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.201+0000 7f08e37fe640 1 -- 172.21.15.157:0/3987591657 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08dc0078e0 con 0x7f08e40c5b70 2024-04-24T23:35:17.203 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.201+0000 7f08e37fe640 1 -- 172.21.15.157:0/3987591657 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08dc007cb0 con 0x7f08e40c5b70 2024-04-24T23:35:17.203 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.201+0000 7f08eb09e640 1 -- 172.21.15.157:0/3987591657 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08e40c5b70 msgr2=0x7f08e4155a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:17.203 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.201+0000 7f08eb09e640 1 --2- 172.21.15.157:0/3987591657 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08e40c5b70 0x7f08e4155a60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f08dc009d90 tx=0x7f08dc02a530 comp rx=0 tx=0).stop 2024-04-24T23:35:17.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.201+0000 7f08eb09e640 1 -- 172.21.15.157:0/3987591657 shutdown_connections 2024-04-24T23:35:17.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.201+0000 7f08eb09e640 1 --2- 172.21.15.157:0/3987591657 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08e40c5b70 0x7f08e4155a60 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:17.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.201+0000 7f08eb09e640 1 --2- 172.21.15.157:0/3987591657 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08e4152b40 0x7f08e40c5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:17.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.201+0000 7f08eb09e640 1 -- 172.21.15.157:0/3987591657 >> 172.21.15.157:0/3987591657 conn(0x7f08e40672c0 msgr2=0x7f08e40676d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:17.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.201+0000 7f08eb09e640 1 -- 172.21.15.157:0/3987591657 shutdown_connections 2024-04-24T23:35:17.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.201+0000 7f08eb09e640 1 -- 172.21.15.157:0/3987591657 wait complete. 2024-04-24T23:35:17.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.202+0000 7f08eb09e640 1 Processor -- start 2024-04-24T23:35:17.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.202+0000 7f08eb09e640 1 -- start start 2024-04-24T23:35:17.206 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.203+0000 7f08eb09e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08e40be360 0x7f08e4155bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:17.206 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.203+0000 7f08eb09e640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08e40c5b70 0x7f08e4156c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:17.206 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.203+0000 7f08eb09e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08e4152b40 0x7f08e41eb390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:17.206 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.203+0000 7f08eb09e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f08e40caa00 con 0x7f08e40be360 2024-04-24T23:35:17.206 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.203+0000 7f08eb09e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f08e40cb210 con 0x7f08e4152b40 2024-04-24T23:35:17.206 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.203+0000 7f08eb09e640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f08e40cba40 con 0x7f08e40c5b70 2024-04-24T23:35:17.207 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.203+0000 7f08e8e3a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08e40be360 0x7f08e4155bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:17.207 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.203+0000 7f08e8e3a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08e40be360 0x7f08e4155bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42496/0 (socket says 172.21.15.157:42496) 2024-04-24T23:35:17.207 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.203+0000 7f08e8e3a640 1 -- 172.21.15.157:0/1508494648 learned_addr learned my addr 172.21.15.157:0/1508494648 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:17.207 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.203+0000 7f08e963b640 1 --2- 172.21.15.157:0/1508494648 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08e4152b40 0x7f08e41eb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:17.207 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.203+0000 7f08e3fff640 1 --2- 172.21.15.157:0/1508494648 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08e40c5b70 0x7f08e4156c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:17.207 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.204+0000 7f08e8e3a640 1 -- 172.21.15.157:0/1508494648 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08e40c5b70 msgr2=0x7f08e4156c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:17.207 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.204+0000 7f08e8e3a640 1 --2- 172.21.15.157:0/1508494648 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08e40c5b70 0x7f08e4156c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:17.207 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.204+0000 7f08e8e3a640 1 -- 172.21.15.157:0/1508494648 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08e4152b40 msgr2=0x7f08e41eb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:17.207 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.204+0000 7f08e8e3a640 1 --2- 172.21.15.157:0/1508494648 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08e4152b40 0x7f08e41eb390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:17.207 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.204+0000 7f08e8e3a640 1 -- 172.21.15.157:0/1508494648 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08e41ef210 con 0x7f08e40be360 2024-04-24T23:35:17.207 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.204+0000 7f08e8e3a640 1 --2- 172.21.15.157:0/1508494648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08e40be360 0x7f08e4155bf0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f08d400ebe0 tx=0x7f08d400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:17.207 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.204+0000 7f08e17fa640 1 -- 172.21.15.157:0/1508494648 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d4017070 con 0x7f08e40be360 2024-04-24T23:35:17.207 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.204+0000 7f08e17fa640 1 -- 172.21.15.157:0/1508494648 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08d400cc30 con 0x7f08e40be360 2024-04-24T23:35:17.207 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.204+0000 7f08eb09e640 1 -- 172.21.15.157:0/1508494648 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08e40ad090 con 0x7f08e40be360 2024-04-24T23:35:17.208 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.204+0000 7f08e17fa640 1 -- 172.21.15.157:0/1508494648 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d4013470 con 0x7f08e40be360 2024-04-24T23:35:17.208 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.204+0000 7f08eb09e640 1 -- 172.21.15.157:0/1508494648 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08e40ad650 con 0x7f08e40be360 2024-04-24T23:35:17.208 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.205+0000 7f08eb09e640 1 -- 172.21.15.157:0/1508494648 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08e40681c0 con 0x7f08e40be360 2024-04-24T23:35:17.208 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.206+0000 7f08e17fa640 1 -- 172.21.15.157:0/1508494648 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f08d401a070 con 0x7f08e40be360 2024-04-24T23:35:17.208 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.206+0000 7f08e17fa640 1 --2- 172.21.15.157:0/1508494648 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f08d006be30 0x7f08d006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:17.208 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.206+0000 7f08e17fa640 1 -- 172.21.15.157:0/1508494648 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f08d408b490 con 0x7f08e40be360 2024-04-24T23:35:17.209 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.206+0000 7f08e3fff640 1 --2- 172.21.15.157:0/1508494648 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f08d006be30 0x7f08d006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:17.209 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.207+0000 7f08e3fff640 1 --2- 172.21.15.157:0/1508494648 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f08d006be30 0x7f08d006e2f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f08e40c4fb0 tx=0x7f08d8007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:17.211 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.210+0000 7f08e17fa640 1 -- 172.21.15.157:0/1508494648 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f08d4059430 con 0x7f08e40be360 2024-04-24T23:35:17.384 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.383+0000 7f08eb09e640 1 -- 172.21.15.157:0/1508494648 --> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f08e41eefc0 con 0x7f08d006be30 2024-04-24T23:35:17.393 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.393+0000 7f08e17fa640 1 -- 172.21.15.157:0/1508494648 <== mgr.4105 v2:172.21.15.169:6832/662378289 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332611 (secure 0 0 0) 0x7f08e41eefc0 con 0x7f08d006be30 2024-04-24T23:35:17.397 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:35:17.399 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2024-04-24T23:35:17.401 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.396+0000 7f08eb09e640 1 -- 172.21.15.157:0/1508494648 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f08d006be30 msgr2=0x7f08d006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:17.401 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.396+0000 7f08eb09e640 1 --2- 172.21.15.157:0/1508494648 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f08d006be30 0x7f08d006e2f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f08e40c4fb0 tx=0x7f08d8007de0 comp rx=0 tx=0).stop 2024-04-24T23:35:17.401 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.396+0000 7f08eb09e640 1 -- 172.21.15.157:0/1508494648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08e40be360 msgr2=0x7f08e4155bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:17.401 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.396+0000 7f08eb09e640 1 --2- 172.21.15.157:0/1508494648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08e40be360 0x7f08e4155bf0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f08d400ebe0 tx=0x7f08d400c520 comp rx=0 tx=0).stop 2024-04-24T23:35:17.401 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.396+0000 7f08eb09e640 1 -- 172.21.15.157:0/1508494648 shutdown_connections 2024-04-24T23:35:17.401 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.396+0000 7f08eb09e640 1 --2- 172.21.15.157:0/1508494648 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f08d006be30 0x7f08d006e2f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:17.401 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.396+0000 7f08eb09e640 1 --2- 172.21.15.157:0/1508494648 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08e4152b40 0x7f08e41eb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:17.401 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.396+0000 7f08eb09e640 1 --2- 172.21.15.157:0/1508494648 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08e40c5b70 0x7f08e4156c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:17.401 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.396+0000 7f08eb09e640 1 --2- 172.21.15.157:0/1508494648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08e40be360 0x7f08e4155bf0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:17.402 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.396+0000 7f08eb09e640 1 -- 172.21.15.157:0/1508494648 >> 172.21.15.157:0/1508494648 conn(0x7f08e40672c0 msgr2=0x7f08e4151b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:17.402 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.396+0000 7f08eb09e640 1 -- 172.21.15.157:0/1508494648 shutdown_connections 2024-04-24T23:35:17.402 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.396+0000 7f08eb09e640 1 -- 172.21.15.157:0/1508494648 wait complete. 2024-04-24T23:35:17.408 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2024-04-24T23:35:15.771641+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235052,"kb_used_data":2240,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739668,"statfs":{"total":773094113280,"available":772853420032,"internally_reserved":0,"allocated":2293760,"data_stored":2015416,"data_compressed":65248,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003169"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.428046+0000","last_change":"2024-04-24T23:34:53.362409+0000","last_active":"2024-04-24T23:34:59.428046+0000","last_peered":"2024-04-24T23:34:59.428046+0000","last_clean":"2024-04-24T23:34:59.428046+0000","last_became_active":"2024-04-24T23:34:53.362104+0000","last_became_peered":"2024-04-24T23:34:53.362104+0000","last_unstale":"2024-04-24T23:34:59.428046+0000","last_undegraded":"2024-04-24T23:34:59.428046+0000","last_fullsized":"2024-04-24T23:34:59.428046+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:53.159219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396596+0000","last_change":"2024-04-24T23:34:54.373560+0000","last_active":"2024-04-24T23:34:59.396596+0000","last_peered":"2024-04-24T23:34:59.396596+0000","last_clean":"2024-04-24T23:34:59.396596+0000","last_became_active":"2024-04-24T23:34:54.373289+0000","last_became_peered":"2024-04-24T23:34:54.373289+0000","last_unstale":"2024-04-24T23:34:59.396596+0000","last_undegraded":"2024-04-24T23:34:59.396596+0000","last_fullsized":"2024-04-24T23:34:59.396596+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:31.447899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.395661+0000","last_change":"2024-04-24T23:34:58.730165+0000","last_active":"2024-04-24T23:34:59.395661+0000","last_peered":"2024-04-24T23:34:59.395661+0000","last_clean":"2024-04-24T23:34:59.395661+0000","last_became_active":"2024-04-24T23:34:58.729816+0000","last_became_peered":"2024-04-24T23:34:58.729816+0000","last_unstale":"2024-04-24T23:34:59.395661+0000","last_undegraded":"2024-04-24T23:34:59.395661+0000","last_fullsized":"2024-04-24T23:34:59.395661+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:02:02.335135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396776+0000","last_change":"2024-04-24T23:34:53.361650+0000","last_active":"2024-04-24T23:34:59.396776+0000","last_peered":"2024-04-24T23:34:59.396776+0000","last_clean":"2024-04-24T23:34:59.396776+0000","last_became_active":"2024-04-24T23:34:53.361366+0000","last_became_peered":"2024-04-24T23:34:53.361366+0000","last_unstale":"2024-04-24T23:34:59.396776+0000","last_undegraded":"2024-04-24T23:34:59.396776+0000","last_fullsized":"2024-04-24T23:34:59.396776+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:56:15.784223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.427859+0000","last_change":"2024-04-24T23:34:54.383602+0000","last_active":"2024-04-24T23:34:59.427859+0000","last_peered":"2024-04-24T23:34:59.427859+0000","last_clean":"2024-04-24T23:34:59.427859+0000","last_became_active":"2024-04-24T23:34:54.383160+0000","last_became_peered":"2024-04-24T23:34:54.383160+0000","last_unstale":"2024-04-24T23:34:59.427859+0000","last_undegraded":"2024-04-24T23:34:59.427859+0000","last_fullsized":"2024-04-24T23:34:59.427859+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:15.477061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396642+0000","last_change":"2024-04-24T23:34:58.404416+0000","last_active":"2024-04-24T23:34:59.396642+0000","last_peered":"2024-04-24T23:34:59.396642+0000","last_clean":"2024-04-24T23:34:59.396642+0000","last_became_active":"2024-04-24T23:34:58.403984+0000","last_became_peered":"2024-04-24T23:34:58.403984+0000","last_unstale":"2024-04-24T23:34:59.396642+0000","last_undegraded":"2024-04-24T23:34:59.396642+0000","last_fullsized":"2024-04-24T23:34:59.396642+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:46.237082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501886+0000","last_change":"2024-04-24T23:34:53.366730+0000","last_active":"2024-04-24T23:34:59.501886+0000","last_peered":"2024-04-24T23:34:59.501886+0000","last_clean":"2024-04-24T23:34:59.501886+0000","last_became_active":"2024-04-24T23:34:53.366430+0000","last_became_peered":"2024-04-24T23:34:53.366430+0000","last_unstale":"2024-04-24T23:34:59.501886+0000","last_undegraded":"2024-04-24T23:34:59.501886+0000","last_fullsized":"2024-04-24T23:34:59.501886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:41.422591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540654+0000","last_change":"2024-04-24T23:34:54.373592+0000","last_active":"2024-04-24T23:35:00.540654+0000","last_peered":"2024-04-24T23:35:00.540654+0000","last_clean":"2024-04-24T23:35:00.540654+0000","last_became_active":"2024-04-24T23:34:54.372844+0000","last_became_peered":"2024-04-24T23:34:54.372844+0000","last_unstale":"2024-04-24T23:35:00.540654+0000","last_undegraded":"2024-04-24T23:35:00.540654+0000","last_fullsized":"2024-04-24T23:35:00.540654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:49.372040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396779+0000","last_change":"2024-04-24T23:34:58.399224+0000","last_active":"2024-04-24T23:34:59.396779+0000","last_peered":"2024-04-24T23:34:59.396779+0000","last_clean":"2024-04-24T23:34:59.396779+0000","last_became_active":"2024-04-24T23:34:58.397817+0000","last_became_peered":"2024-04-24T23:34:58.397817+0000","last_unstale":"2024-04-24T23:34:59.396779+0000","last_undegraded":"2024-04-24T23:34:59.396779+0000","last_fullsized":"2024-04-24T23:34:59.396779+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:53.085366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501568+0000","last_change":"2024-04-24T23:34:53.362994+0000","last_active":"2024-04-24T23:34:59.501568+0000","last_peered":"2024-04-24T23:34:59.501568+0000","last_clean":"2024-04-24T23:34:59.501568+0000","last_became_active":"2024-04-24T23:34:53.362337+0000","last_became_peered":"2024-04-24T23:34:53.362337+0000","last_unstale":"2024-04-24T23:34:59.501568+0000","last_undegraded":"2024-04-24T23:34:59.501568+0000","last_fullsized":"2024-04-24T23:34:59.501568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:12.258211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.541217+0000","last_change":"2024-04-24T23:34:54.379218+0000","last_active":"2024-04-24T23:35:00.541217+0000","last_peered":"2024-04-24T23:35:00.541217+0000","last_clean":"2024-04-24T23:35:00.541217+0000","last_became_active":"2024-04-24T23:34:54.376019+0000","last_became_peered":"2024-04-24T23:34:54.376019+0000","last_unstale":"2024-04-24T23:35:00.541217+0000","last_undegraded":"2024-04-24T23:35:00.541217+0000","last_fullsized":"2024-04-24T23:35:00.541217+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:41:48.207542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396752+0000","last_change":"2024-04-24T23:34:58.397152+0000","last_active":"2024-04-24T23:34:59.396752+0000","last_peered":"2024-04-24T23:34:59.396752+0000","last_clean":"2024-04-24T23:34:59.396752+0000","last_became_active":"2024-04-24T23:34:58.396658+0000","last_became_peered":"2024-04-24T23:34:58.396658+0000","last_unstale":"2024-04-24T23:34:59.396752+0000","last_undegraded":"2024-04-24T23:34:59.396752+0000","last_fullsized":"2024-04-24T23:34:59.396752+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:41.360776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540212+0000","last_change":"2024-04-24T23:34:54.377859+0000","last_active":"2024-04-24T23:35:00.540212+0000","last_peered":"2024-04-24T23:35:00.540212+0000","last_clean":"2024-04-24T23:35:00.540212+0000","last_became_active":"2024-04-24T23:34:54.377576+0000","last_became_peered":"2024-04-24T23:34:54.377576+0000","last_unstale":"2024-04-24T23:35:00.540212+0000","last_undegraded":"2024-04-24T23:35:00.540212+0000","last_fullsized":"2024-04-24T23:35:00.540212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:47.446459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501625+0000","last_change":"2024-04-24T23:34:53.360903+0000","last_active":"2024-04-24T23:34:59.501625+0000","last_peered":"2024-04-24T23:34:59.501625+0000","last_clean":"2024-04-24T23:34:59.501625+0000","last_became_active":"2024-04-24T23:34:53.360635+0000","last_became_peered":"2024-04-24T23:34:53.360635+0000","last_unstale":"2024-04-24T23:34:59.501625+0000","last_undegraded":"2024-04-24T23:34:59.501625+0000","last_fullsized":"2024-04-24T23:34:59.501625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:42.205997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397132+0000","last_change":"2024-04-24T23:34:58.407146+0000","last_active":"2024-04-24T23:34:59.397132+0000","last_peered":"2024-04-24T23:34:59.397132+0000","last_clean":"2024-04-24T23:34:59.397132+0000","last_became_active":"2024-04-24T23:34:58.406844+0000","last_became_peered":"2024-04-24T23:34:58.406844+0000","last_unstale":"2024-04-24T23:34:59.397132+0000","last_undegraded":"2024-04-24T23:34:59.397132+0000","last_fullsized":"2024-04-24T23:34:59.397132+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:41:52.244767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396642+0000","last_change":"2024-04-24T23:34:54.366724+0000","last_active":"2024-04-24T23:34:59.396642+0000","last_peered":"2024-04-24T23:34:59.396642+0000","last_clean":"2024-04-24T23:34:59.396642+0000","last_became_active":"2024-04-24T23:34:54.366426+0000","last_became_peered":"2024-04-24T23:34:54.366426+0000","last_unstale":"2024-04-24T23:34:59.396642+0000","last_undegraded":"2024-04-24T23:34:59.396642+0000","last_fullsized":"2024-04-24T23:34:59.396642+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:47.035356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.881182+0000","last_change":"2024-04-24T23:34:53.359140+0000","last_active":"2024-04-24T23:34:59.881182+0000","last_peered":"2024-04-24T23:34:59.881182+0000","last_clean":"2024-04-24T23:34:59.881182+0000","last_became_active":"2024-04-24T23:34:53.358841+0000","last_became_peered":"2024-04-24T23:34:53.358841+0000","last_unstale":"2024-04-24T23:34:59.881182+0000","last_undegraded":"2024-04-24T23:34:59.881182+0000","last_fullsized":"2024-04-24T23:34:59.881182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:41.422591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397033+0000","last_change":"2024-04-24T23:34:58.400128+0000","last_active":"2024-04-24T23:34:59.397033+0000","last_peered":"2024-04-24T23:34:59.397033+0000","last_clean":"2024-04-24T23:34:59.397033+0000","last_became_active":"2024-04-24T23:34:58.399375+0000","last_became_peered":"2024-04-24T23:34:58.399375+0000","last_unstale":"2024-04-24T23:34:59.397033+0000","last_undegraded":"2024-04-24T23:34:59.397033+0000","last_fullsized":"2024-04-24T23:34:59.397033+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:53.409265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.395824+0000","last_change":"2024-04-24T23:34:54.365012+0000","last_active":"2024-04-24T23:34:59.395824+0000","last_peered":"2024-04-24T23:34:59.395824+0000","last_clean":"2024-04-24T23:34:59.395824+0000","last_became_active":"2024-04-24T23:34:54.364744+0000","last_became_peered":"2024-04-24T23:34:54.364744+0000","last_unstale":"2024-04-24T23:34:59.395824+0000","last_undegraded":"2024-04-24T23:34:59.395824+0000","last_fullsized":"2024-04-24T23:34:59.395824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:35.774122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.880884+0000","last_change":"2024-04-24T23:34:53.360843+0000","last_active":"2024-04-24T23:34:59.880884+0000","last_peered":"2024-04-24T23:34:59.880884+0000","last_clean":"2024-04-24T23:34:59.880884+0000","last_became_active":"2024-04-24T23:34:53.360570+0000","last_became_peered":"2024-04-24T23:34:53.360570+0000","last_unstale":"2024-04-24T23:34:59.880884+0000","last_undegraded":"2024-04-24T23:34:59.880884+0000","last_fullsized":"2024-04-24T23:34:59.880884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:21.089467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.881510+0000","last_change":"2024-04-24T23:34:58.731835+0000","last_active":"2024-04-24T23:34:59.881510+0000","last_peered":"2024-04-24T23:34:59.881510+0000","last_clean":"2024-04-24T23:34:59.881510+0000","last_became_active":"2024-04-24T23:34:58.731485+0000","last_became_peered":"2024-04-24T23:34:58.731485+0000","last_unstale":"2024-04-24T23:34:59.881510+0000","last_undegraded":"2024-04-24T23:34:59.881510+0000","last_fullsized":"2024-04-24T23:34:59.881510+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:56.609480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397598+0000","last_change":"2024-04-24T23:34:54.362447+0000","last_active":"2024-04-24T23:34:59.397598+0000","last_peered":"2024-04-24T23:34:59.397598+0000","last_clean":"2024-04-24T23:34:59.397598+0000","last_became_active":"2024-04-24T23:34:54.362141+0000","last_became_peered":"2024-04-24T23:34:54.362141+0000","last_unstale":"2024-04-24T23:34:59.397598+0000","last_undegraded":"2024-04-24T23:34:59.397598+0000","last_fullsized":"2024-04-24T23:34:59.397598+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:51.227333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501234+0000","last_change":"2024-04-24T23:34:53.366452+0000","last_active":"2024-04-24T23:34:59.501234+0000","last_peered":"2024-04-24T23:34:59.501234+0000","last_clean":"2024-04-24T23:34:59.501234+0000","last_became_active":"2024-04-24T23:34:53.366110+0000","last_became_peered":"2024-04-24T23:34:53.366110+0000","last_unstale":"2024-04-24T23:34:59.501234+0000","last_undegraded":"2024-04-24T23:34:59.501234+0000","last_fullsized":"2024-04-24T23:34:59.501234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:14.716741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396886+0000","last_change":"2024-04-24T23:34:58.402849+0000","last_active":"2024-04-24T23:34:59.396886+0000","last_peered":"2024-04-24T23:34:59.396886+0000","last_clean":"2024-04-24T23:34:59.396886+0000","last_became_active":"2024-04-24T23:34:58.402365+0000","last_became_peered":"2024-04-24T23:34:58.402365+0000","last_unstale":"2024-04-24T23:34:59.396886+0000","last_undegraded":"2024-04-24T23:34:59.396886+0000","last_fullsized":"2024-04-24T23:34:59.396886+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:07:32.822427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.428004+0000","last_change":"2024-04-24T23:34:54.370300+0000","last_active":"2024-04-24T23:34:59.428004+0000","last_peered":"2024-04-24T23:34:59.428004+0000","last_clean":"2024-04-24T23:34:59.428004+0000","last_became_active":"2024-04-24T23:34:54.369835+0000","last_became_peered":"2024-04-24T23:34:54.369835+0000","last_unstale":"2024-04-24T23:34:59.428004+0000","last_undegraded":"2024-04-24T23:34:59.428004+0000","last_fullsized":"2024-04-24T23:34:59.428004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:42.199856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.541551+0000","last_change":"2024-04-24T23:34:53.366110+0000","last_active":"2024-04-24T23:35:00.541551+0000","last_peered":"2024-04-24T23:35:00.541551+0000","last_clean":"2024-04-24T23:35:00.541551+0000","last_became_active":"2024-04-24T23:34:53.365783+0000","last_became_peered":"2024-04-24T23:34:53.365783+0000","last_unstale":"2024-04-24T23:35:00.541551+0000","last_undegraded":"2024-04-24T23:35:00.541551+0000","last_fullsized":"2024-04-24T23:35:00.541551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:41.422591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397333+0000","last_change":"2024-04-24T23:34:58.728191+0000","last_active":"2024-04-24T23:34:59.397333+0000","last_peered":"2024-04-24T23:34:59.397333+0000","last_clean":"2024-04-24T23:34:59.397333+0000","last_became_active":"2024-04-24T23:34:58.727653+0000","last_became_peered":"2024-04-24T23:34:58.727653+0000","last_unstale":"2024-04-24T23:34:59.397333+0000","last_undegraded":"2024-04-24T23:34:59.397333+0000","last_fullsized":"2024-04-24T23:34:59.397333+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:30.299011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397474+0000","last_change":"2024-04-24T23:34:54.368278+0000","last_active":"2024-04-24T23:34:59.397474+0000","last_peered":"2024-04-24T23:34:59.397474+0000","last_clean":"2024-04-24T23:34:59.397474+0000","last_became_active":"2024-04-24T23:34:54.367991+0000","last_became_peered":"2024-04-24T23:34:54.367991+0000","last_unstale":"2024-04-24T23:34:59.397474+0000","last_undegraded":"2024-04-24T23:34:59.397474+0000","last_fullsized":"2024-04-24T23:34:59.397474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:49:31.018719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.881400+0000","last_change":"2024-04-24T23:34:53.364189+0000","last_active":"2024-04-24T23:34:59.881400+0000","last_peered":"2024-04-24T23:34:59.881400+0000","last_clean":"2024-04-24T23:34:59.881400+0000","last_became_active":"2024-04-24T23:34:53.363842+0000","last_became_peered":"2024-04-24T23:34:53.363842+0000","last_unstale":"2024-04-24T23:34:59.881400+0000","last_undegraded":"2024-04-24T23:34:59.881400+0000","last_fullsized":"2024-04-24T23:34:59.881400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:12.258211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.395610+0000","last_change":"2024-04-24T23:34:58.729450+0000","last_active":"2024-04-24T23:34:59.395610+0000","last_peered":"2024-04-24T23:34:59.395610+0000","last_clean":"2024-04-24T23:34:59.395610+0000","last_became_active":"2024-04-24T23:34:58.729082+0000","last_became_peered":"2024-04-24T23:34:58.729082+0000","last_unstale":"2024-04-24T23:34:59.395610+0000","last_undegraded":"2024-04-24T23:34:59.395610+0000","last_fullsized":"2024-04-24T23:34:59.395610+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:45.372053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.881249+0000","last_change":"2024-04-24T23:34:54.357246+0000","last_active":"2024-04-24T23:34:59.881249+0000","last_peered":"2024-04-24T23:34:59.881249+0000","last_clean":"2024-04-24T23:34:59.881249+0000","last_became_active":"2024-04-24T23:34:54.356560+0000","last_became_peered":"2024-04-24T23:34:54.356560+0000","last_unstale":"2024-04-24T23:34:59.881249+0000","last_undegraded":"2024-04-24T23:34:59.881249+0000","last_fullsized":"2024-04-24T23:34:59.881249+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:42.199856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.502011+0000","last_change":"2024-04-24T23:34:53.366296+0000","last_active":"2024-04-24T23:34:59.502011+0000","last_peered":"2024-04-24T23:34:59.502011+0000","last_clean":"2024-04-24T23:34:59.502011+0000","last_became_active":"2024-04-24T23:34:53.365636+0000","last_became_peered":"2024-04-24T23:34:53.365636+0000","last_unstale":"2024-04-24T23:34:59.502011+0000","last_undegraded":"2024-04-24T23:34:59.502011+0000","last_fullsized":"2024-04-24T23:34:59.502011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:50.222015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.427745+0000","last_change":"2024-04-24T23:34:58.728271+0000","last_active":"2024-04-24T23:34:59.427745+0000","last_peered":"2024-04-24T23:34:59.427745+0000","last_clean":"2024-04-24T23:34:59.427745+0000","last_became_active":"2024-04-24T23:34:58.727885+0000","last_became_peered":"2024-04-24T23:34:58.727885+0000","last_unstale":"2024-04-24T23:34:59.427745+0000","last_undegraded":"2024-04-24T23:34:59.427745+0000","last_fullsized":"2024-04-24T23:34:59.427745+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:49.461271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.541247+0000","last_change":"2024-04-24T23:34:54.372245+0000","last_active":"2024-04-24T23:35:00.541247+0000","last_peered":"2024-04-24T23:35:00.541247+0000","last_clean":"2024-04-24T23:35:00.541247+0000","last_became_active":"2024-04-24T23:34:54.371947+0000","last_became_peered":"2024-04-24T23:34:54.371947+0000","last_unstale":"2024-04-24T23:35:00.541247+0000","last_undegraded":"2024-04-24T23:35:00.541247+0000","last_fullsized":"2024-04-24T23:35:00.541247+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:47:52.460508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.881345+0000","last_change":"2024-04-24T23:34:53.360352+0000","last_active":"2024-04-24T23:34:59.881345+0000","last_peered":"2024-04-24T23:34:59.881345+0000","last_clean":"2024-04-24T23:34:59.881345+0000","last_became_active":"2024-04-24T23:34:53.360082+0000","last_became_peered":"2024-04-24T23:34:53.360082+0000","last_unstale":"2024-04-24T23:34:59.881345+0000","last_undegraded":"2024-04-24T23:34:59.881345+0000","last_fullsized":"2024-04-24T23:34:59.881345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:50.222015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.881021+0000","last_change":"2024-04-24T23:34:58.404176+0000","last_active":"2024-04-24T23:34:59.881021+0000","last_peered":"2024-04-24T23:34:59.881021+0000","last_clean":"2024-04-24T23:34:59.881021+0000","last_became_active":"2024-04-24T23:34:58.403735+0000","last_became_peered":"2024-04-24T23:34:58.403735+0000","last_unstale":"2024-04-24T23:34:59.881021+0000","last_undegraded":"2024-04-24T23:34:59.881021+0000","last_fullsized":"2024-04-24T23:34:59.881021+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:49.461271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397652+0000","last_change":"2024-04-24T23:34:54.367638+0000","last_active":"2024-04-24T23:34:59.397652+0000","last_peered":"2024-04-24T23:34:59.397652+0000","last_clean":"2024-04-24T23:34:59.397652+0000","last_became_active":"2024-04-24T23:34:54.367355+0000","last_became_peered":"2024-04-24T23:34:54.367355+0000","last_unstale":"2024-04-24T23:34:59.397652+0000","last_undegraded":"2024-04-24T23:34:59.397652+0000","last_fullsized":"2024-04-24T23:34:59.397652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:15.722059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540545+0000","last_change":"2024-04-24T23:34:53.356111+0000","last_active":"2024-04-24T23:35:00.540545+0000","last_peered":"2024-04-24T23:35:00.540545+0000","last_clean":"2024-04-24T23:35:00.540545+0000","last_became_active":"2024-04-24T23:34:53.355710+0000","last_became_peered":"2024-04-24T23:34:53.355710+0000","last_unstale":"2024-04-24T23:35:00.540545+0000","last_undegraded":"2024-04-24T23:35:00.540545+0000","last_fullsized":"2024-04-24T23:35:00.540545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:21.089467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.881283+0000","last_change":"2024-04-24T23:34:58.403016+0000","last_active":"2024-04-24T23:34:59.881283+0000","last_peered":"2024-04-24T23:34:59.881283+0000","last_clean":"2024-04-24T23:34:59.881283+0000","last_became_active":"2024-04-24T23:34:58.402598+0000","last_became_peered":"2024-04-24T23:34:58.402598+0000","last_unstale":"2024-04-24T23:34:59.881283+0000","last_undegraded":"2024-04-24T23:34:59.881283+0000","last_fullsized":"2024-04-24T23:34:59.881283+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:47:56.497734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.880863+0000","last_change":"2024-04-24T23:34:54.356601+0000","last_active":"2024-04-24T23:34:59.880863+0000","last_peered":"2024-04-24T23:34:59.880863+0000","last_clean":"2024-04-24T23:34:59.880863+0000","last_became_active":"2024-04-24T23:34:54.356326+0000","last_became_peered":"2024-04-24T23:34:54.356326+0000","last_unstale":"2024-04-24T23:34:59.880863+0000","last_undegraded":"2024-04-24T23:34:59.880863+0000","last_fullsized":"2024-04-24T23:34:59.880863+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:47.446459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396563+0000","last_change":"2024-04-24T23:34:53.359238+0000","last_active":"2024-04-24T23:34:59.396563+0000","last_peered":"2024-04-24T23:34:59.396563+0000","last_clean":"2024-04-24T23:34:59.396563+0000","last_became_active":"2024-04-24T23:34:53.358937+0000","last_became_peered":"2024-04-24T23:34:53.358937+0000","last_unstale":"2024-04-24T23:34:59.396563+0000","last_undegraded":"2024-04-24T23:34:59.396563+0000","last_fullsized":"2024-04-24T23:34:59.396563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:50.222015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.881551+0000","last_change":"2024-04-24T23:34:58.403258+0000","last_active":"2024-04-24T23:34:59.881551+0000","last_peered":"2024-04-24T23:34:59.881551+0000","last_clean":"2024-04-24T23:34:59.881551+0000","last_became_active":"2024-04-24T23:34:58.402569+0000","last_became_peered":"2024-04-24T23:34:58.402569+0000","last_unstale":"2024-04-24T23:34:59.881551+0000","last_undegraded":"2024-04-24T23:34:59.881551+0000","last_fullsized":"2024-04-24T23:34:59.881551+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:32:08.782070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.427468+0000","last_change":"2024-04-24T23:34:54.377543+0000","last_active":"2024-04-24T23:34:59.427468+0000","last_peered":"2024-04-24T23:34:59.427468+0000","last_clean":"2024-04-24T23:34:59.427468+0000","last_became_active":"2024-04-24T23:34:54.377228+0000","last_became_peered":"2024-04-24T23:34:54.377228+0000","last_unstale":"2024-04-24T23:34:59.427468+0000","last_undegraded":"2024-04-24T23:34:59.427468+0000","last_fullsized":"2024-04-24T23:34:59.427468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:42.255038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501238+0000","last_change":"2024-04-24T23:34:53.362675+0000","last_active":"2024-04-24T23:34:59.501238+0000","last_peered":"2024-04-24T23:34:59.501238+0000","last_clean":"2024-04-24T23:34:59.501238+0000","last_became_active":"2024-04-24T23:34:53.362425+0000","last_became_peered":"2024-04-24T23:34:53.362425+0000","last_unstale":"2024-04-24T23:34:59.501238+0000","last_undegraded":"2024-04-24T23:34:59.501238+0000","last_fullsized":"2024-04-24T23:34:59.501238+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:21.089467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.427884+0000","last_change":"2024-04-24T23:34:58.728573+0000","last_active":"2024-04-24T23:34:59.427884+0000","last_peered":"2024-04-24T23:34:59.427884+0000","last_clean":"2024-04-24T23:34:59.427884+0000","last_became_active":"2024-04-24T23:34:58.727284+0000","last_became_peered":"2024-04-24T23:34:58.727284+0000","last_unstale":"2024-04-24T23:34:59.427884+0000","last_undegraded":"2024-04-24T23:34:59.427884+0000","last_fullsized":"2024-04-24T23:34:59.427884+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:32:08.782070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396198+0000","last_change":"2024-04-24T23:34:54.382820+0000","last_active":"2024-04-24T23:34:59.396198+0000","last_peered":"2024-04-24T23:34:59.396198+0000","last_clean":"2024-04-24T23:34:59.396198+0000","last_became_active":"2024-04-24T23:34:54.382564+0000","last_became_peered":"2024-04-24T23:34:54.382564+0000","last_unstale":"2024-04-24T23:34:59.396198+0000","last_undegraded":"2024-04-24T23:34:59.396198+0000","last_fullsized":"2024-04-24T23:34:59.396198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:45.622611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.541368+0000","last_change":"2024-04-24T23:34:53.357743+0000","last_active":"2024-04-24T23:35:00.541368+0000","last_peered":"2024-04-24T23:35:00.541368+0000","last_clean":"2024-04-24T23:35:00.541368+0000","last_became_active":"2024-04-24T23:34:53.357456+0000","last_became_peered":"2024-04-24T23:34:53.357456+0000","last_unstale":"2024-04-24T23:35:00.541368+0000","last_undegraded":"2024-04-24T23:35:00.541368+0000","last_fullsized":"2024-04-24T23:35:00.541368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:57:14.381644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":177,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501380+0000","last_change":"2024-04-24T23:34:49.778966+0000","last_active":"2024-04-24T23:34:59.501380+0000","last_peered":"2024-04-24T23:34:59.501380+0000","last_clean":"2024-04-24T23:34:59.501380+0000","last_became_active":"2024-04-24T23:34:49.778083+0000","last_became_peered":"2024-04-24T23:34:49.778083+0000","last_unstale":"2024-04-24T23:34:59.501380+0000","last_undegraded":"2024-04-24T23:34:59.501380+0000","last_fullsized":"2024-04-24T23:34:59.501380+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:48.587078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:48.587078+0000","last_clean_scrub_stamp":"2024-04-24T23:34:48.587078+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:46:59.471089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540517+0000","last_change":"2024-04-24T23:34:58.729779+0000","last_active":"2024-04-24T23:35:00.540517+0000","last_peered":"2024-04-24T23:35:00.540517+0000","last_clean":"2024-04-24T23:35:00.540517+0000","last_became_active":"2024-04-24T23:34:58.729370+0000","last_became_peered":"2024-04-24T23:34:58.729370+0000","last_unstale":"2024-04-24T23:35:00.540517+0000","last_undegraded":"2024-04-24T23:35:00.540517+0000","last_fullsized":"2024-04-24T23:35:00.540517+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:30.299011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501756+0000","last_change":"2024-04-24T23:34:54.384106+0000","last_active":"2024-04-24T23:34:59.501756+0000","last_peered":"2024-04-24T23:34:59.501756+0000","last_clean":"2024-04-24T23:34:59.501756+0000","last_became_active":"2024-04-24T23:34:54.383589+0000","last_became_peered":"2024-04-24T23:34:54.383589+0000","last_unstale":"2024-04-24T23:34:59.501756+0000","last_undegraded":"2024-04-24T23:34:59.501756+0000","last_fullsized":"2024-04-24T23:34:59.501756+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:41:48.207542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397450+0000","last_change":"2024-04-24T23:34:53.354077+0000","last_active":"2024-04-24T23:34:59.397450+0000","last_peered":"2024-04-24T23:34:59.397450+0000","last_clean":"2024-04-24T23:34:59.397450+0000","last_became_active":"2024-04-24T23:34:53.353804+0000","last_became_peered":"2024-04-24T23:34:53.353804+0000","last_unstale":"2024-04-24T23:34:59.397450+0000","last_undegraded":"2024-04-24T23:34:59.397450+0000","last_fullsized":"2024-04-24T23:34:59.397450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:41.422591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540103+0000","last_change":"2024-04-24T23:34:58.728722+0000","last_active":"2024-04-24T23:35:00.540103+0000","last_peered":"2024-04-24T23:35:00.540103+0000","last_clean":"2024-04-24T23:35:00.540103+0000","last_became_active":"2024-04-24T23:34:58.727649+0000","last_became_peered":"2024-04-24T23:34:58.727649+0000","last_unstale":"2024-04-24T23:35:00.540103+0000","last_undegraded":"2024-04-24T23:35:00.540103+0000","last_fullsized":"2024-04-24T23:35:00.540103+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:41:52.244767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540678+0000","last_change":"2024-04-24T23:34:54.373371+0000","last_active":"2024-04-24T23:35:00.540678+0000","last_peered":"2024-04-24T23:35:00.540678+0000","last_clean":"2024-04-24T23:35:00.540678+0000","last_became_active":"2024-04-24T23:34:54.372553+0000","last_became_peered":"2024-04-24T23:34:54.372553+0000","last_unstale":"2024-04-24T23:35:00.540678+0000","last_undegraded":"2024-04-24T23:35:00.540678+0000","last_fullsized":"2024-04-24T23:35:00.540678+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:26.261786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.881287+0000","last_change":"2024-04-24T23:34:53.353505+0000","last_active":"2024-04-24T23:34:59.881287+0000","last_peered":"2024-04-24T23:34:59.881287+0000","last_clean":"2024-04-24T23:34:59.881287+0000","last_became_active":"2024-04-24T23:34:53.353186+0000","last_became_peered":"2024-04-24T23:34:53.353186+0000","last_unstale":"2024-04-24T23:34:59.881287+0000","last_undegraded":"2024-04-24T23:34:59.881287+0000","last_fullsized":"2024-04-24T23:34:59.881287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:42.205997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.427805+0000","last_change":"2024-04-24T23:34:58.400988+0000","last_active":"2024-04-24T23:34:59.427805+0000","last_peered":"2024-04-24T23:34:59.427805+0000","last_clean":"2024-04-24T23:34:59.427805+0000","last_became_active":"2024-04-24T23:34:58.400549+0000","last_became_peered":"2024-04-24T23:34:58.400549+0000","last_unstale":"2024-04-24T23:34:59.427805+0000","last_undegraded":"2024-04-24T23:34:59.427805+0000","last_fullsized":"2024-04-24T23:34:59.427805+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:47:56.497734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.502060+0000","last_change":"2024-04-24T23:34:54.375234+0000","last_active":"2024-04-24T23:34:59.502060+0000","last_peered":"2024-04-24T23:34:59.502060+0000","last_clean":"2024-04-24T23:34:59.502060+0000","last_became_active":"2024-04-24T23:34:54.374963+0000","last_became_peered":"2024-04-24T23:34:54.374963+0000","last_unstale":"2024-04-24T23:34:59.502060+0000","last_undegraded":"2024-04-24T23:34:59.502060+0000","last_fullsized":"2024-04-24T23:34:59.502060+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:47:52.460508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396918+0000","last_change":"2024-04-24T23:34:53.360531+0000","last_active":"2024-04-24T23:34:59.396918+0000","last_peered":"2024-04-24T23:34:59.396918+0000","last_clean":"2024-04-24T23:34:59.396918+0000","last_became_active":"2024-04-24T23:34:53.360247+0000","last_became_peered":"2024-04-24T23:34:53.360247+0000","last_unstale":"2024-04-24T23:34:59.396918+0000","last_undegraded":"2024-04-24T23:34:59.396918+0000","last_fullsized":"2024-04-24T23:34:59.396918+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:57:14.381644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540673+0000","last_change":"2024-04-24T23:34:58.729798+0000","last_active":"2024-04-24T23:35:00.540673+0000","last_peered":"2024-04-24T23:35:00.540673+0000","last_clean":"2024-04-24T23:35:00.540673+0000","last_became_active":"2024-04-24T23:34:58.729345+0000","last_became_peered":"2024-04-24T23:34:58.729345+0000","last_unstale":"2024-04-24T23:35:00.540673+0000","last_undegraded":"2024-04-24T23:35:00.540673+0000","last_fullsized":"2024-04-24T23:35:00.540673+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:06.670518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501916+0000","last_change":"2024-04-24T23:34:54.377593+0000","last_active":"2024-04-24T23:34:59.501916+0000","last_peered":"2024-04-24T23:34:59.501916+0000","last_clean":"2024-04-24T23:34:59.501916+0000","last_became_active":"2024-04-24T23:34:54.377270+0000","last_became_peered":"2024-04-24T23:34:54.377270+0000","last_unstale":"2024-04-24T23:34:59.501916+0000","last_undegraded":"2024-04-24T23:34:59.501916+0000","last_fullsized":"2024-04-24T23:34:59.501916+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:26.261786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397587+0000","last_change":"2024-04-24T23:34:53.352981+0000","last_active":"2024-04-24T23:34:59.397587+0000","last_peered":"2024-04-24T23:34:59.397587+0000","last_clean":"2024-04-24T23:34:59.397587+0000","last_became_active":"2024-04-24T23:34:53.352669+0000","last_became_peered":"2024-04-24T23:34:53.352669+0000","last_unstale":"2024-04-24T23:34:59.397587+0000","last_undegraded":"2024-04-24T23:34:59.397587+0000","last_fullsized":"2024-04-24T23:34:59.397587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:09:49.550050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396650+0000","last_change":"2024-04-24T23:34:58.399234+0000","last_active":"2024-04-24T23:34:59.396650+0000","last_peered":"2024-04-24T23:34:59.396650+0000","last_clean":"2024-04-24T23:34:59.396650+0000","last_became_active":"2024-04-24T23:34:58.398824+0000","last_became_peered":"2024-04-24T23:34:58.398824+0000","last_unstale":"2024-04-24T23:34:59.396650+0000","last_undegraded":"2024-04-24T23:34:59.396650+0000","last_fullsized":"2024-04-24T23:34:59.396650+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:22.244493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501466+0000","last_change":"2024-04-24T23:34:54.376800+0000","last_active":"2024-04-24T23:34:59.501466+0000","last_peered":"2024-04-24T23:34:59.501466+0000","last_clean":"2024-04-24T23:34:59.501466+0000","last_became_active":"2024-04-24T23:34:54.376512+0000","last_became_peered":"2024-04-24T23:34:54.376512+0000","last_unstale":"2024-04-24T23:34:59.501466+0000","last_undegraded":"2024-04-24T23:34:59.501466+0000","last_fullsized":"2024-04-24T23:34:59.501466+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:49.048141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396460+0000","last_change":"2024-04-24T23:34:53.355307+0000","last_active":"2024-04-24T23:34:59.396460+0000","last_peered":"2024-04-24T23:34:59.396460+0000","last_clean":"2024-04-24T23:34:59.396460+0000","last_became_active":"2024-04-24T23:34:53.354988+0000","last_became_peered":"2024-04-24T23:34:53.354988+0000","last_unstale":"2024-04-24T23:34:59.396460+0000","last_undegraded":"2024-04-24T23:34:59.396460+0000","last_fullsized":"2024-04-24T23:34:59.396460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:39.457116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.427781+0000","last_change":"2024-04-24T23:34:58.399188+0000","last_active":"2024-04-24T23:34:59.427781+0000","last_peered":"2024-04-24T23:34:59.427781+0000","last_clean":"2024-04-24T23:34:59.427781+0000","last_became_active":"2024-04-24T23:34:58.398457+0000","last_became_peered":"2024-04-24T23:34:58.398457+0000","last_unstale":"2024-04-24T23:34:59.427781+0000","last_undegraded":"2024-04-24T23:34:59.427781+0000","last_fullsized":"2024-04-24T23:34:59.427781+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:50:21.658540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540120+0000","last_change":"2024-04-24T23:34:54.372150+0000","last_active":"2024-04-24T23:35:00.540120+0000","last_peered":"2024-04-24T23:35:00.540120+0000","last_clean":"2024-04-24T23:35:00.540120+0000","last_became_active":"2024-04-24T23:34:54.371703+0000","last_became_peered":"2024-04-24T23:34:54.371703+0000","last_unstale":"2024-04-24T23:35:00.540120+0000","last_undegraded":"2024-04-24T23:35:00.540120+0000","last_fullsized":"2024-04-24T23:35:00.540120+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:15.477061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540199+0000","last_change":"2024-04-24T23:34:53.363787+0000","last_active":"2024-04-24T23:35:00.540199+0000","last_peered":"2024-04-24T23:35:00.540199+0000","last_clean":"2024-04-24T23:35:00.540199+0000","last_became_active":"2024-04-24T23:34:53.363515+0000","last_became_peered":"2024-04-24T23:34:53.363515+0000","last_unstale":"2024-04-24T23:35:00.540199+0000","last_undegraded":"2024-04-24T23:35:00.540199+0000","last_fullsized":"2024-04-24T23:35:00.540199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:50.222015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397429+0000","last_change":"2024-04-24T23:34:58.728420+0000","last_active":"2024-04-24T23:34:59.397429+0000","last_peered":"2024-04-24T23:34:59.397429+0000","last_clean":"2024-04-24T23:34:59.397429+0000","last_became_active":"2024-04-24T23:34:58.727425+0000","last_became_peered":"2024-04-24T23:34:58.727425+0000","last_unstale":"2024-04-24T23:34:59.397429+0000","last_undegraded":"2024-04-24T23:34:59.397429+0000","last_fullsized":"2024-04-24T23:34:59.397429+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:19:14.487689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501575+0000","last_change":"2024-04-24T23:34:54.379296+0000","last_active":"2024-04-24T23:34:59.501575+0000","last_peered":"2024-04-24T23:34:59.501575+0000","last_clean":"2024-04-24T23:34:59.501575+0000","last_became_active":"2024-04-24T23:34:54.379016+0000","last_became_peered":"2024-04-24T23:34:54.379016+0000","last_unstale":"2024-04-24T23:34:59.501575+0000","last_undegraded":"2024-04-24T23:34:59.501575+0000","last_fullsized":"2024-04-24T23:34:59.501575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:37.323551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540560+0000","last_change":"2024-04-24T23:34:53.360210+0000","last_active":"2024-04-24T23:35:00.540560+0000","last_peered":"2024-04-24T23:35:00.540560+0000","last_clean":"2024-04-24T23:35:00.540560+0000","last_became_active":"2024-04-24T23:34:53.359581+0000","last_became_peered":"2024-04-24T23:34:53.359581+0000","last_unstale":"2024-04-24T23:35:00.540560+0000","last_undegraded":"2024-04-24T23:35:00.540560+0000","last_fullsized":"2024-04-24T23:35:00.540560+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:14.716741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397168+0000","last_change":"2024-04-24T23:34:58.728491+0000","last_active":"2024-04-24T23:34:59.397168+0000","last_peered":"2024-04-24T23:34:59.397168+0000","last_clean":"2024-04-24T23:34:59.397168+0000","last_became_active":"2024-04-24T23:34:58.727686+0000","last_became_peered":"2024-04-24T23:34:58.727686+0000","last_unstale":"2024-04-24T23:34:59.397168+0000","last_undegraded":"2024-04-24T23:34:59.397168+0000","last_fullsized":"2024-04-24T23:34:59.397168+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:46.282712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.427650+0000","last_change":"2024-04-24T23:34:54.372009+0000","last_active":"2024-04-24T23:34:59.427650+0000","last_peered":"2024-04-24T23:34:59.427650+0000","last_clean":"2024-04-24T23:34:59.427650+0000","last_became_active":"2024-04-24T23:34:54.371720+0000","last_became_peered":"2024-04-24T23:34:54.371720+0000","last_unstale":"2024-04-24T23:34:59.427650+0000","last_undegraded":"2024-04-24T23:34:59.427650+0000","last_fullsized":"2024-04-24T23:34:59.427650+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:04:07.487367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.428005+0000","last_change":"2024-04-24T23:34:53.353738+0000","last_active":"2024-04-24T23:34:59.428005+0000","last_peered":"2024-04-24T23:34:59.428005+0000","last_clean":"2024-04-24T23:34:59.428005+0000","last_became_active":"2024-04-24T23:34:53.353452+0000","last_became_peered":"2024-04-24T23:34:53.353452+0000","last_unstale":"2024-04-24T23:34:59.428005+0000","last_undegraded":"2024-04-24T23:34:59.428005+0000","last_fullsized":"2024-04-24T23:34:59.428005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:33:43.693095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.541104+0000","last_change":"2024-04-24T23:34:58.728503+0000","last_active":"2024-04-24T23:35:00.541104+0000","last_peered":"2024-04-24T23:35:00.541104+0000","last_clean":"2024-04-24T23:35:00.541104+0000","last_became_active":"2024-04-24T23:34:58.727921+0000","last_became_peered":"2024-04-24T23:34:58.727921+0000","last_unstale":"2024-04-24T23:35:00.541104+0000","last_undegraded":"2024-04-24T23:35:00.541104+0000","last_fullsized":"2024-04-24T23:35:00.541104+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:19:40.928966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501636+0000","last_change":"2024-04-24T23:34:54.368532+0000","last_active":"2024-04-24T23:34:59.501636+0000","last_peered":"2024-04-24T23:34:59.501636+0000","last_clean":"2024-04-24T23:34:59.501636+0000","last_became_active":"2024-04-24T23:34:54.367903+0000","last_became_peered":"2024-04-24T23:34:54.367903+0000","last_unstale":"2024-04-24T23:34:59.501636+0000","last_undegraded":"2024-04-24T23:34:59.501636+0000","last_fullsized":"2024-04-24T23:34:59.501636+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:49.372040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:03.674709+0000","last_change":"2024-04-24T23:34:53.362758+0000","last_active":"2024-04-24T23:35:03.674709+0000","last_peered":"2024-04-24T23:35:03.674709+0000","last_clean":"2024-04-24T23:35:03.674709+0000","last_became_active":"2024-04-24T23:34:53.362230+0000","last_became_peered":"2024-04-24T23:34:53.362230+0000","last_unstale":"2024-04-24T23:35:03.674709+0000","last_undegraded":"2024-04-24T23:35:03.674709+0000","last_fullsized":"2024-04-24T23:35:03.674709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:14.716741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.881050+0000","last_change":"2024-04-24T23:34:58.400881+0000","last_active":"2024-04-24T23:34:59.881050+0000","last_peered":"2024-04-24T23:34:59.881050+0000","last_clean":"2024-04-24T23:34:59.881050+0000","last_became_active":"2024-04-24T23:34:58.400429+0000","last_became_peered":"2024-04-24T23:34:58.400429+0000","last_unstale":"2024-04-24T23:34:59.881050+0000","last_undegraded":"2024-04-24T23:34:59.881050+0000","last_fullsized":"2024-04-24T23:34:59.881050+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:50:21.658540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397180+0000","last_change":"2024-04-24T23:34:54.373603+0000","last_active":"2024-04-24T23:34:59.397180+0000","last_peered":"2024-04-24T23:34:59.397180+0000","last_clean":"2024-04-24T23:34:59.397180+0000","last_became_active":"2024-04-24T23:34:54.373347+0000","last_became_peered":"2024-04-24T23:34:54.373347+0000","last_unstale":"2024-04-24T23:34:59.397180+0000","last_undegraded":"2024-04-24T23:34:59.397180+0000","last_fullsized":"2024-04-24T23:34:59.397180+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:13.263529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.881250+0000","last_change":"2024-04-24T23:34:53.354673+0000","last_active":"2024-04-24T23:34:59.881250+0000","last_peered":"2024-04-24T23:34:59.881250+0000","last_clean":"2024-04-24T23:34:59.881250+0000","last_became_active":"2024-04-24T23:34:53.354394+0000","last_became_peered":"2024-04-24T23:34:53.354394+0000","last_unstale":"2024-04-24T23:34:59.881250+0000","last_undegraded":"2024-04-24T23:34:59.881250+0000","last_fullsized":"2024-04-24T23:34:59.881250+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:14.716741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397330+0000","last_change":"2024-04-24T23:34:58.397739+0000","last_active":"2024-04-24T23:34:59.397330+0000","last_peered":"2024-04-24T23:34:59.397330+0000","last_clean":"2024-04-24T23:34:59.397330+0000","last_became_active":"2024-04-24T23:34:58.397353+0000","last_became_peered":"2024-04-24T23:34:58.397353+0000","last_unstale":"2024-04-24T23:34:59.397330+0000","last_undegraded":"2024-04-24T23:34:59.397330+0000","last_fullsized":"2024-04-24T23:34:59.397330+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:55.672101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.427967+0000","last_change":"2024-04-24T23:34:54.383348+0000","last_active":"2024-04-24T23:34:59.427967+0000","last_peered":"2024-04-24T23:34:59.427967+0000","last_clean":"2024-04-24T23:34:59.427967+0000","last_became_active":"2024-04-24T23:34:54.383082+0000","last_became_peered":"2024-04-24T23:34:54.383082+0000","last_unstale":"2024-04-24T23:34:59.427967+0000","last_undegraded":"2024-04-24T23:34:59.427967+0000","last_fullsized":"2024-04-24T23:34:59.427967+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:56:17.763159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540591+0000","last_change":"2024-04-24T23:34:53.364540+0000","last_active":"2024-04-24T23:35:00.540591+0000","last_peered":"2024-04-24T23:35:00.540591+0000","last_clean":"2024-04-24T23:35:00.540591+0000","last_became_active":"2024-04-24T23:34:53.364259+0000","last_became_peered":"2024-04-24T23:34:53.364259+0000","last_unstale":"2024-04-24T23:35:00.540591+0000","last_undegraded":"2024-04-24T23:35:00.540591+0000","last_fullsized":"2024-04-24T23:35:00.540591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:41.422591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.395771+0000","last_change":"2024-04-24T23:34:58.405865+0000","last_active":"2024-04-24T23:34:59.395771+0000","last_peered":"2024-04-24T23:34:59.395771+0000","last_clean":"2024-04-24T23:34:59.395771+0000","last_became_active":"2024-04-24T23:34:58.405032+0000","last_became_peered":"2024-04-24T23:34:58.405032+0000","last_unstale":"2024-04-24T23:34:59.395771+0000","last_undegraded":"2024-04-24T23:34:59.395771+0000","last_fullsized":"2024-04-24T23:34:59.395771+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:54.012389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540495+0000","last_change":"2024-04-24T23:34:54.383631+0000","last_active":"2024-04-24T23:35:00.540495+0000","last_peered":"2024-04-24T23:35:00.540495+0000","last_clean":"2024-04-24T23:35:00.540495+0000","last_became_active":"2024-04-24T23:34:54.383354+0000","last_became_peered":"2024-04-24T23:34:54.383354+0000","last_unstale":"2024-04-24T23:35:00.540495+0000","last_undegraded":"2024-04-24T23:35:00.540495+0000","last_fullsized":"2024-04-24T23:35:00.540495+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:50:17.621315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.541421+0000","last_change":"2024-04-24T23:34:53.362409+0000","last_active":"2024-04-24T23:35:00.541421+0000","last_peered":"2024-04-24T23:35:00.541421+0000","last_clean":"2024-04-24T23:35:00.541421+0000","last_became_active":"2024-04-24T23:34:53.362103+0000","last_became_peered":"2024-04-24T23:34:53.362103+0000","last_unstale":"2024-04-24T23:35:00.541421+0000","last_undegraded":"2024-04-24T23:35:00.541421+0000","last_fullsized":"2024-04-24T23:35:00.541421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:57:14.381644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.395785+0000","last_change":"2024-04-24T23:34:58.730088+0000","last_active":"2024-04-24T23:34:59.395785+0000","last_peered":"2024-04-24T23:34:59.395785+0000","last_clean":"2024-04-24T23:34:59.395785+0000","last_became_active":"2024-04-24T23:34:58.729217+0000","last_became_peered":"2024-04-24T23:34:58.729217+0000","last_unstale":"2024-04-24T23:34:59.395785+0000","last_undegraded":"2024-04-24T23:34:59.395785+0000","last_fullsized":"2024-04-24T23:34:59.395785+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:12:14.243770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.427823+0000","last_change":"2024-04-24T23:34:58.728892+0000","last_active":"2024-04-24T23:34:59.427823+0000","last_peered":"2024-04-24T23:34:59.427823+0000","last_clean":"2024-04-24T23:34:59.427823+0000","last_became_active":"2024-04-24T23:34:58.727914+0000","last_became_peered":"2024-04-24T23:34:58.727914+0000","last_unstale":"2024-04-24T23:34:59.427823+0000","last_undegraded":"2024-04-24T23:34:59.427823+0000","last_fullsized":"2024-04-24T23:34:59.427823+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:56.609480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.880773+0000","last_change":"2024-04-24T23:34:54.363973+0000","last_active":"2024-04-24T23:34:59.880773+0000","last_peered":"2024-04-24T23:34:59.880773+0000","last_clean":"2024-04-24T23:34:59.880773+0000","last_became_active":"2024-04-24T23:34:54.363724+0000","last_became_peered":"2024-04-24T23:34:54.363724+0000","last_unstale":"2024-04-24T23:34:59.880773+0000","last_undegraded":"2024-04-24T23:34:59.880773+0000","last_fullsized":"2024-04-24T23:34:59.880773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:03:24.745895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.541110+0000","last_change":"2024-04-24T23:34:53.360041+0000","last_active":"2024-04-24T23:35:00.541110+0000","last_peered":"2024-04-24T23:35:00.541110+0000","last_clean":"2024-04-24T23:35:00.541110+0000","last_became_active":"2024-04-24T23:34:53.359707+0000","last_became_peered":"2024-04-24T23:34:53.359707+0000","last_unstale":"2024-04-24T23:35:00.541110+0000","last_undegraded":"2024-04-24T23:35:00.541110+0000","last_fullsized":"2024-04-24T23:35:00.541110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:50.222015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.541453+0000","last_change":"2024-04-24T23:34:58.399618+0000","last_active":"2024-04-24T23:35:00.541453+0000","last_peered":"2024-04-24T23:35:00.541453+0000","last_clean":"2024-04-24T23:35:00.541453+0000","last_became_active":"2024-04-24T23:34:58.399182+0000","last_became_peered":"2024-04-24T23:34:58.399182+0000","last_unstale":"2024-04-24T23:35:00.541453+0000","last_undegraded":"2024-04-24T23:35:00.541453+0000","last_fullsized":"2024-04-24T23:35:00.541453+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:55.672101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.395731+0000","last_change":"2024-04-24T23:34:54.369729+0000","last_active":"2024-04-24T23:34:59.395731+0000","last_peered":"2024-04-24T23:34:59.395731+0000","last_clean":"2024-04-24T23:34:59.395731+0000","last_became_active":"2024-04-24T23:34:54.369155+0000","last_became_peered":"2024-04-24T23:34:54.369155+0000","last_unstale":"2024-04-24T23:34:59.395731+0000","last_undegraded":"2024-04-24T23:34:59.395731+0000","last_fullsized":"2024-04-24T23:34:59.395731+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:46:28.181996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.395890+0000","last_change":"2024-04-24T23:34:53.365328+0000","last_active":"2024-04-24T23:34:59.395890+0000","last_peered":"2024-04-24T23:34:59.395890+0000","last_clean":"2024-04-24T23:34:59.395890+0000","last_became_active":"2024-04-24T23:34:53.365029+0000","last_became_peered":"2024-04-24T23:34:53.365029+0000","last_unstale":"2024-04-24T23:34:59.395890+0000","last_undegraded":"2024-04-24T23:34:59.395890+0000","last_fullsized":"2024-04-24T23:34:59.395890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:48:30.961870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540306+0000","last_change":"2024-04-24T23:34:53.365123+0000","last_active":"2024-04-24T23:35:00.540306+0000","last_peered":"2024-04-24T23:35:00.540306+0000","last_clean":"2024-04-24T23:35:00.540306+0000","last_became_active":"2024-04-24T23:34:53.364821+0000","last_became_peered":"2024-04-24T23:34:53.364821+0000","last_unstale":"2024-04-24T23:35:00.540306+0000","last_undegraded":"2024-04-24T23:35:00.540306+0000","last_fullsized":"2024-04-24T23:35:00.540306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:39.457116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.427901+0000","last_change":"2024-04-24T23:34:54.376037+0000","last_active":"2024-04-24T23:34:59.427901+0000","last_peered":"2024-04-24T23:34:59.427901+0000","last_clean":"2024-04-24T23:34:59.427901+0000","last_became_active":"2024-04-24T23:34:54.375689+0000","last_became_peered":"2024-04-24T23:34:54.375689+0000","last_unstale":"2024-04-24T23:34:59.427901+0000","last_undegraded":"2024-04-24T23:34:59.427901+0000","last_fullsized":"2024-04-24T23:34:59.427901+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:47.446459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397192+0000","last_change":"2024-04-24T23:34:58.404439+0000","last_active":"2024-04-24T23:34:59.397192+0000","last_peered":"2024-04-24T23:34:59.397192+0000","last_clean":"2024-04-24T23:34:59.397192+0000","last_became_active":"2024-04-24T23:34:58.404053+0000","last_became_peered":"2024-04-24T23:34:58.404053+0000","last_unstale":"2024-04-24T23:34:59.397192+0000","last_undegraded":"2024-04-24T23:34:59.397192+0000","last_fullsized":"2024-04-24T23:34:59.397192+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:46.292263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396770+0000","last_change":"2024-04-24T23:34:53.363066+0000","last_active":"2024-04-24T23:34:59.396770+0000","last_peered":"2024-04-24T23:34:59.396770+0000","last_clean":"2024-04-24T23:34:59.396770+0000","last_became_active":"2024-04-24T23:34:53.362548+0000","last_became_peered":"2024-04-24T23:34:53.362548+0000","last_unstale":"2024-04-24T23:34:59.396770+0000","last_undegraded":"2024-04-24T23:34:59.396770+0000","last_fullsized":"2024-04-24T23:34:59.396770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:21:13.338964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397349+0000","last_change":"2024-04-24T23:34:54.365704+0000","last_active":"2024-04-24T23:34:59.397349+0000","last_peered":"2024-04-24T23:34:59.397349+0000","last_clean":"2024-04-24T23:34:59.397349+0000","last_became_active":"2024-04-24T23:34:54.364814+0000","last_became_peered":"2024-04-24T23:34:54.364814+0000","last_unstale":"2024-04-24T23:34:59.397349+0000","last_undegraded":"2024-04-24T23:34:59.397349+0000","last_fullsized":"2024-04-24T23:34:59.397349+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:22.094785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397247+0000","last_change":"2024-04-24T23:34:58.407186+0000","last_active":"2024-04-24T23:34:59.397247+0000","last_peered":"2024-04-24T23:34:59.397247+0000","last_clean":"2024-04-24T23:34:59.397247+0000","last_became_active":"2024-04-24T23:34:58.406811+0000","last_became_peered":"2024-04-24T23:34:58.406811+0000","last_unstale":"2024-04-24T23:34:59.397247+0000","last_undegraded":"2024-04-24T23:34:59.397247+0000","last_fullsized":"2024-04-24T23:34:59.397247+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:55.861555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.539983+0000","last_change":"2024-04-24T23:34:54.375142+0000","last_active":"2024-04-24T23:35:00.539983+0000","last_peered":"2024-04-24T23:35:00.539983+0000","last_clean":"2024-04-24T23:35:00.539983+0000","last_became_active":"2024-04-24T23:34:54.374844+0000","last_became_peered":"2024-04-24T23:34:54.374844+0000","last_unstale":"2024-04-24T23:35:00.539983+0000","last_undegraded":"2024-04-24T23:35:00.539983+0000","last_fullsized":"2024-04-24T23:35:00.539983+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:29:07.593498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396456+0000","last_change":"2024-04-24T23:34:54.383574+0000","last_active":"2024-04-24T23:34:59.396456+0000","last_peered":"2024-04-24T23:34:59.396456+0000","last_clean":"2024-04-24T23:34:59.396456+0000","last_became_active":"2024-04-24T23:34:54.383253+0000","last_became_peered":"2024-04-24T23:34:54.383253+0000","last_unstale":"2024-04-24T23:34:59.396456+0000","last_undegraded":"2024-04-24T23:34:59.396456+0000","last_fullsized":"2024-04-24T23:34:59.396456+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:43.211314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540447+0000","last_change":"2024-04-24T23:34:54.372208+0000","last_active":"2024-04-24T23:35:00.540447+0000","last_peered":"2024-04-24T23:35:00.540447+0000","last_clean":"2024-04-24T23:35:00.540447+0000","last_became_active":"2024-04-24T23:34:54.371910+0000","last_became_peered":"2024-04-24T23:34:54.371910+0000","last_unstale":"2024-04-24T23:35:00.540447+0000","last_undegraded":"2024-04-24T23:35:00.540447+0000","last_fullsized":"2024-04-24T23:35:00.540447+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:07:28.785202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.427860+0000","last_change":"2024-04-24T23:34:54.366271+0000","last_active":"2024-04-24T23:34:59.427860+0000","last_peered":"2024-04-24T23:34:59.427860+0000","last_clean":"2024-04-24T23:34:59.427860+0000","last_became_active":"2024-04-24T23:34:54.366005+0000","last_became_peered":"2024-04-24T23:34:54.366005+0000","last_unstale":"2024-04-24T23:34:59.427860+0000","last_undegraded":"2024-04-24T23:34:59.427860+0000","last_fullsized":"2024-04-24T23:34:59.427860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:03:24.745895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397136+0000","last_change":"2024-04-24T23:34:54.360529+0000","last_active":"2024-04-24T23:34:59.397136+0000","last_peered":"2024-04-24T23:34:59.397136+0000","last_clean":"2024-04-24T23:34:59.397136+0000","last_became_active":"2024-04-24T23:34:54.360266+0000","last_became_peered":"2024-04-24T23:34:54.360266+0000","last_unstale":"2024-04-24T23:34:59.397136+0000","last_undegraded":"2024-04-24T23:34:59.397136+0000","last_fullsized":"2024-04-24T23:34:59.397136+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:49:47.246193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.881089+0000","last_change":"2024-04-24T23:34:54.365744+0000","last_active":"2024-04-24T23:34:59.881089+0000","last_peered":"2024-04-24T23:34:59.881089+0000","last_clean":"2024-04-24T23:34:59.881089+0000","last_became_active":"2024-04-24T23:34:54.365493+0000","last_became_peered":"2024-04-24T23:34:54.365493+0000","last_unstale":"2024-04-24T23:34:59.881089+0000","last_undegraded":"2024-04-24T23:34:59.881089+0000","last_fullsized":"2024-04-24T23:34:59.881089+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:15.477061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.541221+0000","last_change":"2024-04-24T23:34:54.379642+0000","last_active":"2024-04-24T23:35:00.541221+0000","last_peered":"2024-04-24T23:35:00.541221+0000","last_clean":"2024-04-24T23:35:00.541221+0000","last_became_active":"2024-04-24T23:34:54.378568+0000","last_became_peered":"2024-04-24T23:34:54.378568+0000","last_unstale":"2024-04-24T23:35:00.541221+0000","last_undegraded":"2024-04-24T23:35:00.541221+0000","last_fullsized":"2024-04-24T23:35:00.541221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:42.199856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397390+0000","last_change":"2024-04-24T23:34:54.363700+0000","last_active":"2024-04-24T23:34:59.397390+0000","last_peered":"2024-04-24T23:34:59.397390+0000","last_clean":"2024-04-24T23:34:59.397390+0000","last_became_active":"2024-04-24T23:34:54.363439+0000","last_became_peered":"2024-04-24T23:34:54.363439+0000","last_unstale":"2024-04-24T23:34:59.397390+0000","last_undegraded":"2024-04-24T23:34:59.397390+0000","last_fullsized":"2024-04-24T23:34:59.397390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:28:56.048178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.541000+0000","last_change":"2024-04-24T23:34:54.375460+0000","last_active":"2024-04-24T23:35:00.541000+0000","last_peered":"2024-04-24T23:35:00.541000+0000","last_clean":"2024-04-24T23:35:00.541000+0000","last_became_active":"2024-04-24T23:34:54.375123+0000","last_became_peered":"2024-04-24T23:34:54.375123+0000","last_unstale":"2024-04-24T23:35:00.541000+0000","last_undegraded":"2024-04-24T23:35:00.541000+0000","last_fullsized":"2024-04-24T23:35:00.541000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:18.207267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396028+0000","last_change":"2024-04-24T23:34:54.369509+0000","last_active":"2024-04-24T23:34:59.396028+0000","last_peered":"2024-04-24T23:34:59.396028+0000","last_clean":"2024-04-24T23:34:59.396028+0000","last_became_active":"2024-04-24T23:34:54.369219+0000","last_became_peered":"2024-04-24T23:34:54.369219+0000","last_unstale":"2024-04-24T23:34:59.396028+0000","last_undegraded":"2024-04-24T23:34:59.396028+0000","last_fullsized":"2024-04-24T23:34:59.396028+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:09:37.649503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540426+0000","last_change":"2024-04-24T23:34:54.384240+0000","last_active":"2024-04-24T23:35:00.540426+0000","last_peered":"2024-04-24T23:35:00.540426+0000","last_clean":"2024-04-24T23:35:00.540426+0000","last_became_active":"2024-04-24T23:34:54.383780+0000","last_became_peered":"2024-04-24T23:34:54.383780+0000","last_unstale":"2024-04-24T23:35:00.540426+0000","last_undegraded":"2024-04-24T23:35:00.540426+0000","last_fullsized":"2024-04-24T23:35:00.540426+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:49.048141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.427635+0000","last_change":"2024-04-24T23:34:54.371052+0000","last_active":"2024-04-24T23:34:59.427635+0000","last_peered":"2024-04-24T23:34:59.427635+0000","last_clean":"2024-04-24T23:34:59.427635+0000","last_became_active":"2024-04-24T23:34:54.370793+0000","last_became_peered":"2024-04-24T23:34:54.370793+0000","last_unstale":"2024-04-24T23:34:59.427635+0000","last_undegraded":"2024-04-24T23:34:59.427635+0000","last_fullsized":"2024-04-24T23:34:59.427635+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:29:07.593498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540693+0000","last_change":"2024-04-24T23:34:54.377190+0000","last_active":"2024-04-24T23:35:00.540693+0000","last_peered":"2024-04-24T23:35:00.540693+0000","last_clean":"2024-04-24T23:35:00.540693+0000","last_became_active":"2024-04-24T23:34:54.376940+0000","last_became_peered":"2024-04-24T23:34:54.376940+0000","last_unstale":"2024-04-24T23:35:00.540693+0000","last_undegraded":"2024-04-24T23:35:00.540693+0000","last_fullsized":"2024-04-24T23:35:00.540693+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:03:24.745895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501917+0000","last_change":"2024-04-24T23:34:54.368770+0000","last_active":"2024-04-24T23:34:59.501917+0000","last_peered":"2024-04-24T23:34:59.501917+0000","last_clean":"2024-04-24T23:34:59.501917+0000","last_became_active":"2024-04-24T23:34:54.368228+0000","last_became_peered":"2024-04-24T23:34:54.368228+0000","last_unstale":"2024-04-24T23:34:59.501917+0000","last_undegraded":"2024-04-24T23:34:59.501917+0000","last_fullsized":"2024-04-24T23:34:59.501917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:51.634876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396819+0000","last_change":"2024-04-24T23:34:54.373038+0000","last_active":"2024-04-24T23:34:59.396819+0000","last_peered":"2024-04-24T23:34:59.396819+0000","last_clean":"2024-04-24T23:34:59.396819+0000","last_became_active":"2024-04-24T23:34:54.372766+0000","last_became_peered":"2024-04-24T23:34:54.372766+0000","last_unstale":"2024-04-24T23:34:59.396819+0000","last_undegraded":"2024-04-24T23:34:59.396819+0000","last_fullsized":"2024-04-24T23:34:59.396819+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:13.263529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.541061+0000","last_change":"2024-04-24T23:34:54.383733+0000","last_active":"2024-04-24T23:35:00.541061+0000","last_peered":"2024-04-24T23:35:00.541061+0000","last_clean":"2024-04-24T23:35:00.541061+0000","last_became_active":"2024-04-24T23:34:54.383463+0000","last_became_peered":"2024-04-24T23:34:54.383463+0000","last_unstale":"2024-04-24T23:35:00.541061+0000","last_undegraded":"2024-04-24T23:35:00.541061+0000","last_fullsized":"2024-04-24T23:35:00.541061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:51.634876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540883+0000","last_change":"2024-04-24T23:34:54.380959+0000","last_active":"2024-04-24T23:35:00.540883+0000","last_peered":"2024-04-24T23:35:00.540883+0000","last_clean":"2024-04-24T23:35:00.540883+0000","last_became_active":"2024-04-24T23:34:54.380702+0000","last_became_peered":"2024-04-24T23:34:54.380702+0000","last_unstale":"2024-04-24T23:35:00.540883+0000","last_undegraded":"2024-04-24T23:35:00.540883+0000","last_fullsized":"2024-04-24T23:35:00.540883+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:45.424046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.880840+0000","last_change":"2024-04-24T23:34:54.358461+0000","last_active":"2024-04-24T23:34:59.880840+0000","last_peered":"2024-04-24T23:34:59.880840+0000","last_clean":"2024-04-24T23:34:59.880840+0000","last_became_active":"2024-04-24T23:34:54.358203+0000","last_became_peered":"2024-04-24T23:34:54.358203+0000","last_unstale":"2024-04-24T23:34:59.880840+0000","last_undegraded":"2024-04-24T23:34:59.880840+0000","last_fullsized":"2024-04-24T23:34:59.880840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:18.207267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.427594+0000","last_change":"2024-04-24T23:34:54.367860+0000","last_active":"2024-04-24T23:34:59.427594+0000","last_peered":"2024-04-24T23:34:59.427594+0000","last_clean":"2024-04-24T23:34:59.427594+0000","last_became_active":"2024-04-24T23:34:54.367595+0000","last_became_peered":"2024-04-24T23:34:54.367595+0000","last_unstale":"2024-04-24T23:34:59.427594+0000","last_undegraded":"2024-04-24T23:34:59.427594+0000","last_fullsized":"2024-04-24T23:34:59.427594+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:40:40.928386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396558+0000","last_change":"2024-04-24T23:34:54.373640+0000","last_active":"2024-04-24T23:34:59.396558+0000","last_peered":"2024-04-24T23:34:59.396558+0000","last_clean":"2024-04-24T23:34:59.396558+0000","last_became_active":"2024-04-24T23:34:54.373404+0000","last_became_peered":"2024-04-24T23:34:54.373404+0000","last_unstale":"2024-04-24T23:34:59.396558+0000","last_undegraded":"2024-04-24T23:34:59.396558+0000","last_fullsized":"2024-04-24T23:34:59.396558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:41:21.841042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397052+0000","last_change":"2024-04-24T23:34:54.377573+0000","last_active":"2024-04-24T23:34:59.397052+0000","last_peered":"2024-04-24T23:34:59.397052+0000","last_clean":"2024-04-24T23:34:59.397052+0000","last_became_active":"2024-04-24T23:34:54.377277+0000","last_became_peered":"2024-04-24T23:34:54.377277+0000","last_unstale":"2024-04-24T23:34:59.397052+0000","last_undegraded":"2024-04-24T23:34:59.397052+0000","last_fullsized":"2024-04-24T23:34:59.397052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:43.211314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540693+0000","last_change":"2024-04-24T23:34:54.388134+0000","last_active":"2024-04-24T23:35:00.540693+0000","last_peered":"2024-04-24T23:35:00.540693+0000","last_clean":"2024-04-24T23:35:00.540693+0000","last_became_active":"2024-04-24T23:34:54.387781+0000","last_became_peered":"2024-04-24T23:34:54.387781+0000","last_unstale":"2024-04-24T23:35:00.540693+0000","last_undegraded":"2024-04-24T23:35:00.540693+0000","last_fullsized":"2024-04-24T23:35:00.540693+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:56:17.763159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.541198+0000","last_change":"2024-04-24T23:34:54.388182+0000","last_active":"2024-04-24T23:35:00.541198+0000","last_peered":"2024-04-24T23:35:00.541198+0000","last_clean":"2024-04-24T23:35:00.541198+0000","last_became_active":"2024-04-24T23:34:54.387913+0000","last_became_peered":"2024-04-24T23:34:54.387913+0000","last_unstale":"2024-04-24T23:35:00.541198+0000","last_undegraded":"2024-04-24T23:35:00.541198+0000","last_fullsized":"2024-04-24T23:35:00.541198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:18.207267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.502012+0000","last_change":"2024-04-24T23:34:54.383303+0000","last_active":"2024-04-24T23:34:59.502012+0000","last_peered":"2024-04-24T23:34:59.502012+0000","last_clean":"2024-04-24T23:34:59.502012+0000","last_became_active":"2024-04-24T23:34:54.383024+0000","last_became_peered":"2024-04-24T23:34:54.383024+0000","last_unstale":"2024-04-24T23:34:59.502012+0000","last_undegraded":"2024-04-24T23:34:59.502012+0000","last_fullsized":"2024-04-24T23:34:59.502012+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:45.424046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.395629+0000","last_change":"2024-04-24T23:34:54.382803+0000","last_active":"2024-04-24T23:34:59.395629+0000","last_peered":"2024-04-24T23:34:59.395629+0000","last_clean":"2024-04-24T23:34:59.395629+0000","last_became_active":"2024-04-24T23:34:54.382482+0000","last_became_peered":"2024-04-24T23:34:54.382482+0000","last_unstale":"2024-04-24T23:34:59.395629+0000","last_undegraded":"2024-04-24T23:34:59.395629+0000","last_fullsized":"2024-04-24T23:34:59.395629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:27:33.721584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501319+0000","last_change":"2024-04-24T23:34:54.369167+0000","last_active":"2024-04-24T23:34:59.501319+0000","last_peered":"2024-04-24T23:34:59.501319+0000","last_clean":"2024-04-24T23:34:59.501319+0000","last_became_active":"2024-04-24T23:34:54.368663+0000","last_became_peered":"2024-04-24T23:34:54.368663+0000","last_unstale":"2024-04-24T23:34:59.501319+0000","last_undegraded":"2024-04-24T23:34:59.501319+0000","last_fullsized":"2024-04-24T23:34:59.501319+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:18.207267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540351+0000","last_change":"2024-04-24T23:34:54.380345+0000","last_active":"2024-04-24T23:35:00.540351+0000","last_peered":"2024-04-24T23:35:00.540351+0000","last_clean":"2024-04-24T23:35:00.540351+0000","last_became_active":"2024-04-24T23:34:54.380052+0000","last_became_peered":"2024-04-24T23:34:54.380052+0000","last_unstale":"2024-04-24T23:35:00.540351+0000","last_undegraded":"2024-04-24T23:35:00.540351+0000","last_fullsized":"2024-04-24T23:35:00.540351+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:42.199856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501438+0000","last_change":"2024-04-24T23:34:54.365114+0000","last_active":"2024-04-24T23:34:59.501438+0000","last_peered":"2024-04-24T23:34:59.501438+0000","last_clean":"2024-04-24T23:34:59.501438+0000","last_became_active":"2024-04-24T23:34:54.364605+0000","last_became_peered":"2024-04-24T23:34:54.364605+0000","last_unstale":"2024-04-24T23:34:59.501438+0000","last_undegraded":"2024-04-24T23:34:59.501438+0000","last_fullsized":"2024-04-24T23:34:59.501438+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:07:28.785202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540539+0000","last_change":"2024-04-24T23:34:54.374843+0000","last_active":"2024-04-24T23:35:00.540539+0000","last_peered":"2024-04-24T23:35:00.540539+0000","last_clean":"2024-04-24T23:35:00.540539+0000","last_became_active":"2024-04-24T23:34:54.373117+0000","last_became_peered":"2024-04-24T23:34:54.373117+0000","last_unstale":"2024-04-24T23:35:00.540539+0000","last_undegraded":"2024-04-24T23:35:00.540539+0000","last_fullsized":"2024-04-24T23:35:00.540539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:37.323551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.427656+0000","last_change":"2024-04-24T23:34:54.366079+0000","last_active":"2024-04-24T23:34:59.427656+0000","last_peered":"2024-04-24T23:34:59.427656+0000","last_clean":"2024-04-24T23:34:59.427656+0000","last_became_active":"2024-04-24T23:34:54.365814+0000","last_became_peered":"2024-04-24T23:34:54.365814+0000","last_unstale":"2024-04-24T23:34:59.427656+0000","last_undegraded":"2024-04-24T23:34:59.427656+0000","last_fullsized":"2024-04-24T23:34:59.427656+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:18.207267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.881393+0000","last_change":"2024-04-24T23:34:54.368976+0000","last_active":"2024-04-24T23:34:59.881393+0000","last_peered":"2024-04-24T23:34:59.881393+0000","last_clean":"2024-04-24T23:34:59.881393+0000","last_became_active":"2024-04-24T23:34:54.368663+0000","last_became_peered":"2024-04-24T23:34:54.368663+0000","last_unstale":"2024-04-24T23:34:59.881393+0000","last_undegraded":"2024-04-24T23:34:59.881393+0000","last_fullsized":"2024-04-24T23:34:59.881393+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:56:17.763159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501905+0000","last_change":"2024-04-24T23:34:54.373810+0000","last_active":"2024-04-24T23:34:59.501905+0000","last_peered":"2024-04-24T23:34:59.501905+0000","last_clean":"2024-04-24T23:34:59.501905+0000","last_became_active":"2024-04-24T23:34:54.373531+0000","last_became_peered":"2024-04-24T23:34:54.373531+0000","last_unstale":"2024-04-24T23:34:59.501905+0000","last_undegraded":"2024-04-24T23:34:59.501905+0000","last_fullsized":"2024-04-24T23:34:59.501905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:50:17.621315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65248,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":331776,"data_stored":694125,"data_compressed":32624,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542151,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":274432,"data_stored":107017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":103265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":103301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":262144,"data_stored":103175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31444,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340396,"statfs":{"total":96636764160,"available":96604565504,"internally_reserved":0,"allocated":270336,"data_stored":103335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31520,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340320,"statfs":{"total":96636764160,"available":96604487680,"internally_reserved":0,"allocated":339968,"data_stored":694181,"data_compressed":32624,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31444,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340396,"statfs":{"total":96636764160,"available":96604565504,"internally_reserved":0,"allocated":274432,"data_stored":107017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32624,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32624,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:35:17.414 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:35:17.546 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.545+0000 7efd9c310640 1 Processor -- start 2024-04-24T23:35:17.546 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.545+0000 7efd9c310640 1 -- start start 2024-04-24T23:35:17.548 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.546+0000 7efd9c310640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7efd94152b40 0x7efd940c5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:17.548 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.546+0000 7efd9c310640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7efd940c5b70 0x7efd94155a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:17.548 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.546+0000 7efd9c310640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd940c8bb0 con 0x7efd940be360 2024-04-24T23:35:17.548 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.546+0000 7efd9c310640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7efd940c8e90 con 0x7efd940c5b70 2024-04-24T23:35:17.548 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.546+0000 7efd9c310640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7efd94159480 con 0x7efd94152b40 2024-04-24T23:35:17.548 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.546+0000 7efd9a8ad640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7efd940c5b70 0x7efd94155a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:17.549 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.546+0000 7efd9a8ad640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7efd940c5b70 0x7efd94155a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42512/0 (socket says 172.21.15.157:42512) 2024-04-24T23:35:17.549 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.546+0000 7efd998ab640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7efd940be360 0x7efd94155350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:47616/0 (socket says 172.21.15.157:47616) 2024-04-24T23:35:17.549 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.546+0000 7efd9a8ad640 1 -- 172.21.15.157:0/28041472 learned_addr learned my addr 172.21.15.157:0/28041472 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:17.549 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.547+0000 7efd9a0ac640 1 --2- 172.21.15.157:0/28041472 >> v2:172.21.15.169:3300/0 conn(0x7efd94152b40 0x7efd940c5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:17.549 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.547+0000 7efd9a8ad640 1 -- 172.21.15.157:0/28041472 >> v2:172.21.15.169:3300/0 conn(0x7efd94152b40 msgr2=0x7efd940c5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:17.549 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.547+0000 7efd9a8ad640 1 --2- 172.21.15.157:0/28041472 >> v2:172.21.15.169:3300/0 conn(0x7efd94152b40 0x7efd940c5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:17.549 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.547+0000 7efd9a8ad640 1 -- 172.21.15.157:0/28041472 >> v1:172.21.15.157:6789/0 conn(0x7efd940be360 legacy=0x7efd94155350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:17.549 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.547+0000 7efd990aa640 1 -- 172.21.15.157:0/28041472 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 673559739 0 0) 0x7efd940c8bb0 con 0x7efd940be360 2024-04-24T23:35:17.549 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.547+0000 7efd9a8ad640 1 -- 172.21.15.157:0/28041472 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd94159760 con 0x7efd940c5b70 2024-04-24T23:35:17.549 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.547+0000 7efd9a0ac640 1 --2- 172.21.15.157:0/28041472 >> v2:172.21.15.169:3300/0 conn(0x7efd94152b40 0x7efd940c5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:35:17.549 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.547+0000 7efd9a8ad640 1 --2- 172.21.15.157:0/28041472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd940c5b70 0x7efd94155a60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7efd9406cce0 tx=0x7efd8c02a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0e40346cf9a10c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:17.549 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.547+0000 7efd990aa640 1 -- 172.21.15.157:0/28041472 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8c037070 con 0x7efd940c5b70 2024-04-24T23:35:17.549 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.548+0000 7efd990aa640 1 -- 172.21.15.157:0/28041472 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd8c032440 con 0x7efd940c5b70 2024-04-24T23:35:17.549 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.548+0000 7efd990aa640 1 -- 172.21.15.157:0/28041472 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8c032810 con 0x7efd940c5b70 2024-04-24T23:35:17.552 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.548+0000 7efd9c310640 1 -- 172.21.15.157:0/28041472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd940c5b70 msgr2=0x7efd94155a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:17.552 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.548+0000 7efd9c310640 1 --2- 172.21.15.157:0/28041472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd940c5b70 0x7efd94155a60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7efd9406cce0 tx=0x7efd8c02a150 comp rx=0 tx=0).stop 2024-04-24T23:35:17.552 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.548+0000 7efd9c310640 1 -- 172.21.15.157:0/28041472 shutdown_connections 2024-04-24T23:35:17.552 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.548+0000 7efd9c310640 1 --2- 172.21.15.157:0/28041472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd940c5b70 0x7efd94155a60 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:17.552 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.548+0000 7efd9c310640 1 --2- 172.21.15.157:0/28041472 >> v2:172.21.15.169:3300/0 conn(0x7efd94152b40 0x7efd940c5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:17.553 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.548+0000 7efd9c310640 1 -- 172.21.15.157:0/28041472 >> 172.21.15.157:0/28041472 conn(0x7efd940672c0 msgr2=0x7efd940676d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:17.553 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.548+0000 7efd9c310640 1 -- 172.21.15.157:0/28041472 shutdown_connections 2024-04-24T23:35:17.553 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.548+0000 7efd9c310640 1 -- 172.21.15.157:0/28041472 wait complete. 2024-04-24T23:35:17.553 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.549+0000 7efd9c310640 1 Processor -- start 2024-04-24T23:35:17.553 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.549+0000 7efd9c310640 1 -- start start 2024-04-24T23:35:17.553 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.550+0000 7efd9c310640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd940be360 0x7efd940c1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:17.553 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.550+0000 7efd9c310640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd940c5b70 0x7efd941ee850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:17.553 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.550+0000 7efd9c310640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd94152b40 0x7efd94149a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:17.553 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.550+0000 7efd9c310640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7efd940caa00 con 0x7efd94152b40 2024-04-24T23:35:17.553 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.550+0000 7efd9c310640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efd940cb210 con 0x7efd940c5b70 2024-04-24T23:35:17.553 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.550+0000 7efd9c310640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efd940cba40 con 0x7efd940be360 2024-04-24T23:35:17.553 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.550+0000 7efd9a8ad640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd94152b40 0x7efd94149a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:17.553 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.550+0000 7efd9a8ad640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd94152b40 0x7efd94149a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42522/0 (socket says 172.21.15.157:42522) 2024-04-24T23:35:17.554 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.550+0000 7efd9a8ad640 1 -- 172.21.15.157:0/1470886940 learned_addr learned my addr 172.21.15.157:0/1470886940 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:17.554 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.550+0000 7efd998ab640 1 --2- 172.21.15.157:0/1470886940 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd940c5b70 0x7efd941ee850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:17.554 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.550+0000 7efd9a0ac640 1 --2- 172.21.15.157:0/1470886940 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd940be360 0x7efd940c1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:17.554 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.550+0000 7efd9a8ad640 1 -- 172.21.15.157:0/1470886940 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd940be360 msgr2=0x7efd940c1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:17.554 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.550+0000 7efd9a8ad640 1 --2- 172.21.15.157:0/1470886940 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd940be360 0x7efd940c1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:17.554 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.550+0000 7efd9a8ad640 1 -- 172.21.15.157:0/1470886940 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd940c5b70 msgr2=0x7efd941ee850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:17.554 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.550+0000 7efd9a8ad640 1 --2- 172.21.15.157:0/1470886940 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd940c5b70 0x7efd941ee850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:17.554 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.550+0000 7efd9a8ad640 1 -- 172.21.15.157:0/1470886940 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd941ef090 con 0x7efd94152b40 2024-04-24T23:35:17.554 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.550+0000 7efd998ab640 1 --2- 172.21.15.157:0/1470886940 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd940c5b70 0x7efd941ee850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:35:17.554 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.550+0000 7efd9a8ad640 1 --2- 172.21.15.157:0/1470886940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd94152b40 0x7efd94149a60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7efd8c035040 tx=0x7efd8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:17.554 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.551+0000 7efd8affd640 1 -- 172.21.15.157:0/1470886940 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8c037040 con 0x7efd94152b40 2024-04-24T23:35:17.554 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.551+0000 7efd8affd640 1 -- 172.21.15.157:0/1470886940 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd8c004230 con 0x7efd94152b40 2024-04-24T23:35:17.554 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.551+0000 7efd8affd640 1 -- 172.21.15.157:0/1470886940 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8c003610 con 0x7efd94152b40 2024-04-24T23:35:17.555 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.551+0000 7efd9c310640 1 -- 172.21.15.157:0/1470886940 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd941ef330 con 0x7efd94152b40 2024-04-24T23:35:17.555 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.551+0000 7efd9c310640 1 -- 172.21.15.157:0/1470886940 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd941eed90 con 0x7efd94152b40 2024-04-24T23:35:17.555 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.552+0000 7efd9c310640 1 -- 172.21.15.157:0/1470886940 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd940681c0 con 0x7efd94152b40 2024-04-24T23:35:17.555 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.552+0000 7efd8affd640 1 -- 172.21.15.157:0/1470886940 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efd8c0037b0 con 0x7efd94152b40 2024-04-24T23:35:17.555 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.552+0000 7efd8affd640 1 --2- 172.21.15.157:0/1470886940 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7efd7806be30 0x7efd7806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:17.556 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.552+0000 7efd8affd640 1 -- 172.21.15.157:0/1470886940 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7efd8c0abf80 con 0x7efd94152b40 2024-04-24T23:35:17.556 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.553+0000 7efd9a0ac640 1 --2- 172.21.15.157:0/1470886940 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7efd7806be30 0x7efd7806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:17.556 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.553+0000 7efd9a0ac640 1 --2- 172.21.15.157:0/1470886940 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7efd7806be30 0x7efd7806e2f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7efd90003fd0 tx=0x7efd900039f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:17.557 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.557+0000 7efd8affd640 1 -- 172.21.15.157:0/1470886940 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7efd8c07a680 con 0x7efd94152b40 2024-04-24T23:35:17.727 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.726+0000 7efd9c310640 1 -- 172.21.15.157:0/1470886940 --> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efd94156520 con 0x7efd7806be30 2024-04-24T23:35:17.735 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.735+0000 7efd8affd640 1 -- 172.21.15.157:0/1470886940 <== mgr.4105 v2:172.21.15.169:6832/662378289 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332611 (secure 0 0 0) 0x7efd94156520 con 0x7efd7806be30 2024-04-24T23:35:17.739 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:35:17.740 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2024-04-24T23:35:17.742 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.738+0000 7efd9c310640 1 -- 172.21.15.157:0/1470886940 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7efd7806be30 msgr2=0x7efd7806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:17.742 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.738+0000 7efd9c310640 1 --2- 172.21.15.157:0/1470886940 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7efd7806be30 0x7efd7806e2f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7efd90003fd0 tx=0x7efd900039f0 comp rx=0 tx=0).stop 2024-04-24T23:35:17.742 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.738+0000 7efd9c310640 1 -- 172.21.15.157:0/1470886940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd94152b40 msgr2=0x7efd94149a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:17.742 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.738+0000 7efd9c310640 1 --2- 172.21.15.157:0/1470886940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd94152b40 0x7efd94149a60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7efd8c035040 tx=0x7efd8c0046c0 comp rx=0 tx=0).stop 2024-04-24T23:35:17.742 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.738+0000 7efd9c310640 1 -- 172.21.15.157:0/1470886940 shutdown_connections 2024-04-24T23:35:17.742 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.738+0000 7efd9c310640 1 --2- 172.21.15.157:0/1470886940 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7efd7806be30 0x7efd7806e2f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:17.743 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.738+0000 7efd9c310640 1 --2- 172.21.15.157:0/1470886940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd94152b40 0x7efd94149a60 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:17.743 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.738+0000 7efd9c310640 1 --2- 172.21.15.157:0/1470886940 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd940c5b70 0x7efd941ee850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:17.743 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.738+0000 7efd9c310640 1 --2- 172.21.15.157:0/1470886940 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd940be360 0x7efd940c1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:17.743 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.738+0000 7efd9c310640 1 -- 172.21.15.157:0/1470886940 >> 172.21.15.157:0/1470886940 conn(0x7efd940672c0 msgr2=0x7efd9414ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:17.743 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.738+0000 7efd9c310640 1 -- 172.21.15.157:0/1470886940 shutdown_connections 2024-04-24T23:35:17.743 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.738+0000 7efd9c310640 1 -- 172.21.15.157:0/1470886940 wait complete. 2024-04-24T23:35:17.750 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2024-04-24T23:35:15.771641+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235052,"kb_used_data":2240,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739668,"statfs":{"total":773094113280,"available":772853420032,"internally_reserved":0,"allocated":2293760,"data_stored":2015416,"data_compressed":65248,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003169"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.428046+0000","last_change":"2024-04-24T23:34:53.362409+0000","last_active":"2024-04-24T23:34:59.428046+0000","last_peered":"2024-04-24T23:34:59.428046+0000","last_clean":"2024-04-24T23:34:59.428046+0000","last_became_active":"2024-04-24T23:34:53.362104+0000","last_became_peered":"2024-04-24T23:34:53.362104+0000","last_unstale":"2024-04-24T23:34:59.428046+0000","last_undegraded":"2024-04-24T23:34:59.428046+0000","last_fullsized":"2024-04-24T23:34:59.428046+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:53.159219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396596+0000","last_change":"2024-04-24T23:34:54.373560+0000","last_active":"2024-04-24T23:34:59.396596+0000","last_peered":"2024-04-24T23:34:59.396596+0000","last_clean":"2024-04-24T23:34:59.396596+0000","last_became_active":"2024-04-24T23:34:54.373289+0000","last_became_peered":"2024-04-24T23:34:54.373289+0000","last_unstale":"2024-04-24T23:34:59.396596+0000","last_undegraded":"2024-04-24T23:34:59.396596+0000","last_fullsized":"2024-04-24T23:34:59.396596+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:31.447899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.395661+0000","last_change":"2024-04-24T23:34:58.730165+0000","last_active":"2024-04-24T23:34:59.395661+0000","last_peered":"2024-04-24T23:34:59.395661+0000","last_clean":"2024-04-24T23:34:59.395661+0000","last_became_active":"2024-04-24T23:34:58.729816+0000","last_became_peered":"2024-04-24T23:34:58.729816+0000","last_unstale":"2024-04-24T23:34:59.395661+0000","last_undegraded":"2024-04-24T23:34:59.395661+0000","last_fullsized":"2024-04-24T23:34:59.395661+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:02:02.335135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396776+0000","last_change":"2024-04-24T23:34:53.361650+0000","last_active":"2024-04-24T23:34:59.396776+0000","last_peered":"2024-04-24T23:34:59.396776+0000","last_clean":"2024-04-24T23:34:59.396776+0000","last_became_active":"2024-04-24T23:34:53.361366+0000","last_became_peered":"2024-04-24T23:34:53.361366+0000","last_unstale":"2024-04-24T23:34:59.396776+0000","last_undegraded":"2024-04-24T23:34:59.396776+0000","last_fullsized":"2024-04-24T23:34:59.396776+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:56:15.784223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.427859+0000","last_change":"2024-04-24T23:34:54.383602+0000","last_active":"2024-04-24T23:34:59.427859+0000","last_peered":"2024-04-24T23:34:59.427859+0000","last_clean":"2024-04-24T23:34:59.427859+0000","last_became_active":"2024-04-24T23:34:54.383160+0000","last_became_peered":"2024-04-24T23:34:54.383160+0000","last_unstale":"2024-04-24T23:34:59.427859+0000","last_undegraded":"2024-04-24T23:34:59.427859+0000","last_fullsized":"2024-04-24T23:34:59.427859+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:15.477061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396642+0000","last_change":"2024-04-24T23:34:58.404416+0000","last_active":"2024-04-24T23:34:59.396642+0000","last_peered":"2024-04-24T23:34:59.396642+0000","last_clean":"2024-04-24T23:34:59.396642+0000","last_became_active":"2024-04-24T23:34:58.403984+0000","last_became_peered":"2024-04-24T23:34:58.403984+0000","last_unstale":"2024-04-24T23:34:59.396642+0000","last_undegraded":"2024-04-24T23:34:59.396642+0000","last_fullsized":"2024-04-24T23:34:59.396642+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:46.237082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501886+0000","last_change":"2024-04-24T23:34:53.366730+0000","last_active":"2024-04-24T23:34:59.501886+0000","last_peered":"2024-04-24T23:34:59.501886+0000","last_clean":"2024-04-24T23:34:59.501886+0000","last_became_active":"2024-04-24T23:34:53.366430+0000","last_became_peered":"2024-04-24T23:34:53.366430+0000","last_unstale":"2024-04-24T23:34:59.501886+0000","last_undegraded":"2024-04-24T23:34:59.501886+0000","last_fullsized":"2024-04-24T23:34:59.501886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:41.422591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540654+0000","last_change":"2024-04-24T23:34:54.373592+0000","last_active":"2024-04-24T23:35:00.540654+0000","last_peered":"2024-04-24T23:35:00.540654+0000","last_clean":"2024-04-24T23:35:00.540654+0000","last_became_active":"2024-04-24T23:34:54.372844+0000","last_became_peered":"2024-04-24T23:34:54.372844+0000","last_unstale":"2024-04-24T23:35:00.540654+0000","last_undegraded":"2024-04-24T23:35:00.540654+0000","last_fullsized":"2024-04-24T23:35:00.540654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:49.372040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396779+0000","last_change":"2024-04-24T23:34:58.399224+0000","last_active":"2024-04-24T23:34:59.396779+0000","last_peered":"2024-04-24T23:34:59.396779+0000","last_clean":"2024-04-24T23:34:59.396779+0000","last_became_active":"2024-04-24T23:34:58.397817+0000","last_became_peered":"2024-04-24T23:34:58.397817+0000","last_unstale":"2024-04-24T23:34:59.396779+0000","last_undegraded":"2024-04-24T23:34:59.396779+0000","last_fullsized":"2024-04-24T23:34:59.396779+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:53.085366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501568+0000","last_change":"2024-04-24T23:34:53.362994+0000","last_active":"2024-04-24T23:34:59.501568+0000","last_peered":"2024-04-24T23:34:59.501568+0000","last_clean":"2024-04-24T23:34:59.501568+0000","last_became_active":"2024-04-24T23:34:53.362337+0000","last_became_peered":"2024-04-24T23:34:53.362337+0000","last_unstale":"2024-04-24T23:34:59.501568+0000","last_undegraded":"2024-04-24T23:34:59.501568+0000","last_fullsized":"2024-04-24T23:34:59.501568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:12.258211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.541217+0000","last_change":"2024-04-24T23:34:54.379218+0000","last_active":"2024-04-24T23:35:00.541217+0000","last_peered":"2024-04-24T23:35:00.541217+0000","last_clean":"2024-04-24T23:35:00.541217+0000","last_became_active":"2024-04-24T23:34:54.376019+0000","last_became_peered":"2024-04-24T23:34:54.376019+0000","last_unstale":"2024-04-24T23:35:00.541217+0000","last_undegraded":"2024-04-24T23:35:00.541217+0000","last_fullsized":"2024-04-24T23:35:00.541217+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:41:48.207542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396752+0000","last_change":"2024-04-24T23:34:58.397152+0000","last_active":"2024-04-24T23:34:59.396752+0000","last_peered":"2024-04-24T23:34:59.396752+0000","last_clean":"2024-04-24T23:34:59.396752+0000","last_became_active":"2024-04-24T23:34:58.396658+0000","last_became_peered":"2024-04-24T23:34:58.396658+0000","last_unstale":"2024-04-24T23:34:59.396752+0000","last_undegraded":"2024-04-24T23:34:59.396752+0000","last_fullsized":"2024-04-24T23:34:59.396752+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:41.360776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540212+0000","last_change":"2024-04-24T23:34:54.377859+0000","last_active":"2024-04-24T23:35:00.540212+0000","last_peered":"2024-04-24T23:35:00.540212+0000","last_clean":"2024-04-24T23:35:00.540212+0000","last_became_active":"2024-04-24T23:34:54.377576+0000","last_became_peered":"2024-04-24T23:34:54.377576+0000","last_unstale":"2024-04-24T23:35:00.540212+0000","last_undegraded":"2024-04-24T23:35:00.540212+0000","last_fullsized":"2024-04-24T23:35:00.540212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:47.446459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501625+0000","last_change":"2024-04-24T23:34:53.360903+0000","last_active":"2024-04-24T23:34:59.501625+0000","last_peered":"2024-04-24T23:34:59.501625+0000","last_clean":"2024-04-24T23:34:59.501625+0000","last_became_active":"2024-04-24T23:34:53.360635+0000","last_became_peered":"2024-04-24T23:34:53.360635+0000","last_unstale":"2024-04-24T23:34:59.501625+0000","last_undegraded":"2024-04-24T23:34:59.501625+0000","last_fullsized":"2024-04-24T23:34:59.501625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:42.205997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397132+0000","last_change":"2024-04-24T23:34:58.407146+0000","last_active":"2024-04-24T23:34:59.397132+0000","last_peered":"2024-04-24T23:34:59.397132+0000","last_clean":"2024-04-24T23:34:59.397132+0000","last_became_active":"2024-04-24T23:34:58.406844+0000","last_became_peered":"2024-04-24T23:34:58.406844+0000","last_unstale":"2024-04-24T23:34:59.397132+0000","last_undegraded":"2024-04-24T23:34:59.397132+0000","last_fullsized":"2024-04-24T23:34:59.397132+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:41:52.244767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396642+0000","last_change":"2024-04-24T23:34:54.366724+0000","last_active":"2024-04-24T23:34:59.396642+0000","last_peered":"2024-04-24T23:34:59.396642+0000","last_clean":"2024-04-24T23:34:59.396642+0000","last_became_active":"2024-04-24T23:34:54.366426+0000","last_became_peered":"2024-04-24T23:34:54.366426+0000","last_unstale":"2024-04-24T23:34:59.396642+0000","last_undegraded":"2024-04-24T23:34:59.396642+0000","last_fullsized":"2024-04-24T23:34:59.396642+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:47.035356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.881182+0000","last_change":"2024-04-24T23:34:53.359140+0000","last_active":"2024-04-24T23:34:59.881182+0000","last_peered":"2024-04-24T23:34:59.881182+0000","last_clean":"2024-04-24T23:34:59.881182+0000","last_became_active":"2024-04-24T23:34:53.358841+0000","last_became_peered":"2024-04-24T23:34:53.358841+0000","last_unstale":"2024-04-24T23:34:59.881182+0000","last_undegraded":"2024-04-24T23:34:59.881182+0000","last_fullsized":"2024-04-24T23:34:59.881182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:41.422591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397033+0000","last_change":"2024-04-24T23:34:58.400128+0000","last_active":"2024-04-24T23:34:59.397033+0000","last_peered":"2024-04-24T23:34:59.397033+0000","last_clean":"2024-04-24T23:34:59.397033+0000","last_became_active":"2024-04-24T23:34:58.399375+0000","last_became_peered":"2024-04-24T23:34:58.399375+0000","last_unstale":"2024-04-24T23:34:59.397033+0000","last_undegraded":"2024-04-24T23:34:59.397033+0000","last_fullsized":"2024-04-24T23:34:59.397033+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:53.409265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.395824+0000","last_change":"2024-04-24T23:34:54.365012+0000","last_active":"2024-04-24T23:34:59.395824+0000","last_peered":"2024-04-24T23:34:59.395824+0000","last_clean":"2024-04-24T23:34:59.395824+0000","last_became_active":"2024-04-24T23:34:54.364744+0000","last_became_peered":"2024-04-24T23:34:54.364744+0000","last_unstale":"2024-04-24T23:34:59.395824+0000","last_undegraded":"2024-04-24T23:34:59.395824+0000","last_fullsized":"2024-04-24T23:34:59.395824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:35.774122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.880884+0000","last_change":"2024-04-24T23:34:53.360843+0000","last_active":"2024-04-24T23:34:59.880884+0000","last_peered":"2024-04-24T23:34:59.880884+0000","last_clean":"2024-04-24T23:34:59.880884+0000","last_became_active":"2024-04-24T23:34:53.360570+0000","last_became_peered":"2024-04-24T23:34:53.360570+0000","last_unstale":"2024-04-24T23:34:59.880884+0000","last_undegraded":"2024-04-24T23:34:59.880884+0000","last_fullsized":"2024-04-24T23:34:59.880884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:21.089467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.881510+0000","last_change":"2024-04-24T23:34:58.731835+0000","last_active":"2024-04-24T23:34:59.881510+0000","last_peered":"2024-04-24T23:34:59.881510+0000","last_clean":"2024-04-24T23:34:59.881510+0000","last_became_active":"2024-04-24T23:34:58.731485+0000","last_became_peered":"2024-04-24T23:34:58.731485+0000","last_unstale":"2024-04-24T23:34:59.881510+0000","last_undegraded":"2024-04-24T23:34:59.881510+0000","last_fullsized":"2024-04-24T23:34:59.881510+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:56.609480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397598+0000","last_change":"2024-04-24T23:34:54.362447+0000","last_active":"2024-04-24T23:34:59.397598+0000","last_peered":"2024-04-24T23:34:59.397598+0000","last_clean":"2024-04-24T23:34:59.397598+0000","last_became_active":"2024-04-24T23:34:54.362141+0000","last_became_peered":"2024-04-24T23:34:54.362141+0000","last_unstale":"2024-04-24T23:34:59.397598+0000","last_undegraded":"2024-04-24T23:34:59.397598+0000","last_fullsized":"2024-04-24T23:34:59.397598+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:51.227333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501234+0000","last_change":"2024-04-24T23:34:53.366452+0000","last_active":"2024-04-24T23:34:59.501234+0000","last_peered":"2024-04-24T23:34:59.501234+0000","last_clean":"2024-04-24T23:34:59.501234+0000","last_became_active":"2024-04-24T23:34:53.366110+0000","last_became_peered":"2024-04-24T23:34:53.366110+0000","last_unstale":"2024-04-24T23:34:59.501234+0000","last_undegraded":"2024-04-24T23:34:59.501234+0000","last_fullsized":"2024-04-24T23:34:59.501234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:14.716741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396886+0000","last_change":"2024-04-24T23:34:58.402849+0000","last_active":"2024-04-24T23:34:59.396886+0000","last_peered":"2024-04-24T23:34:59.396886+0000","last_clean":"2024-04-24T23:34:59.396886+0000","last_became_active":"2024-04-24T23:34:58.402365+0000","last_became_peered":"2024-04-24T23:34:58.402365+0000","last_unstale":"2024-04-24T23:34:59.396886+0000","last_undegraded":"2024-04-24T23:34:59.396886+0000","last_fullsized":"2024-04-24T23:34:59.396886+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:07:32.822427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.428004+0000","last_change":"2024-04-24T23:34:54.370300+0000","last_active":"2024-04-24T23:34:59.428004+0000","last_peered":"2024-04-24T23:34:59.428004+0000","last_clean":"2024-04-24T23:34:59.428004+0000","last_became_active":"2024-04-24T23:34:54.369835+0000","last_became_peered":"2024-04-24T23:34:54.369835+0000","last_unstale":"2024-04-24T23:34:59.428004+0000","last_undegraded":"2024-04-24T23:34:59.428004+0000","last_fullsized":"2024-04-24T23:34:59.428004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:42.199856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.541551+0000","last_change":"2024-04-24T23:34:53.366110+0000","last_active":"2024-04-24T23:35:00.541551+0000","last_peered":"2024-04-24T23:35:00.541551+0000","last_clean":"2024-04-24T23:35:00.541551+0000","last_became_active":"2024-04-24T23:34:53.365783+0000","last_became_peered":"2024-04-24T23:34:53.365783+0000","last_unstale":"2024-04-24T23:35:00.541551+0000","last_undegraded":"2024-04-24T23:35:00.541551+0000","last_fullsized":"2024-04-24T23:35:00.541551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:41.422591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397333+0000","last_change":"2024-04-24T23:34:58.728191+0000","last_active":"2024-04-24T23:34:59.397333+0000","last_peered":"2024-04-24T23:34:59.397333+0000","last_clean":"2024-04-24T23:34:59.397333+0000","last_became_active":"2024-04-24T23:34:58.727653+0000","last_became_peered":"2024-04-24T23:34:58.727653+0000","last_unstale":"2024-04-24T23:34:59.397333+0000","last_undegraded":"2024-04-24T23:34:59.397333+0000","last_fullsized":"2024-04-24T23:34:59.397333+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:30.299011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397474+0000","last_change":"2024-04-24T23:34:54.368278+0000","last_active":"2024-04-24T23:34:59.397474+0000","last_peered":"2024-04-24T23:34:59.397474+0000","last_clean":"2024-04-24T23:34:59.397474+0000","last_became_active":"2024-04-24T23:34:54.367991+0000","last_became_peered":"2024-04-24T23:34:54.367991+0000","last_unstale":"2024-04-24T23:34:59.397474+0000","last_undegraded":"2024-04-24T23:34:59.397474+0000","last_fullsized":"2024-04-24T23:34:59.397474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:49:31.018719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.881400+0000","last_change":"2024-04-24T23:34:53.364189+0000","last_active":"2024-04-24T23:34:59.881400+0000","last_peered":"2024-04-24T23:34:59.881400+0000","last_clean":"2024-04-24T23:34:59.881400+0000","last_became_active":"2024-04-24T23:34:53.363842+0000","last_became_peered":"2024-04-24T23:34:53.363842+0000","last_unstale":"2024-04-24T23:34:59.881400+0000","last_undegraded":"2024-04-24T23:34:59.881400+0000","last_fullsized":"2024-04-24T23:34:59.881400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:12.258211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.395610+0000","last_change":"2024-04-24T23:34:58.729450+0000","last_active":"2024-04-24T23:34:59.395610+0000","last_peered":"2024-04-24T23:34:59.395610+0000","last_clean":"2024-04-24T23:34:59.395610+0000","last_became_active":"2024-04-24T23:34:58.729082+0000","last_became_peered":"2024-04-24T23:34:58.729082+0000","last_unstale":"2024-04-24T23:34:59.395610+0000","last_undegraded":"2024-04-24T23:34:59.395610+0000","last_fullsized":"2024-04-24T23:34:59.395610+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:45.372053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.881249+0000","last_change":"2024-04-24T23:34:54.357246+0000","last_active":"2024-04-24T23:34:59.881249+0000","last_peered":"2024-04-24T23:34:59.881249+0000","last_clean":"2024-04-24T23:34:59.881249+0000","last_became_active":"2024-04-24T23:34:54.356560+0000","last_became_peered":"2024-04-24T23:34:54.356560+0000","last_unstale":"2024-04-24T23:34:59.881249+0000","last_undegraded":"2024-04-24T23:34:59.881249+0000","last_fullsized":"2024-04-24T23:34:59.881249+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:42.199856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.502011+0000","last_change":"2024-04-24T23:34:53.366296+0000","last_active":"2024-04-24T23:34:59.502011+0000","last_peered":"2024-04-24T23:34:59.502011+0000","last_clean":"2024-04-24T23:34:59.502011+0000","last_became_active":"2024-04-24T23:34:53.365636+0000","last_became_peered":"2024-04-24T23:34:53.365636+0000","last_unstale":"2024-04-24T23:34:59.502011+0000","last_undegraded":"2024-04-24T23:34:59.502011+0000","last_fullsized":"2024-04-24T23:34:59.502011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:50.222015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.427745+0000","last_change":"2024-04-24T23:34:58.728271+0000","last_active":"2024-04-24T23:34:59.427745+0000","last_peered":"2024-04-24T23:34:59.427745+0000","last_clean":"2024-04-24T23:34:59.427745+0000","last_became_active":"2024-04-24T23:34:58.727885+0000","last_became_peered":"2024-04-24T23:34:58.727885+0000","last_unstale":"2024-04-24T23:34:59.427745+0000","last_undegraded":"2024-04-24T23:34:59.427745+0000","last_fullsized":"2024-04-24T23:34:59.427745+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:49.461271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.541247+0000","last_change":"2024-04-24T23:34:54.372245+0000","last_active":"2024-04-24T23:35:00.541247+0000","last_peered":"2024-04-24T23:35:00.541247+0000","last_clean":"2024-04-24T23:35:00.541247+0000","last_became_active":"2024-04-24T23:34:54.371947+0000","last_became_peered":"2024-04-24T23:34:54.371947+0000","last_unstale":"2024-04-24T23:35:00.541247+0000","last_undegraded":"2024-04-24T23:35:00.541247+0000","last_fullsized":"2024-04-24T23:35:00.541247+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:47:52.460508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.881345+0000","last_change":"2024-04-24T23:34:53.360352+0000","last_active":"2024-04-24T23:34:59.881345+0000","last_peered":"2024-04-24T23:34:59.881345+0000","last_clean":"2024-04-24T23:34:59.881345+0000","last_became_active":"2024-04-24T23:34:53.360082+0000","last_became_peered":"2024-04-24T23:34:53.360082+0000","last_unstale":"2024-04-24T23:34:59.881345+0000","last_undegraded":"2024-04-24T23:34:59.881345+0000","last_fullsized":"2024-04-24T23:34:59.881345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:50.222015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.881021+0000","last_change":"2024-04-24T23:34:58.404176+0000","last_active":"2024-04-24T23:34:59.881021+0000","last_peered":"2024-04-24T23:34:59.881021+0000","last_clean":"2024-04-24T23:34:59.881021+0000","last_became_active":"2024-04-24T23:34:58.403735+0000","last_became_peered":"2024-04-24T23:34:58.403735+0000","last_unstale":"2024-04-24T23:34:59.881021+0000","last_undegraded":"2024-04-24T23:34:59.881021+0000","last_fullsized":"2024-04-24T23:34:59.881021+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:49.461271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397652+0000","last_change":"2024-04-24T23:34:54.367638+0000","last_active":"2024-04-24T23:34:59.397652+0000","last_peered":"2024-04-24T23:34:59.397652+0000","last_clean":"2024-04-24T23:34:59.397652+0000","last_became_active":"2024-04-24T23:34:54.367355+0000","last_became_peered":"2024-04-24T23:34:54.367355+0000","last_unstale":"2024-04-24T23:34:59.397652+0000","last_undegraded":"2024-04-24T23:34:59.397652+0000","last_fullsized":"2024-04-24T23:34:59.397652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:15.722059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540545+0000","last_change":"2024-04-24T23:34:53.356111+0000","last_active":"2024-04-24T23:35:00.540545+0000","last_peered":"2024-04-24T23:35:00.540545+0000","last_clean":"2024-04-24T23:35:00.540545+0000","last_became_active":"2024-04-24T23:34:53.355710+0000","last_became_peered":"2024-04-24T23:34:53.355710+0000","last_unstale":"2024-04-24T23:35:00.540545+0000","last_undegraded":"2024-04-24T23:35:00.540545+0000","last_fullsized":"2024-04-24T23:35:00.540545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:21.089467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.881283+0000","last_change":"2024-04-24T23:34:58.403016+0000","last_active":"2024-04-24T23:34:59.881283+0000","last_peered":"2024-04-24T23:34:59.881283+0000","last_clean":"2024-04-24T23:34:59.881283+0000","last_became_active":"2024-04-24T23:34:58.402598+0000","last_became_peered":"2024-04-24T23:34:58.402598+0000","last_unstale":"2024-04-24T23:34:59.881283+0000","last_undegraded":"2024-04-24T23:34:59.881283+0000","last_fullsized":"2024-04-24T23:34:59.881283+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:47:56.497734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.880863+0000","last_change":"2024-04-24T23:34:54.356601+0000","last_active":"2024-04-24T23:34:59.880863+0000","last_peered":"2024-04-24T23:34:59.880863+0000","last_clean":"2024-04-24T23:34:59.880863+0000","last_became_active":"2024-04-24T23:34:54.356326+0000","last_became_peered":"2024-04-24T23:34:54.356326+0000","last_unstale":"2024-04-24T23:34:59.880863+0000","last_undegraded":"2024-04-24T23:34:59.880863+0000","last_fullsized":"2024-04-24T23:34:59.880863+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:47.446459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396563+0000","last_change":"2024-04-24T23:34:53.359238+0000","last_active":"2024-04-24T23:34:59.396563+0000","last_peered":"2024-04-24T23:34:59.396563+0000","last_clean":"2024-04-24T23:34:59.396563+0000","last_became_active":"2024-04-24T23:34:53.358937+0000","last_became_peered":"2024-04-24T23:34:53.358937+0000","last_unstale":"2024-04-24T23:34:59.396563+0000","last_undegraded":"2024-04-24T23:34:59.396563+0000","last_fullsized":"2024-04-24T23:34:59.396563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:50.222015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.881551+0000","last_change":"2024-04-24T23:34:58.403258+0000","last_active":"2024-04-24T23:34:59.881551+0000","last_peered":"2024-04-24T23:34:59.881551+0000","last_clean":"2024-04-24T23:34:59.881551+0000","last_became_active":"2024-04-24T23:34:58.402569+0000","last_became_peered":"2024-04-24T23:34:58.402569+0000","last_unstale":"2024-04-24T23:34:59.881551+0000","last_undegraded":"2024-04-24T23:34:59.881551+0000","last_fullsized":"2024-04-24T23:34:59.881551+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:32:08.782070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.427468+0000","last_change":"2024-04-24T23:34:54.377543+0000","last_active":"2024-04-24T23:34:59.427468+0000","last_peered":"2024-04-24T23:34:59.427468+0000","last_clean":"2024-04-24T23:34:59.427468+0000","last_became_active":"2024-04-24T23:34:54.377228+0000","last_became_peered":"2024-04-24T23:34:54.377228+0000","last_unstale":"2024-04-24T23:34:59.427468+0000","last_undegraded":"2024-04-24T23:34:59.427468+0000","last_fullsized":"2024-04-24T23:34:59.427468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:42.255038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501238+0000","last_change":"2024-04-24T23:34:53.362675+0000","last_active":"2024-04-24T23:34:59.501238+0000","last_peered":"2024-04-24T23:34:59.501238+0000","last_clean":"2024-04-24T23:34:59.501238+0000","last_became_active":"2024-04-24T23:34:53.362425+0000","last_became_peered":"2024-04-24T23:34:53.362425+0000","last_unstale":"2024-04-24T23:34:59.501238+0000","last_undegraded":"2024-04-24T23:34:59.501238+0000","last_fullsized":"2024-04-24T23:34:59.501238+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:21.089467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.427884+0000","last_change":"2024-04-24T23:34:58.728573+0000","last_active":"2024-04-24T23:34:59.427884+0000","last_peered":"2024-04-24T23:34:59.427884+0000","last_clean":"2024-04-24T23:34:59.427884+0000","last_became_active":"2024-04-24T23:34:58.727284+0000","last_became_peered":"2024-04-24T23:34:58.727284+0000","last_unstale":"2024-04-24T23:34:59.427884+0000","last_undegraded":"2024-04-24T23:34:59.427884+0000","last_fullsized":"2024-04-24T23:34:59.427884+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:32:08.782070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396198+0000","last_change":"2024-04-24T23:34:54.382820+0000","last_active":"2024-04-24T23:34:59.396198+0000","last_peered":"2024-04-24T23:34:59.396198+0000","last_clean":"2024-04-24T23:34:59.396198+0000","last_became_active":"2024-04-24T23:34:54.382564+0000","last_became_peered":"2024-04-24T23:34:54.382564+0000","last_unstale":"2024-04-24T23:34:59.396198+0000","last_undegraded":"2024-04-24T23:34:59.396198+0000","last_fullsized":"2024-04-24T23:34:59.396198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:45.622611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.541368+0000","last_change":"2024-04-24T23:34:53.357743+0000","last_active":"2024-04-24T23:35:00.541368+0000","last_peered":"2024-04-24T23:35:00.541368+0000","last_clean":"2024-04-24T23:35:00.541368+0000","last_became_active":"2024-04-24T23:34:53.357456+0000","last_became_peered":"2024-04-24T23:34:53.357456+0000","last_unstale":"2024-04-24T23:35:00.541368+0000","last_undegraded":"2024-04-24T23:35:00.541368+0000","last_fullsized":"2024-04-24T23:35:00.541368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:57:14.381644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":177,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501380+0000","last_change":"2024-04-24T23:34:49.778966+0000","last_active":"2024-04-24T23:34:59.501380+0000","last_peered":"2024-04-24T23:34:59.501380+0000","last_clean":"2024-04-24T23:34:59.501380+0000","last_became_active":"2024-04-24T23:34:49.778083+0000","last_became_peered":"2024-04-24T23:34:49.778083+0000","last_unstale":"2024-04-24T23:34:59.501380+0000","last_undegraded":"2024-04-24T23:34:59.501380+0000","last_fullsized":"2024-04-24T23:34:59.501380+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:48.587078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:48.587078+0000","last_clean_scrub_stamp":"2024-04-24T23:34:48.587078+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:46:59.471089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540517+0000","last_change":"2024-04-24T23:34:58.729779+0000","last_active":"2024-04-24T23:35:00.540517+0000","last_peered":"2024-04-24T23:35:00.540517+0000","last_clean":"2024-04-24T23:35:00.540517+0000","last_became_active":"2024-04-24T23:34:58.729370+0000","last_became_peered":"2024-04-24T23:34:58.729370+0000","last_unstale":"2024-04-24T23:35:00.540517+0000","last_undegraded":"2024-04-24T23:35:00.540517+0000","last_fullsized":"2024-04-24T23:35:00.540517+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:30.299011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501756+0000","last_change":"2024-04-24T23:34:54.384106+0000","last_active":"2024-04-24T23:34:59.501756+0000","last_peered":"2024-04-24T23:34:59.501756+0000","last_clean":"2024-04-24T23:34:59.501756+0000","last_became_active":"2024-04-24T23:34:54.383589+0000","last_became_peered":"2024-04-24T23:34:54.383589+0000","last_unstale":"2024-04-24T23:34:59.501756+0000","last_undegraded":"2024-04-24T23:34:59.501756+0000","last_fullsized":"2024-04-24T23:34:59.501756+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:41:48.207542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397450+0000","last_change":"2024-04-24T23:34:53.354077+0000","last_active":"2024-04-24T23:34:59.397450+0000","last_peered":"2024-04-24T23:34:59.397450+0000","last_clean":"2024-04-24T23:34:59.397450+0000","last_became_active":"2024-04-24T23:34:53.353804+0000","last_became_peered":"2024-04-24T23:34:53.353804+0000","last_unstale":"2024-04-24T23:34:59.397450+0000","last_undegraded":"2024-04-24T23:34:59.397450+0000","last_fullsized":"2024-04-24T23:34:59.397450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:41.422591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540103+0000","last_change":"2024-04-24T23:34:58.728722+0000","last_active":"2024-04-24T23:35:00.540103+0000","last_peered":"2024-04-24T23:35:00.540103+0000","last_clean":"2024-04-24T23:35:00.540103+0000","last_became_active":"2024-04-24T23:34:58.727649+0000","last_became_peered":"2024-04-24T23:34:58.727649+0000","last_unstale":"2024-04-24T23:35:00.540103+0000","last_undegraded":"2024-04-24T23:35:00.540103+0000","last_fullsized":"2024-04-24T23:35:00.540103+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:41:52.244767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540678+0000","last_change":"2024-04-24T23:34:54.373371+0000","last_active":"2024-04-24T23:35:00.540678+0000","last_peered":"2024-04-24T23:35:00.540678+0000","last_clean":"2024-04-24T23:35:00.540678+0000","last_became_active":"2024-04-24T23:34:54.372553+0000","last_became_peered":"2024-04-24T23:34:54.372553+0000","last_unstale":"2024-04-24T23:35:00.540678+0000","last_undegraded":"2024-04-24T23:35:00.540678+0000","last_fullsized":"2024-04-24T23:35:00.540678+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:26.261786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.881287+0000","last_change":"2024-04-24T23:34:53.353505+0000","last_active":"2024-04-24T23:34:59.881287+0000","last_peered":"2024-04-24T23:34:59.881287+0000","last_clean":"2024-04-24T23:34:59.881287+0000","last_became_active":"2024-04-24T23:34:53.353186+0000","last_became_peered":"2024-04-24T23:34:53.353186+0000","last_unstale":"2024-04-24T23:34:59.881287+0000","last_undegraded":"2024-04-24T23:34:59.881287+0000","last_fullsized":"2024-04-24T23:34:59.881287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:42.205997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.427805+0000","last_change":"2024-04-24T23:34:58.400988+0000","last_active":"2024-04-24T23:34:59.427805+0000","last_peered":"2024-04-24T23:34:59.427805+0000","last_clean":"2024-04-24T23:34:59.427805+0000","last_became_active":"2024-04-24T23:34:58.400549+0000","last_became_peered":"2024-04-24T23:34:58.400549+0000","last_unstale":"2024-04-24T23:34:59.427805+0000","last_undegraded":"2024-04-24T23:34:59.427805+0000","last_fullsized":"2024-04-24T23:34:59.427805+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:47:56.497734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.502060+0000","last_change":"2024-04-24T23:34:54.375234+0000","last_active":"2024-04-24T23:34:59.502060+0000","last_peered":"2024-04-24T23:34:59.502060+0000","last_clean":"2024-04-24T23:34:59.502060+0000","last_became_active":"2024-04-24T23:34:54.374963+0000","last_became_peered":"2024-04-24T23:34:54.374963+0000","last_unstale":"2024-04-24T23:34:59.502060+0000","last_undegraded":"2024-04-24T23:34:59.502060+0000","last_fullsized":"2024-04-24T23:34:59.502060+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:47:52.460508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396918+0000","last_change":"2024-04-24T23:34:53.360531+0000","last_active":"2024-04-24T23:34:59.396918+0000","last_peered":"2024-04-24T23:34:59.396918+0000","last_clean":"2024-04-24T23:34:59.396918+0000","last_became_active":"2024-04-24T23:34:53.360247+0000","last_became_peered":"2024-04-24T23:34:53.360247+0000","last_unstale":"2024-04-24T23:34:59.396918+0000","last_undegraded":"2024-04-24T23:34:59.396918+0000","last_fullsized":"2024-04-24T23:34:59.396918+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:57:14.381644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540673+0000","last_change":"2024-04-24T23:34:58.729798+0000","last_active":"2024-04-24T23:35:00.540673+0000","last_peered":"2024-04-24T23:35:00.540673+0000","last_clean":"2024-04-24T23:35:00.540673+0000","last_became_active":"2024-04-24T23:34:58.729345+0000","last_became_peered":"2024-04-24T23:34:58.729345+0000","last_unstale":"2024-04-24T23:35:00.540673+0000","last_undegraded":"2024-04-24T23:35:00.540673+0000","last_fullsized":"2024-04-24T23:35:00.540673+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:06.670518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501916+0000","last_change":"2024-04-24T23:34:54.377593+0000","last_active":"2024-04-24T23:34:59.501916+0000","last_peered":"2024-04-24T23:34:59.501916+0000","last_clean":"2024-04-24T23:34:59.501916+0000","last_became_active":"2024-04-24T23:34:54.377270+0000","last_became_peered":"2024-04-24T23:34:54.377270+0000","last_unstale":"2024-04-24T23:34:59.501916+0000","last_undegraded":"2024-04-24T23:34:59.501916+0000","last_fullsized":"2024-04-24T23:34:59.501916+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:26.261786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397587+0000","last_change":"2024-04-24T23:34:53.352981+0000","last_active":"2024-04-24T23:34:59.397587+0000","last_peered":"2024-04-24T23:34:59.397587+0000","last_clean":"2024-04-24T23:34:59.397587+0000","last_became_active":"2024-04-24T23:34:53.352669+0000","last_became_peered":"2024-04-24T23:34:53.352669+0000","last_unstale":"2024-04-24T23:34:59.397587+0000","last_undegraded":"2024-04-24T23:34:59.397587+0000","last_fullsized":"2024-04-24T23:34:59.397587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:09:49.550050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396650+0000","last_change":"2024-04-24T23:34:58.399234+0000","last_active":"2024-04-24T23:34:59.396650+0000","last_peered":"2024-04-24T23:34:59.396650+0000","last_clean":"2024-04-24T23:34:59.396650+0000","last_became_active":"2024-04-24T23:34:58.398824+0000","last_became_peered":"2024-04-24T23:34:58.398824+0000","last_unstale":"2024-04-24T23:34:59.396650+0000","last_undegraded":"2024-04-24T23:34:59.396650+0000","last_fullsized":"2024-04-24T23:34:59.396650+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:22.244493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501466+0000","last_change":"2024-04-24T23:34:54.376800+0000","last_active":"2024-04-24T23:34:59.501466+0000","last_peered":"2024-04-24T23:34:59.501466+0000","last_clean":"2024-04-24T23:34:59.501466+0000","last_became_active":"2024-04-24T23:34:54.376512+0000","last_became_peered":"2024-04-24T23:34:54.376512+0000","last_unstale":"2024-04-24T23:34:59.501466+0000","last_undegraded":"2024-04-24T23:34:59.501466+0000","last_fullsized":"2024-04-24T23:34:59.501466+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:49.048141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396460+0000","last_change":"2024-04-24T23:34:53.355307+0000","last_active":"2024-04-24T23:34:59.396460+0000","last_peered":"2024-04-24T23:34:59.396460+0000","last_clean":"2024-04-24T23:34:59.396460+0000","last_became_active":"2024-04-24T23:34:53.354988+0000","last_became_peered":"2024-04-24T23:34:53.354988+0000","last_unstale":"2024-04-24T23:34:59.396460+0000","last_undegraded":"2024-04-24T23:34:59.396460+0000","last_fullsized":"2024-04-24T23:34:59.396460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:39.457116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.427781+0000","last_change":"2024-04-24T23:34:58.399188+0000","last_active":"2024-04-24T23:34:59.427781+0000","last_peered":"2024-04-24T23:34:59.427781+0000","last_clean":"2024-04-24T23:34:59.427781+0000","last_became_active":"2024-04-24T23:34:58.398457+0000","last_became_peered":"2024-04-24T23:34:58.398457+0000","last_unstale":"2024-04-24T23:34:59.427781+0000","last_undegraded":"2024-04-24T23:34:59.427781+0000","last_fullsized":"2024-04-24T23:34:59.427781+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:50:21.658540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540120+0000","last_change":"2024-04-24T23:34:54.372150+0000","last_active":"2024-04-24T23:35:00.540120+0000","last_peered":"2024-04-24T23:35:00.540120+0000","last_clean":"2024-04-24T23:35:00.540120+0000","last_became_active":"2024-04-24T23:34:54.371703+0000","last_became_peered":"2024-04-24T23:34:54.371703+0000","last_unstale":"2024-04-24T23:35:00.540120+0000","last_undegraded":"2024-04-24T23:35:00.540120+0000","last_fullsized":"2024-04-24T23:35:00.540120+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:15.477061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540199+0000","last_change":"2024-04-24T23:34:53.363787+0000","last_active":"2024-04-24T23:35:00.540199+0000","last_peered":"2024-04-24T23:35:00.540199+0000","last_clean":"2024-04-24T23:35:00.540199+0000","last_became_active":"2024-04-24T23:34:53.363515+0000","last_became_peered":"2024-04-24T23:34:53.363515+0000","last_unstale":"2024-04-24T23:35:00.540199+0000","last_undegraded":"2024-04-24T23:35:00.540199+0000","last_fullsized":"2024-04-24T23:35:00.540199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:50.222015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397429+0000","last_change":"2024-04-24T23:34:58.728420+0000","last_active":"2024-04-24T23:34:59.397429+0000","last_peered":"2024-04-24T23:34:59.397429+0000","last_clean":"2024-04-24T23:34:59.397429+0000","last_became_active":"2024-04-24T23:34:58.727425+0000","last_became_peered":"2024-04-24T23:34:58.727425+0000","last_unstale":"2024-04-24T23:34:59.397429+0000","last_undegraded":"2024-04-24T23:34:59.397429+0000","last_fullsized":"2024-04-24T23:34:59.397429+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:19:14.487689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501575+0000","last_change":"2024-04-24T23:34:54.379296+0000","last_active":"2024-04-24T23:34:59.501575+0000","last_peered":"2024-04-24T23:34:59.501575+0000","last_clean":"2024-04-24T23:34:59.501575+0000","last_became_active":"2024-04-24T23:34:54.379016+0000","last_became_peered":"2024-04-24T23:34:54.379016+0000","last_unstale":"2024-04-24T23:34:59.501575+0000","last_undegraded":"2024-04-24T23:34:59.501575+0000","last_fullsized":"2024-04-24T23:34:59.501575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:37.323551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540560+0000","last_change":"2024-04-24T23:34:53.360210+0000","last_active":"2024-04-24T23:35:00.540560+0000","last_peered":"2024-04-24T23:35:00.540560+0000","last_clean":"2024-04-24T23:35:00.540560+0000","last_became_active":"2024-04-24T23:34:53.359581+0000","last_became_peered":"2024-04-24T23:34:53.359581+0000","last_unstale":"2024-04-24T23:35:00.540560+0000","last_undegraded":"2024-04-24T23:35:00.540560+0000","last_fullsized":"2024-04-24T23:35:00.540560+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:14.716741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397168+0000","last_change":"2024-04-24T23:34:58.728491+0000","last_active":"2024-04-24T23:34:59.397168+0000","last_peered":"2024-04-24T23:34:59.397168+0000","last_clean":"2024-04-24T23:34:59.397168+0000","last_became_active":"2024-04-24T23:34:58.727686+0000","last_became_peered":"2024-04-24T23:34:58.727686+0000","last_unstale":"2024-04-24T23:34:59.397168+0000","last_undegraded":"2024-04-24T23:34:59.397168+0000","last_fullsized":"2024-04-24T23:34:59.397168+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:46.282712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.427650+0000","last_change":"2024-04-24T23:34:54.372009+0000","last_active":"2024-04-24T23:34:59.427650+0000","last_peered":"2024-04-24T23:34:59.427650+0000","last_clean":"2024-04-24T23:34:59.427650+0000","last_became_active":"2024-04-24T23:34:54.371720+0000","last_became_peered":"2024-04-24T23:34:54.371720+0000","last_unstale":"2024-04-24T23:34:59.427650+0000","last_undegraded":"2024-04-24T23:34:59.427650+0000","last_fullsized":"2024-04-24T23:34:59.427650+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:04:07.487367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.428005+0000","last_change":"2024-04-24T23:34:53.353738+0000","last_active":"2024-04-24T23:34:59.428005+0000","last_peered":"2024-04-24T23:34:59.428005+0000","last_clean":"2024-04-24T23:34:59.428005+0000","last_became_active":"2024-04-24T23:34:53.353452+0000","last_became_peered":"2024-04-24T23:34:53.353452+0000","last_unstale":"2024-04-24T23:34:59.428005+0000","last_undegraded":"2024-04-24T23:34:59.428005+0000","last_fullsized":"2024-04-24T23:34:59.428005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:33:43.693095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.541104+0000","last_change":"2024-04-24T23:34:58.728503+0000","last_active":"2024-04-24T23:35:00.541104+0000","last_peered":"2024-04-24T23:35:00.541104+0000","last_clean":"2024-04-24T23:35:00.541104+0000","last_became_active":"2024-04-24T23:34:58.727921+0000","last_became_peered":"2024-04-24T23:34:58.727921+0000","last_unstale":"2024-04-24T23:35:00.541104+0000","last_undegraded":"2024-04-24T23:35:00.541104+0000","last_fullsized":"2024-04-24T23:35:00.541104+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:19:40.928966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501636+0000","last_change":"2024-04-24T23:34:54.368532+0000","last_active":"2024-04-24T23:34:59.501636+0000","last_peered":"2024-04-24T23:34:59.501636+0000","last_clean":"2024-04-24T23:34:59.501636+0000","last_became_active":"2024-04-24T23:34:54.367903+0000","last_became_peered":"2024-04-24T23:34:54.367903+0000","last_unstale":"2024-04-24T23:34:59.501636+0000","last_undegraded":"2024-04-24T23:34:59.501636+0000","last_fullsized":"2024-04-24T23:34:59.501636+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:49.372040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:03.674709+0000","last_change":"2024-04-24T23:34:53.362758+0000","last_active":"2024-04-24T23:35:03.674709+0000","last_peered":"2024-04-24T23:35:03.674709+0000","last_clean":"2024-04-24T23:35:03.674709+0000","last_became_active":"2024-04-24T23:34:53.362230+0000","last_became_peered":"2024-04-24T23:34:53.362230+0000","last_unstale":"2024-04-24T23:35:03.674709+0000","last_undegraded":"2024-04-24T23:35:03.674709+0000","last_fullsized":"2024-04-24T23:35:03.674709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:14.716741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.881050+0000","last_change":"2024-04-24T23:34:58.400881+0000","last_active":"2024-04-24T23:34:59.881050+0000","last_peered":"2024-04-24T23:34:59.881050+0000","last_clean":"2024-04-24T23:34:59.881050+0000","last_became_active":"2024-04-24T23:34:58.400429+0000","last_became_peered":"2024-04-24T23:34:58.400429+0000","last_unstale":"2024-04-24T23:34:59.881050+0000","last_undegraded":"2024-04-24T23:34:59.881050+0000","last_fullsized":"2024-04-24T23:34:59.881050+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:50:21.658540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397180+0000","last_change":"2024-04-24T23:34:54.373603+0000","last_active":"2024-04-24T23:34:59.397180+0000","last_peered":"2024-04-24T23:34:59.397180+0000","last_clean":"2024-04-24T23:34:59.397180+0000","last_became_active":"2024-04-24T23:34:54.373347+0000","last_became_peered":"2024-04-24T23:34:54.373347+0000","last_unstale":"2024-04-24T23:34:59.397180+0000","last_undegraded":"2024-04-24T23:34:59.397180+0000","last_fullsized":"2024-04-24T23:34:59.397180+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:13.263529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.881250+0000","last_change":"2024-04-24T23:34:53.354673+0000","last_active":"2024-04-24T23:34:59.881250+0000","last_peered":"2024-04-24T23:34:59.881250+0000","last_clean":"2024-04-24T23:34:59.881250+0000","last_became_active":"2024-04-24T23:34:53.354394+0000","last_became_peered":"2024-04-24T23:34:53.354394+0000","last_unstale":"2024-04-24T23:34:59.881250+0000","last_undegraded":"2024-04-24T23:34:59.881250+0000","last_fullsized":"2024-04-24T23:34:59.881250+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:14.716741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397330+0000","last_change":"2024-04-24T23:34:58.397739+0000","last_active":"2024-04-24T23:34:59.397330+0000","last_peered":"2024-04-24T23:34:59.397330+0000","last_clean":"2024-04-24T23:34:59.397330+0000","last_became_active":"2024-04-24T23:34:58.397353+0000","last_became_peered":"2024-04-24T23:34:58.397353+0000","last_unstale":"2024-04-24T23:34:59.397330+0000","last_undegraded":"2024-04-24T23:34:59.397330+0000","last_fullsized":"2024-04-24T23:34:59.397330+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:55.672101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.427967+0000","last_change":"2024-04-24T23:34:54.383348+0000","last_active":"2024-04-24T23:34:59.427967+0000","last_peered":"2024-04-24T23:34:59.427967+0000","last_clean":"2024-04-24T23:34:59.427967+0000","last_became_active":"2024-04-24T23:34:54.383082+0000","last_became_peered":"2024-04-24T23:34:54.383082+0000","last_unstale":"2024-04-24T23:34:59.427967+0000","last_undegraded":"2024-04-24T23:34:59.427967+0000","last_fullsized":"2024-04-24T23:34:59.427967+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:56:17.763159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540591+0000","last_change":"2024-04-24T23:34:53.364540+0000","last_active":"2024-04-24T23:35:00.540591+0000","last_peered":"2024-04-24T23:35:00.540591+0000","last_clean":"2024-04-24T23:35:00.540591+0000","last_became_active":"2024-04-24T23:34:53.364259+0000","last_became_peered":"2024-04-24T23:34:53.364259+0000","last_unstale":"2024-04-24T23:35:00.540591+0000","last_undegraded":"2024-04-24T23:35:00.540591+0000","last_fullsized":"2024-04-24T23:35:00.540591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:41.422591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.395771+0000","last_change":"2024-04-24T23:34:58.405865+0000","last_active":"2024-04-24T23:34:59.395771+0000","last_peered":"2024-04-24T23:34:59.395771+0000","last_clean":"2024-04-24T23:34:59.395771+0000","last_became_active":"2024-04-24T23:34:58.405032+0000","last_became_peered":"2024-04-24T23:34:58.405032+0000","last_unstale":"2024-04-24T23:34:59.395771+0000","last_undegraded":"2024-04-24T23:34:59.395771+0000","last_fullsized":"2024-04-24T23:34:59.395771+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:54.012389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540495+0000","last_change":"2024-04-24T23:34:54.383631+0000","last_active":"2024-04-24T23:35:00.540495+0000","last_peered":"2024-04-24T23:35:00.540495+0000","last_clean":"2024-04-24T23:35:00.540495+0000","last_became_active":"2024-04-24T23:34:54.383354+0000","last_became_peered":"2024-04-24T23:34:54.383354+0000","last_unstale":"2024-04-24T23:35:00.540495+0000","last_undegraded":"2024-04-24T23:35:00.540495+0000","last_fullsized":"2024-04-24T23:35:00.540495+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:50:17.621315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.541421+0000","last_change":"2024-04-24T23:34:53.362409+0000","last_active":"2024-04-24T23:35:00.541421+0000","last_peered":"2024-04-24T23:35:00.541421+0000","last_clean":"2024-04-24T23:35:00.541421+0000","last_became_active":"2024-04-24T23:34:53.362103+0000","last_became_peered":"2024-04-24T23:34:53.362103+0000","last_unstale":"2024-04-24T23:35:00.541421+0000","last_undegraded":"2024-04-24T23:35:00.541421+0000","last_fullsized":"2024-04-24T23:35:00.541421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:57:14.381644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.395785+0000","last_change":"2024-04-24T23:34:58.730088+0000","last_active":"2024-04-24T23:34:59.395785+0000","last_peered":"2024-04-24T23:34:59.395785+0000","last_clean":"2024-04-24T23:34:59.395785+0000","last_became_active":"2024-04-24T23:34:58.729217+0000","last_became_peered":"2024-04-24T23:34:58.729217+0000","last_unstale":"2024-04-24T23:34:59.395785+0000","last_undegraded":"2024-04-24T23:34:59.395785+0000","last_fullsized":"2024-04-24T23:34:59.395785+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:12:14.243770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.427823+0000","last_change":"2024-04-24T23:34:58.728892+0000","last_active":"2024-04-24T23:34:59.427823+0000","last_peered":"2024-04-24T23:34:59.427823+0000","last_clean":"2024-04-24T23:34:59.427823+0000","last_became_active":"2024-04-24T23:34:58.727914+0000","last_became_peered":"2024-04-24T23:34:58.727914+0000","last_unstale":"2024-04-24T23:34:59.427823+0000","last_undegraded":"2024-04-24T23:34:59.427823+0000","last_fullsized":"2024-04-24T23:34:59.427823+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:56.609480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.880773+0000","last_change":"2024-04-24T23:34:54.363973+0000","last_active":"2024-04-24T23:34:59.880773+0000","last_peered":"2024-04-24T23:34:59.880773+0000","last_clean":"2024-04-24T23:34:59.880773+0000","last_became_active":"2024-04-24T23:34:54.363724+0000","last_became_peered":"2024-04-24T23:34:54.363724+0000","last_unstale":"2024-04-24T23:34:59.880773+0000","last_undegraded":"2024-04-24T23:34:59.880773+0000","last_fullsized":"2024-04-24T23:34:59.880773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:03:24.745895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.541110+0000","last_change":"2024-04-24T23:34:53.360041+0000","last_active":"2024-04-24T23:35:00.541110+0000","last_peered":"2024-04-24T23:35:00.541110+0000","last_clean":"2024-04-24T23:35:00.541110+0000","last_became_active":"2024-04-24T23:34:53.359707+0000","last_became_peered":"2024-04-24T23:34:53.359707+0000","last_unstale":"2024-04-24T23:35:00.541110+0000","last_undegraded":"2024-04-24T23:35:00.541110+0000","last_fullsized":"2024-04-24T23:35:00.541110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:50.222015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.541453+0000","last_change":"2024-04-24T23:34:58.399618+0000","last_active":"2024-04-24T23:35:00.541453+0000","last_peered":"2024-04-24T23:35:00.541453+0000","last_clean":"2024-04-24T23:35:00.541453+0000","last_became_active":"2024-04-24T23:34:58.399182+0000","last_became_peered":"2024-04-24T23:34:58.399182+0000","last_unstale":"2024-04-24T23:35:00.541453+0000","last_undegraded":"2024-04-24T23:35:00.541453+0000","last_fullsized":"2024-04-24T23:35:00.541453+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:55.672101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.395731+0000","last_change":"2024-04-24T23:34:54.369729+0000","last_active":"2024-04-24T23:34:59.395731+0000","last_peered":"2024-04-24T23:34:59.395731+0000","last_clean":"2024-04-24T23:34:59.395731+0000","last_became_active":"2024-04-24T23:34:54.369155+0000","last_became_peered":"2024-04-24T23:34:54.369155+0000","last_unstale":"2024-04-24T23:34:59.395731+0000","last_undegraded":"2024-04-24T23:34:59.395731+0000","last_fullsized":"2024-04-24T23:34:59.395731+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:46:28.181996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.395890+0000","last_change":"2024-04-24T23:34:53.365328+0000","last_active":"2024-04-24T23:34:59.395890+0000","last_peered":"2024-04-24T23:34:59.395890+0000","last_clean":"2024-04-24T23:34:59.395890+0000","last_became_active":"2024-04-24T23:34:53.365029+0000","last_became_peered":"2024-04-24T23:34:53.365029+0000","last_unstale":"2024-04-24T23:34:59.395890+0000","last_undegraded":"2024-04-24T23:34:59.395890+0000","last_fullsized":"2024-04-24T23:34:59.395890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:48:30.961870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540306+0000","last_change":"2024-04-24T23:34:53.365123+0000","last_active":"2024-04-24T23:35:00.540306+0000","last_peered":"2024-04-24T23:35:00.540306+0000","last_clean":"2024-04-24T23:35:00.540306+0000","last_became_active":"2024-04-24T23:34:53.364821+0000","last_became_peered":"2024-04-24T23:34:53.364821+0000","last_unstale":"2024-04-24T23:35:00.540306+0000","last_undegraded":"2024-04-24T23:35:00.540306+0000","last_fullsized":"2024-04-24T23:35:00.540306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:39.457116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.427901+0000","last_change":"2024-04-24T23:34:54.376037+0000","last_active":"2024-04-24T23:34:59.427901+0000","last_peered":"2024-04-24T23:34:59.427901+0000","last_clean":"2024-04-24T23:34:59.427901+0000","last_became_active":"2024-04-24T23:34:54.375689+0000","last_became_peered":"2024-04-24T23:34:54.375689+0000","last_unstale":"2024-04-24T23:34:59.427901+0000","last_undegraded":"2024-04-24T23:34:59.427901+0000","last_fullsized":"2024-04-24T23:34:59.427901+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:47.446459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397192+0000","last_change":"2024-04-24T23:34:58.404439+0000","last_active":"2024-04-24T23:34:59.397192+0000","last_peered":"2024-04-24T23:34:59.397192+0000","last_clean":"2024-04-24T23:34:59.397192+0000","last_became_active":"2024-04-24T23:34:58.404053+0000","last_became_peered":"2024-04-24T23:34:58.404053+0000","last_unstale":"2024-04-24T23:34:59.397192+0000","last_undegraded":"2024-04-24T23:34:59.397192+0000","last_fullsized":"2024-04-24T23:34:59.397192+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:46.292263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396770+0000","last_change":"2024-04-24T23:34:53.363066+0000","last_active":"2024-04-24T23:34:59.396770+0000","last_peered":"2024-04-24T23:34:59.396770+0000","last_clean":"2024-04-24T23:34:59.396770+0000","last_became_active":"2024-04-24T23:34:53.362548+0000","last_became_peered":"2024-04-24T23:34:53.362548+0000","last_unstale":"2024-04-24T23:34:59.396770+0000","last_undegraded":"2024-04-24T23:34:59.396770+0000","last_fullsized":"2024-04-24T23:34:59.396770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:21:13.338964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397349+0000","last_change":"2024-04-24T23:34:54.365704+0000","last_active":"2024-04-24T23:34:59.397349+0000","last_peered":"2024-04-24T23:34:59.397349+0000","last_clean":"2024-04-24T23:34:59.397349+0000","last_became_active":"2024-04-24T23:34:54.364814+0000","last_became_peered":"2024-04-24T23:34:54.364814+0000","last_unstale":"2024-04-24T23:34:59.397349+0000","last_undegraded":"2024-04-24T23:34:59.397349+0000","last_fullsized":"2024-04-24T23:34:59.397349+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:22.094785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397247+0000","last_change":"2024-04-24T23:34:58.407186+0000","last_active":"2024-04-24T23:34:59.397247+0000","last_peered":"2024-04-24T23:34:59.397247+0000","last_clean":"2024-04-24T23:34:59.397247+0000","last_became_active":"2024-04-24T23:34:58.406811+0000","last_became_peered":"2024-04-24T23:34:58.406811+0000","last_unstale":"2024-04-24T23:34:59.397247+0000","last_undegraded":"2024-04-24T23:34:59.397247+0000","last_fullsized":"2024-04-24T23:34:59.397247+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:55.861555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.539983+0000","last_change":"2024-04-24T23:34:54.375142+0000","last_active":"2024-04-24T23:35:00.539983+0000","last_peered":"2024-04-24T23:35:00.539983+0000","last_clean":"2024-04-24T23:35:00.539983+0000","last_became_active":"2024-04-24T23:34:54.374844+0000","last_became_peered":"2024-04-24T23:34:54.374844+0000","last_unstale":"2024-04-24T23:35:00.539983+0000","last_undegraded":"2024-04-24T23:35:00.539983+0000","last_fullsized":"2024-04-24T23:35:00.539983+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:29:07.593498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396456+0000","last_change":"2024-04-24T23:34:54.383574+0000","last_active":"2024-04-24T23:34:59.396456+0000","last_peered":"2024-04-24T23:34:59.396456+0000","last_clean":"2024-04-24T23:34:59.396456+0000","last_became_active":"2024-04-24T23:34:54.383253+0000","last_became_peered":"2024-04-24T23:34:54.383253+0000","last_unstale":"2024-04-24T23:34:59.396456+0000","last_undegraded":"2024-04-24T23:34:59.396456+0000","last_fullsized":"2024-04-24T23:34:59.396456+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:43.211314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540447+0000","last_change":"2024-04-24T23:34:54.372208+0000","last_active":"2024-04-24T23:35:00.540447+0000","last_peered":"2024-04-24T23:35:00.540447+0000","last_clean":"2024-04-24T23:35:00.540447+0000","last_became_active":"2024-04-24T23:34:54.371910+0000","last_became_peered":"2024-04-24T23:34:54.371910+0000","last_unstale":"2024-04-24T23:35:00.540447+0000","last_undegraded":"2024-04-24T23:35:00.540447+0000","last_fullsized":"2024-04-24T23:35:00.540447+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:07:28.785202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.427860+0000","last_change":"2024-04-24T23:34:54.366271+0000","last_active":"2024-04-24T23:34:59.427860+0000","last_peered":"2024-04-24T23:34:59.427860+0000","last_clean":"2024-04-24T23:34:59.427860+0000","last_became_active":"2024-04-24T23:34:54.366005+0000","last_became_peered":"2024-04-24T23:34:54.366005+0000","last_unstale":"2024-04-24T23:34:59.427860+0000","last_undegraded":"2024-04-24T23:34:59.427860+0000","last_fullsized":"2024-04-24T23:34:59.427860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:03:24.745895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397136+0000","last_change":"2024-04-24T23:34:54.360529+0000","last_active":"2024-04-24T23:34:59.397136+0000","last_peered":"2024-04-24T23:34:59.397136+0000","last_clean":"2024-04-24T23:34:59.397136+0000","last_became_active":"2024-04-24T23:34:54.360266+0000","last_became_peered":"2024-04-24T23:34:54.360266+0000","last_unstale":"2024-04-24T23:34:59.397136+0000","last_undegraded":"2024-04-24T23:34:59.397136+0000","last_fullsized":"2024-04-24T23:34:59.397136+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:49:47.246193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.881089+0000","last_change":"2024-04-24T23:34:54.365744+0000","last_active":"2024-04-24T23:34:59.881089+0000","last_peered":"2024-04-24T23:34:59.881089+0000","last_clean":"2024-04-24T23:34:59.881089+0000","last_became_active":"2024-04-24T23:34:54.365493+0000","last_became_peered":"2024-04-24T23:34:54.365493+0000","last_unstale":"2024-04-24T23:34:59.881089+0000","last_undegraded":"2024-04-24T23:34:59.881089+0000","last_fullsized":"2024-04-24T23:34:59.881089+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:15.477061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.541221+0000","last_change":"2024-04-24T23:34:54.379642+0000","last_active":"2024-04-24T23:35:00.541221+0000","last_peered":"2024-04-24T23:35:00.541221+0000","last_clean":"2024-04-24T23:35:00.541221+0000","last_became_active":"2024-04-24T23:34:54.378568+0000","last_became_peered":"2024-04-24T23:34:54.378568+0000","last_unstale":"2024-04-24T23:35:00.541221+0000","last_undegraded":"2024-04-24T23:35:00.541221+0000","last_fullsized":"2024-04-24T23:35:00.541221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:42.199856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397390+0000","last_change":"2024-04-24T23:34:54.363700+0000","last_active":"2024-04-24T23:34:59.397390+0000","last_peered":"2024-04-24T23:34:59.397390+0000","last_clean":"2024-04-24T23:34:59.397390+0000","last_became_active":"2024-04-24T23:34:54.363439+0000","last_became_peered":"2024-04-24T23:34:54.363439+0000","last_unstale":"2024-04-24T23:34:59.397390+0000","last_undegraded":"2024-04-24T23:34:59.397390+0000","last_fullsized":"2024-04-24T23:34:59.397390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:28:56.048178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.541000+0000","last_change":"2024-04-24T23:34:54.375460+0000","last_active":"2024-04-24T23:35:00.541000+0000","last_peered":"2024-04-24T23:35:00.541000+0000","last_clean":"2024-04-24T23:35:00.541000+0000","last_became_active":"2024-04-24T23:34:54.375123+0000","last_became_peered":"2024-04-24T23:34:54.375123+0000","last_unstale":"2024-04-24T23:35:00.541000+0000","last_undegraded":"2024-04-24T23:35:00.541000+0000","last_fullsized":"2024-04-24T23:35:00.541000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:18.207267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396028+0000","last_change":"2024-04-24T23:34:54.369509+0000","last_active":"2024-04-24T23:34:59.396028+0000","last_peered":"2024-04-24T23:34:59.396028+0000","last_clean":"2024-04-24T23:34:59.396028+0000","last_became_active":"2024-04-24T23:34:54.369219+0000","last_became_peered":"2024-04-24T23:34:54.369219+0000","last_unstale":"2024-04-24T23:34:59.396028+0000","last_undegraded":"2024-04-24T23:34:59.396028+0000","last_fullsized":"2024-04-24T23:34:59.396028+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:09:37.649503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540426+0000","last_change":"2024-04-24T23:34:54.384240+0000","last_active":"2024-04-24T23:35:00.540426+0000","last_peered":"2024-04-24T23:35:00.540426+0000","last_clean":"2024-04-24T23:35:00.540426+0000","last_became_active":"2024-04-24T23:34:54.383780+0000","last_became_peered":"2024-04-24T23:34:54.383780+0000","last_unstale":"2024-04-24T23:35:00.540426+0000","last_undegraded":"2024-04-24T23:35:00.540426+0000","last_fullsized":"2024-04-24T23:35:00.540426+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:49.048141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.427635+0000","last_change":"2024-04-24T23:34:54.371052+0000","last_active":"2024-04-24T23:34:59.427635+0000","last_peered":"2024-04-24T23:34:59.427635+0000","last_clean":"2024-04-24T23:34:59.427635+0000","last_became_active":"2024-04-24T23:34:54.370793+0000","last_became_peered":"2024-04-24T23:34:54.370793+0000","last_unstale":"2024-04-24T23:34:59.427635+0000","last_undegraded":"2024-04-24T23:34:59.427635+0000","last_fullsized":"2024-04-24T23:34:59.427635+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:29:07.593498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540693+0000","last_change":"2024-04-24T23:34:54.377190+0000","last_active":"2024-04-24T23:35:00.540693+0000","last_peered":"2024-04-24T23:35:00.540693+0000","last_clean":"2024-04-24T23:35:00.540693+0000","last_became_active":"2024-04-24T23:34:54.376940+0000","last_became_peered":"2024-04-24T23:34:54.376940+0000","last_unstale":"2024-04-24T23:35:00.540693+0000","last_undegraded":"2024-04-24T23:35:00.540693+0000","last_fullsized":"2024-04-24T23:35:00.540693+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:03:24.745895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501917+0000","last_change":"2024-04-24T23:34:54.368770+0000","last_active":"2024-04-24T23:34:59.501917+0000","last_peered":"2024-04-24T23:34:59.501917+0000","last_clean":"2024-04-24T23:34:59.501917+0000","last_became_active":"2024-04-24T23:34:54.368228+0000","last_became_peered":"2024-04-24T23:34:54.368228+0000","last_unstale":"2024-04-24T23:34:59.501917+0000","last_undegraded":"2024-04-24T23:34:59.501917+0000","last_fullsized":"2024-04-24T23:34:59.501917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:51.634876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396819+0000","last_change":"2024-04-24T23:34:54.373038+0000","last_active":"2024-04-24T23:34:59.396819+0000","last_peered":"2024-04-24T23:34:59.396819+0000","last_clean":"2024-04-24T23:34:59.396819+0000","last_became_active":"2024-04-24T23:34:54.372766+0000","last_became_peered":"2024-04-24T23:34:54.372766+0000","last_unstale":"2024-04-24T23:34:59.396819+0000","last_undegraded":"2024-04-24T23:34:59.396819+0000","last_fullsized":"2024-04-24T23:34:59.396819+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:13.263529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.541061+0000","last_change":"2024-04-24T23:34:54.383733+0000","last_active":"2024-04-24T23:35:00.541061+0000","last_peered":"2024-04-24T23:35:00.541061+0000","last_clean":"2024-04-24T23:35:00.541061+0000","last_became_active":"2024-04-24T23:34:54.383463+0000","last_became_peered":"2024-04-24T23:34:54.383463+0000","last_unstale":"2024-04-24T23:35:00.541061+0000","last_undegraded":"2024-04-24T23:35:00.541061+0000","last_fullsized":"2024-04-24T23:35:00.541061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:51.634876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540883+0000","last_change":"2024-04-24T23:34:54.380959+0000","last_active":"2024-04-24T23:35:00.540883+0000","last_peered":"2024-04-24T23:35:00.540883+0000","last_clean":"2024-04-24T23:35:00.540883+0000","last_became_active":"2024-04-24T23:34:54.380702+0000","last_became_peered":"2024-04-24T23:34:54.380702+0000","last_unstale":"2024-04-24T23:35:00.540883+0000","last_undegraded":"2024-04-24T23:35:00.540883+0000","last_fullsized":"2024-04-24T23:35:00.540883+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:45.424046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.880840+0000","last_change":"2024-04-24T23:34:54.358461+0000","last_active":"2024-04-24T23:34:59.880840+0000","last_peered":"2024-04-24T23:34:59.880840+0000","last_clean":"2024-04-24T23:34:59.880840+0000","last_became_active":"2024-04-24T23:34:54.358203+0000","last_became_peered":"2024-04-24T23:34:54.358203+0000","last_unstale":"2024-04-24T23:34:59.880840+0000","last_undegraded":"2024-04-24T23:34:59.880840+0000","last_fullsized":"2024-04-24T23:34:59.880840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:18.207267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.427594+0000","last_change":"2024-04-24T23:34:54.367860+0000","last_active":"2024-04-24T23:34:59.427594+0000","last_peered":"2024-04-24T23:34:59.427594+0000","last_clean":"2024-04-24T23:34:59.427594+0000","last_became_active":"2024-04-24T23:34:54.367595+0000","last_became_peered":"2024-04-24T23:34:54.367595+0000","last_unstale":"2024-04-24T23:34:59.427594+0000","last_undegraded":"2024-04-24T23:34:59.427594+0000","last_fullsized":"2024-04-24T23:34:59.427594+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:40:40.928386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396558+0000","last_change":"2024-04-24T23:34:54.373640+0000","last_active":"2024-04-24T23:34:59.396558+0000","last_peered":"2024-04-24T23:34:59.396558+0000","last_clean":"2024-04-24T23:34:59.396558+0000","last_became_active":"2024-04-24T23:34:54.373404+0000","last_became_peered":"2024-04-24T23:34:54.373404+0000","last_unstale":"2024-04-24T23:34:59.396558+0000","last_undegraded":"2024-04-24T23:34:59.396558+0000","last_fullsized":"2024-04-24T23:34:59.396558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:41:21.841042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397052+0000","last_change":"2024-04-24T23:34:54.377573+0000","last_active":"2024-04-24T23:34:59.397052+0000","last_peered":"2024-04-24T23:34:59.397052+0000","last_clean":"2024-04-24T23:34:59.397052+0000","last_became_active":"2024-04-24T23:34:54.377277+0000","last_became_peered":"2024-04-24T23:34:54.377277+0000","last_unstale":"2024-04-24T23:34:59.397052+0000","last_undegraded":"2024-04-24T23:34:59.397052+0000","last_fullsized":"2024-04-24T23:34:59.397052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:43.211314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540693+0000","last_change":"2024-04-24T23:34:54.388134+0000","last_active":"2024-04-24T23:35:00.540693+0000","last_peered":"2024-04-24T23:35:00.540693+0000","last_clean":"2024-04-24T23:35:00.540693+0000","last_became_active":"2024-04-24T23:34:54.387781+0000","last_became_peered":"2024-04-24T23:34:54.387781+0000","last_unstale":"2024-04-24T23:35:00.540693+0000","last_undegraded":"2024-04-24T23:35:00.540693+0000","last_fullsized":"2024-04-24T23:35:00.540693+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:56:17.763159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.541198+0000","last_change":"2024-04-24T23:34:54.388182+0000","last_active":"2024-04-24T23:35:00.541198+0000","last_peered":"2024-04-24T23:35:00.541198+0000","last_clean":"2024-04-24T23:35:00.541198+0000","last_became_active":"2024-04-24T23:34:54.387913+0000","last_became_peered":"2024-04-24T23:34:54.387913+0000","last_unstale":"2024-04-24T23:35:00.541198+0000","last_undegraded":"2024-04-24T23:35:00.541198+0000","last_fullsized":"2024-04-24T23:35:00.541198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:18.207267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.502012+0000","last_change":"2024-04-24T23:34:54.383303+0000","last_active":"2024-04-24T23:34:59.502012+0000","last_peered":"2024-04-24T23:34:59.502012+0000","last_clean":"2024-04-24T23:34:59.502012+0000","last_became_active":"2024-04-24T23:34:54.383024+0000","last_became_peered":"2024-04-24T23:34:54.383024+0000","last_unstale":"2024-04-24T23:34:59.502012+0000","last_undegraded":"2024-04-24T23:34:59.502012+0000","last_fullsized":"2024-04-24T23:34:59.502012+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:45.424046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.395629+0000","last_change":"2024-04-24T23:34:54.382803+0000","last_active":"2024-04-24T23:34:59.395629+0000","last_peered":"2024-04-24T23:34:59.395629+0000","last_clean":"2024-04-24T23:34:59.395629+0000","last_became_active":"2024-04-24T23:34:54.382482+0000","last_became_peered":"2024-04-24T23:34:54.382482+0000","last_unstale":"2024-04-24T23:34:59.395629+0000","last_undegraded":"2024-04-24T23:34:59.395629+0000","last_fullsized":"2024-04-24T23:34:59.395629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:27:33.721584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501319+0000","last_change":"2024-04-24T23:34:54.369167+0000","last_active":"2024-04-24T23:34:59.501319+0000","last_peered":"2024-04-24T23:34:59.501319+0000","last_clean":"2024-04-24T23:34:59.501319+0000","last_became_active":"2024-04-24T23:34:54.368663+0000","last_became_peered":"2024-04-24T23:34:54.368663+0000","last_unstale":"2024-04-24T23:34:59.501319+0000","last_undegraded":"2024-04-24T23:34:59.501319+0000","last_fullsized":"2024-04-24T23:34:59.501319+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:18.207267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540351+0000","last_change":"2024-04-24T23:34:54.380345+0000","last_active":"2024-04-24T23:35:00.540351+0000","last_peered":"2024-04-24T23:35:00.540351+0000","last_clean":"2024-04-24T23:35:00.540351+0000","last_became_active":"2024-04-24T23:34:54.380052+0000","last_became_peered":"2024-04-24T23:34:54.380052+0000","last_unstale":"2024-04-24T23:35:00.540351+0000","last_undegraded":"2024-04-24T23:35:00.540351+0000","last_fullsized":"2024-04-24T23:35:00.540351+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:42.199856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501438+0000","last_change":"2024-04-24T23:34:54.365114+0000","last_active":"2024-04-24T23:34:59.501438+0000","last_peered":"2024-04-24T23:34:59.501438+0000","last_clean":"2024-04-24T23:34:59.501438+0000","last_became_active":"2024-04-24T23:34:54.364605+0000","last_became_peered":"2024-04-24T23:34:54.364605+0000","last_unstale":"2024-04-24T23:34:59.501438+0000","last_undegraded":"2024-04-24T23:34:59.501438+0000","last_fullsized":"2024-04-24T23:34:59.501438+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:07:28.785202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540539+0000","last_change":"2024-04-24T23:34:54.374843+0000","last_active":"2024-04-24T23:35:00.540539+0000","last_peered":"2024-04-24T23:35:00.540539+0000","last_clean":"2024-04-24T23:35:00.540539+0000","last_became_active":"2024-04-24T23:34:54.373117+0000","last_became_peered":"2024-04-24T23:34:54.373117+0000","last_unstale":"2024-04-24T23:35:00.540539+0000","last_undegraded":"2024-04-24T23:35:00.540539+0000","last_fullsized":"2024-04-24T23:35:00.540539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:37.323551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.427656+0000","last_change":"2024-04-24T23:34:54.366079+0000","last_active":"2024-04-24T23:34:59.427656+0000","last_peered":"2024-04-24T23:34:59.427656+0000","last_clean":"2024-04-24T23:34:59.427656+0000","last_became_active":"2024-04-24T23:34:54.365814+0000","last_became_peered":"2024-04-24T23:34:54.365814+0000","last_unstale":"2024-04-24T23:34:59.427656+0000","last_undegraded":"2024-04-24T23:34:59.427656+0000","last_fullsized":"2024-04-24T23:34:59.427656+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:18.207267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.881393+0000","last_change":"2024-04-24T23:34:54.368976+0000","last_active":"2024-04-24T23:34:59.881393+0000","last_peered":"2024-04-24T23:34:59.881393+0000","last_clean":"2024-04-24T23:34:59.881393+0000","last_became_active":"2024-04-24T23:34:54.368663+0000","last_became_peered":"2024-04-24T23:34:54.368663+0000","last_unstale":"2024-04-24T23:34:59.881393+0000","last_undegraded":"2024-04-24T23:34:59.881393+0000","last_fullsized":"2024-04-24T23:34:59.881393+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:56:17.763159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501905+0000","last_change":"2024-04-24T23:34:54.373810+0000","last_active":"2024-04-24T23:34:59.501905+0000","last_peered":"2024-04-24T23:34:59.501905+0000","last_clean":"2024-04-24T23:34:59.501905+0000","last_became_active":"2024-04-24T23:34:54.373531+0000","last_became_peered":"2024-04-24T23:34:54.373531+0000","last_unstale":"2024-04-24T23:34:59.501905+0000","last_undegraded":"2024-04-24T23:34:59.501905+0000","last_fullsized":"2024-04-24T23:34:59.501905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:50:17.621315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65248,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":331776,"data_stored":694125,"data_compressed":32624,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542151,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":274432,"data_stored":107017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":103265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":103301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":262144,"data_stored":103175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31444,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340396,"statfs":{"total":96636764160,"available":96604565504,"internally_reserved":0,"allocated":270336,"data_stored":103335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31520,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340320,"statfs":{"total":96636764160,"available":96604487680,"internally_reserved":0,"allocated":339968,"data_stored":694181,"data_compressed":32624,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31444,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340396,"statfs":{"total":96636764160,"available":96604565504,"internally_reserved":0,"allocated":274432,"data_stored":107017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32624,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32624,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:35:17.755 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-24T23:35:17.755 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-24T23:35:17.755 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-24T23:35:17.755 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-24T23:35:17.756 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-24T23:35:17.887 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.886+0000 7fef922a5640 1 Processor -- start 2024-04-24T23:35:17.887 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.887+0000 7fef922a5640 1 -- start start 2024-04-24T23:35:17.888 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.887+0000 7fef922a5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fef8c152b40 0x7fef8c0c5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:17.888 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.887+0000 7fef922a5640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef8c0c8bb0 con 0x7fef8c0c5b70 2024-04-24T23:35:17.888 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.887+0000 7fef922a5640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef8c159480 con 0x7fef8c0be360 2024-04-24T23:35:17.888 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.887+0000 7fef922a5640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fef8c0c8e90 con 0x7fef8c152b40 2024-04-24T23:35:17.889 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.887+0000 7fef8bfff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fef8c152b40 0x7fef8c0c5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:17.889 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.887+0000 7fef90842640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fef8c0c5b70 0x7fef8c15fcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:47622/0 (socket says 172.21.15.157:47622) 2024-04-24T23:35:17.889 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.887+0000 7fef90842640 1 -- 172.21.15.157:0/648346380 learned_addr learned my addr 172.21.15.157:0/648346380 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-24T23:35:17.889 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.888+0000 7fef8bfff640 1 -- 172.21.15.157:0/648346380 >> v1:172.21.15.169:6789/0 conn(0x7fef8c0be360 legacy=0x7fef8c155350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:17.889 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.888+0000 7fef8bfff640 1 -- 172.21.15.157:0/648346380 >> v1:172.21.15.157:6789/0 conn(0x7fef8c0c5b70 legacy=0x7fef8c15fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:17.889 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.888+0000 7fef8affd640 1 -- 172.21.15.157:0/648346380 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3573434328 0 0) 0x7fef8c0c8bb0 con 0x7fef8c0c5b70 2024-04-24T23:35:17.889 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.888+0000 7fef8bfff640 1 -- 172.21.15.157:0/648346380 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef8c159760 con 0x7fef8c152b40 2024-04-24T23:35:17.891 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.888+0000 7fef8bfff640 1 --2- 172.21.15.157:0/648346380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef8c152b40 0x7fef8c0c5630 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fef7c0097a0 tx=0x7fef7c02a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5647406766c3858 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:17.891 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.888+0000 7fef8affd640 1 -- 172.21.15.157:0/648346380 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7c037070 con 0x7fef8c152b40 2024-04-24T23:35:17.891 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.888+0000 7fef8affd640 1 -- 172.21.15.157:0/648346380 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef7c02adc0 con 0x7fef8c152b40 2024-04-24T23:35:17.891 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.889+0000 7fef8affd640 1 -- 172.21.15.157:0/648346380 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7c032660 con 0x7fef8c152b40 2024-04-24T23:35:17.891 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.889+0000 7fef922a5640 1 -- 172.21.15.157:0/648346380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef8c152b40 msgr2=0x7fef8c0c5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:17.891 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.889+0000 7fef922a5640 1 --2- 172.21.15.157:0/648346380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef8c152b40 0x7fef8c0c5630 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fef7c0097a0 tx=0x7fef7c02a0b0 comp rx=0 tx=0).stop 2024-04-24T23:35:17.891 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.889+0000 7fef922a5640 1 -- 172.21.15.157:0/648346380 shutdown_connections 2024-04-24T23:35:17.891 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.889+0000 7fef922a5640 1 --2- 172.21.15.157:0/648346380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef8c152b40 0x7fef8c0c5630 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:17.891 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.889+0000 7fef922a5640 1 -- 172.21.15.157:0/648346380 >> 172.21.15.157:0/648346380 conn(0x7fef8c0672c0 msgr2=0x7fef8c0676d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:17.891 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.889+0000 7fef922a5640 1 -- 172.21.15.157:0/648346380 shutdown_connections 2024-04-24T23:35:17.891 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.889+0000 7fef922a5640 1 -- 172.21.15.157:0/648346380 wait complete. 2024-04-24T23:35:17.891 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.890+0000 7fef922a5640 1 Processor -- start 2024-04-24T23:35:17.891 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.890+0000 7fef922a5640 1 -- start start 2024-04-24T23:35:17.894 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.891+0000 7fef922a5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef8c0be360 0x7fef8c157820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:17.894 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.891+0000 7fef922a5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef8c0c5b70 0x7fef8c0c4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:17.894 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.891+0000 7fef922a5640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef8c152b40 0x7fef8c147e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:17.894 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.891+0000 7fef922a5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fef8c0caa00 con 0x7fef8c0be360 2024-04-24T23:35:17.894 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.891+0000 7fef922a5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fef8c0cb210 con 0x7fef8c0c5b70 2024-04-24T23:35:17.894 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.891+0000 7fef922a5640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fef8c0cba40 con 0x7fef8c152b40 2024-04-24T23:35:17.894 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.891+0000 7fef8bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef8c0be360 0x7fef8c157820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:17.894 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.891+0000 7fef8bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef8c0be360 0x7fef8c157820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42532/0 (socket says 172.21.15.157:42532) 2024-04-24T23:35:17.895 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.891+0000 7fef8bfff640 1 -- 172.21.15.157:0/1236643145 learned_addr learned my addr 172.21.15.157:0/1236643145 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:17.895 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.891+0000 7fef8b7fe640 1 --2- 172.21.15.157:0/1236643145 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef8c0c5b70 0x7fef8c0c4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:17.895 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.891+0000 7fef90842640 1 --2- 172.21.15.157:0/1236643145 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef8c152b40 0x7fef8c147e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:17.895 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.891+0000 7fef8bfff640 1 -- 172.21.15.157:0/1236643145 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef8c152b40 msgr2=0x7fef8c147e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:17.895 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.891+0000 7fef8bfff640 1 --2- 172.21.15.157:0/1236643145 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef8c152b40 0x7fef8c147e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:17.895 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.891+0000 7fef8bfff640 1 -- 172.21.15.157:0/1236643145 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef8c0c5b70 msgr2=0x7fef8c0c4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:17.895 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.891+0000 7fef8bfff640 1 --2- 172.21.15.157:0/1236643145 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef8c0c5b70 0x7fef8c0c4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:17.895 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.891+0000 7fef8bfff640 1 -- 172.21.15.157:0/1236643145 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef8c1ebc00 con 0x7fef8c0be360 2024-04-24T23:35:17.895 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.891+0000 7fef90842640 1 --2- 172.21.15.157:0/1236643145 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef8c152b40 0x7fef8c147e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:35:17.895 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.891+0000 7fef8bfff640 1 --2- 172.21.15.157:0/1236643145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef8c0be360 0x7fef8c157820 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fef7c0098c0 tx=0x7fef7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:17.895 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.892+0000 7fef88ff9640 1 -- 172.21.15.157:0/1236643145 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7c041070 con 0x7fef8c0be360 2024-04-24T23:35:17.895 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.892+0000 7fef88ff9640 1 -- 172.21.15.157:0/1236643145 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef7c03c670 con 0x7fef8c0be360 2024-04-24T23:35:17.895 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.892+0000 7fef922a5640 1 -- 172.21.15.157:0/1236643145 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef8c151c40 con 0x7fef8c0be360 2024-04-24T23:35:17.895 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.892+0000 7fef88ff9640 1 -- 172.21.15.157:0/1236643145 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7c037040 con 0x7fef8c0be360 2024-04-24T23:35:17.895 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.892+0000 7fef922a5640 1 -- 172.21.15.157:0/1236643145 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef8c155dc0 con 0x7fef8c0be360 2024-04-24T23:35:17.896 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.893+0000 7fef922a5640 1 -- 172.21.15.157:0/1236643145 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef8c0c5070 con 0x7fef8c0be360 2024-04-24T23:35:17.896 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.893+0000 7fef88ff9640 1 -- 172.21.15.157:0/1236643145 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fef7c044070 con 0x7fef8c0be360 2024-04-24T23:35:17.896 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.893+0000 7fef88ff9640 1 --2- 172.21.15.157:0/1236643145 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fef7806be30 0x7fef7806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:17.896 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.894+0000 7fef88ff9640 1 -- 172.21.15.157:0/1236643145 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fef7c0ab8c0 con 0x7fef8c0be360 2024-04-24T23:35:17.896 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.894+0000 7fef8b7fe640 1 --2- 172.21.15.157:0/1236643145 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fef7806be30 0x7fef7806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:17.896 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.894+0000 7fef8b7fe640 1 --2- 172.21.15.157:0/1236643145 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fef7806be30 0x7fef7806e2f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fef8c0bdcc0 tx=0x7fef80009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:17.898 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:17.898+0000 7fef88ff9640 1 -- 172.21.15.157:0/1236643145 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fef8c0c5070 con 0x7fef8c0be360 2024-04-24T23:35:18.117 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.117+0000 7fef922a5640 1 -- 172.21.15.157:0/1236643145 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fef8c0681c0 con 0x7fef8c0be360 2024-04-24T23:35:18.120 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.120+0000 7fef88ff9640 1 -- 172.21.15.157:0/1236643145 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+172943 (secure 0 0 0) 0x7fef7c079fc0 con 0x7fef8c0be360 2024-04-24T23:35:18.122 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:35:18.124 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.122+0000 7fef922a5640 1 -- 172.21.15.157:0/1236643145 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fef7806be30 msgr2=0x7fef7806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:18.124 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.122+0000 7fef922a5640 1 --2- 172.21.15.157:0/1236643145 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fef7806be30 0x7fef7806e2f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fef8c0bdcc0 tx=0x7fef80009040 comp rx=0 tx=0).stop 2024-04-24T23:35:18.124 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.122+0000 7fef922a5640 1 -- 172.21.15.157:0/1236643145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef8c0be360 msgr2=0x7fef8c157820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:18.124 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.122+0000 7fef922a5640 1 --2- 172.21.15.157:0/1236643145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef8c0be360 0x7fef8c157820 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fef7c0098c0 tx=0x7fef7c0046c0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.124 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.122+0000 7fef922a5640 1 -- 172.21.15.157:0/1236643145 shutdown_connections 2024-04-24T23:35:18.124 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.122+0000 7fef922a5640 1 --2- 172.21.15.157:0/1236643145 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fef7806be30 0x7fef7806e2f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.124 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.122+0000 7fef922a5640 1 --2- 172.21.15.157:0/1236643145 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef8c152b40 0x7fef8c147e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.124 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.122+0000 7fef922a5640 1 --2- 172.21.15.157:0/1236643145 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef8c0c5b70 0x7fef8c0c4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.124 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.122+0000 7fef922a5640 1 --2- 172.21.15.157:0/1236643145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef8c0be360 0x7fef8c157820 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.124 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.123+0000 7fef922a5640 1 -- 172.21.15.157:0/1236643145 >> 172.21.15.157:0/1236643145 conn(0x7fef8c0672c0 msgr2=0x7fef8c14d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:18.125 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.123+0000 7fef922a5640 1 -- 172.21.15.157:0/1236643145 shutdown_connections 2024-04-24T23:35:18.125 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.123+0000 7fef922a5640 1 -- 172.21.15.157:0/1236643145 wait complete. 2024-04-24T23:35:18.134 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":5,"flags":0,"active_gid":4105,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":662378289},{"type":"v1","addr":"172.21.15.169:6833","nonce":662378289}]},"active_addr":"172.21.15.169:6833/662378289","active_change":"2024-04-24T23:34:41.756689+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.169:0","nonce":3897594677}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.169:0","nonce":1904984559}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.169:0","nonce":3363202111}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.169:0","nonce":2200307966}]}]} 2024-04-24T23:35:18.137 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-24T23:35:18.137 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-24T23:35:18.137 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:35:18.234 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.233+0000 7f40a1c9e640 1 Processor -- start 2024-04-24T23:35:18.234 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.233+0000 7f40a1c9e640 1 -- start start 2024-04-24T23:35:18.235 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.234+0000 7f40a1c9e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f409c070860 0x7f409c070c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.235 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.234+0000 7f40a1c9e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f409c153110 con 0x7f409c0c1590 2024-04-24T23:35:18.235 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.234+0000 7f40a1c9e640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f409c1533d0 con 0x7f409c0a5c50 2024-04-24T23:35:18.235 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.234+0000 7f40a1c9e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f409c153d00 con 0x7f409c070860 2024-04-24T23:35:18.235 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.234+0000 7f409b7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f409c070860 0x7f409c070c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:18.236 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.234+0000 7f409affd640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f409c0c1590 0x7f409c0c1e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:47636/0 (socket says 172.21.15.157:47636) 2024-04-24T23:35:18.236 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.234+0000 7f409affd640 1 -- 172.21.15.157:0/1233703588 learned_addr learned my addr 172.21.15.157:0/1233703588 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-24T23:35:18.236 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.235+0000 7f409b7fe640 1 -- 172.21.15.157:0/1233703588 >> v1:172.21.15.169:6789/0 conn(0x7f409c0a5c50 legacy=0x7f409c15da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:18.236 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.235+0000 7f409b7fe640 1 -- 172.21.15.157:0/1233703588 >> v1:172.21.15.157:6789/0 conn(0x7f409c0c1590 legacy=0x7f409c0c1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:18.236 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.235+0000 7f409b7fe640 1 -- 172.21.15.157:0/1233703588 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f409c1536b0 con 0x7f409c070860 2024-04-24T23:35:18.237 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.235+0000 7f409b7fe640 1 --2- 172.21.15.157:0/1233703588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f409c070860 0x7f409c070c40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f408c0097a0 tx=0x7f408c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79df74a8934bb776 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:18.237 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.235+0000 7f409a7fc640 1 -- 172.21.15.157:0/1233703588 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408c037070 con 0x7f409c070860 2024-04-24T23:35:18.237 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.235+0000 7f409a7fc640 1 -- 172.21.15.157:0/1233703588 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f408c032440 con 0x7f409c070860 2024-04-24T23:35:18.237 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.235+0000 7f409a7fc640 1 -- 172.21.15.157:0/1233703588 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408c032810 con 0x7f409c070860 2024-04-24T23:35:18.237 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.236+0000 7f40a1c9e640 1 -- 172.21.15.157:0/1233703588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f409c070860 msgr2=0x7f409c070c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:18.238 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.236+0000 7f40a1c9e640 1 --2- 172.21.15.157:0/1233703588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f409c070860 0x7f409c070c40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f408c0097a0 tx=0x7f408c02a1a0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.238 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.236+0000 7f40a1c9e640 1 -- 172.21.15.157:0/1233703588 shutdown_connections 2024-04-24T23:35:18.238 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.236+0000 7f40a1c9e640 1 --2- 172.21.15.157:0/1233703588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f409c070860 0x7f409c070c40 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.238 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.236+0000 7f40a1c9e640 1 -- 172.21.15.157:0/1233703588 >> 172.21.15.157:0/1233703588 conn(0x7f409c0a8720 msgr2=0x7f409c0a8b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:18.238 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.236+0000 7f40a1c9e640 1 -- 172.21.15.157:0/1233703588 shutdown_connections 2024-04-24T23:35:18.238 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.236+0000 7f40a1c9e640 1 -- 172.21.15.157:0/1233703588 wait complete. 2024-04-24T23:35:18.238 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.237+0000 7f40a1c9e640 1 Processor -- start 2024-04-24T23:35:18.240 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.237+0000 7f40a1c9e640 1 -- start start 2024-04-24T23:35:18.240 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.238+0000 7f40a1c9e640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f409c070860 0x7f409c1e94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.240 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.238+0000 7f40a1c9e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f409c0a5c50 0x7f409c149c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.241 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.238+0000 7f40a1c9e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f409c1e9a20 0x7f409c14a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.241 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.238+0000 7f40a1c9e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f409c150b40 con 0x7f409c1e9a20 2024-04-24T23:35:18.241 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.238+0000 7f40a1c9e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f409c1509c0 con 0x7f409c0a5c50 2024-04-24T23:35:18.241 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.238+0000 7f40a1c9e640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f409c0a62c0 con 0x7f409c070860 2024-04-24T23:35:18.241 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.238+0000 7f409bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f409c1e9a20 0x7f409c14a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:18.241 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.238+0000 7f409bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f409c1e9a20 0x7f409c14a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42556/0 (socket says 172.21.15.157:42556) 2024-04-24T23:35:18.241 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.238+0000 7f409bfff640 1 -- 172.21.15.157:0/1224659667 learned_addr learned my addr 172.21.15.157:0/1224659667 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:18.241 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.238+0000 7f409affd640 1 --2- 172.21.15.157:0/1224659667 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f409c0a5c50 0x7f409c149c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:18.241 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.238+0000 7f409b7fe640 1 --2- 172.21.15.157:0/1224659667 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f409c070860 0x7f409c1e94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:18.241 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.238+0000 7f409bfff640 1 -- 172.21.15.157:0/1224659667 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f409c070860 msgr2=0x7f409c1e94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:18.241 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.238+0000 7f409bfff640 1 --2- 172.21.15.157:0/1224659667 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f409c070860 0x7f409c1e94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.241 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.238+0000 7f409bfff640 1 -- 172.21.15.157:0/1224659667 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f409c0a5c50 msgr2=0x7f409c149c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:18.241 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.238+0000 7f409bfff640 1 --2- 172.21.15.157:0/1224659667 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f409c0a5c50 0x7f409c149c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.241 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.238+0000 7f409bfff640 1 -- 172.21.15.157:0/1224659667 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f409c0c59c0 con 0x7f409c1e9a20 2024-04-24T23:35:18.242 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.238+0000 7f409affd640 1 --2- 172.21.15.157:0/1224659667 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f409c0a5c50 0x7f409c149c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:35:18.242 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.238+0000 7f409b7fe640 1 --2- 172.21.15.157:0/1224659667 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f409c070860 0x7f409c1e94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:35:18.242 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.238+0000 7f409bfff640 1 --2- 172.21.15.157:0/1224659667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f409c1e9a20 0x7f409c14a190 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f409400c7a0 tx=0x7f409400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:18.242 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.238+0000 7f4083fff640 1 -- 172.21.15.157:0/1224659667 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4094003b70 con 0x7f409c1e9a20 2024-04-24T23:35:18.242 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.238+0000 7f4083fff640 1 -- 172.21.15.157:0/1224659667 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4094003d30 con 0x7f409c1e9a20 2024-04-24T23:35:18.242 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.238+0000 7f40a1c9e640 1 -- 172.21.15.157:0/1224659667 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f409c1e9f60 con 0x7f409c1e9a20 2024-04-24T23:35:18.242 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.239+0000 7f4083fff640 1 -- 172.21.15.157:0/1224659667 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f409400aa30 con 0x7f409c1e9a20 2024-04-24T23:35:18.242 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.239+0000 7f40a1c9e640 1 -- 172.21.15.157:0/1224659667 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f409c1e03b0 con 0x7f409c1e9a20 2024-04-24T23:35:18.242 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.240+0000 7f40a1c9e640 1 -- 172.21.15.157:0/1224659667 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f409c0ac2c0 con 0x7f409c1e9a20 2024-04-24T23:35:18.242 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.240+0000 7f4083fff640 1 -- 172.21.15.157:0/1224659667 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f409400abd0 con 0x7f409c1e9a20 2024-04-24T23:35:18.243 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.240+0000 7f4083fff640 1 --2- 172.21.15.157:0/1224659667 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f408806be30 0x7f408806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.243 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.241+0000 7f4083fff640 1 -- 172.21.15.157:0/1224659667 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f409408b130 con 0x7f409c1e9a20 2024-04-24T23:35:18.243 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.241+0000 7f409b7fe640 1 --2- 172.21.15.157:0/1224659667 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f408806be30 0x7f408806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:18.243 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.241+0000 7f409b7fe640 1 --2- 172.21.15.157:0/1224659667 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f408806be30 0x7f408806e2f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f408c034000 tx=0x7f408c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:18.245 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.245+0000 7f4083fff640 1 -- 172.21.15.157:0/1224659667 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f40940597a0 con 0x7f409c1e9a20 2024-04-24T23:35:18.422 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.421+0000 7f40a1c9e640 1 -- 172.21.15.157:0/1224659667 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f409c0681c0 con 0x7f409c1e9a20 2024-04-24T23:35:18.423 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.422+0000 7f4083fff640 1 -- 172.21.15.157:0/1224659667 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18538 (secure 0 0 0) 0x7f40940590a0 con 0x7f409c1e9a20 2024-04-24T23:35:18.423 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:35:18.423 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":25,"fsid":"223685fb-5b64-46a5-9857-3791c9c07df2","created":"2024-04-24T23:34:37.504202+0000","modified":"2024-04-24T23:34:58.744500+0000","last_up_change":"2024-04-24T23:34:47.584963+0000","last_in_change":"2024-04-24T23:34:39.632954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:34:47.796655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:34:52.177208+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:34:52.667022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:34:56.395517+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"95d14ac8-fccf-4261-be8e-edafa746f629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6809","nonce":4003822160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6811","nonce":4003822160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6815","nonce":4003822160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6813","nonce":4003822160}]},"public_addr":"172.21.15.157:6809/4003822160","cluster_addr":"172.21.15.157:6811/4003822160","heartbeat_back_addr":"172.21.15.157:6815/4003822160","heartbeat_front_addr":"172.21.15.157:6813/4003822160","state":["exists","up"]},{"osd":1,"uuid":"7be79da9-8eec-4990-9178-71d602e39d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6817","nonce":1557575576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6819","nonce":1557575576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6823","nonce":1557575576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6821","nonce":1557575576}]},"public_addr":"172.21.15.157:6817/1557575576","cluster_addr":"172.21.15.157:6819/1557575576","heartbeat_back_addr":"172.21.15.157:6823/1557575576","heartbeat_front_addr":"172.21.15.157:6821/1557575576","state":["exists","up"]},{"osd":2,"uuid":"0127ed00-b7c3-466d-bfb0-01b0fd315f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6825","nonce":2793948092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6827","nonce":2793948092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6831","nonce":2793948092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6829","nonce":2793948092}]},"public_addr":"172.21.15.157:6825/2793948092","cluster_addr":"172.21.15.157:6827/2793948092","heartbeat_back_addr":"172.21.15.157:6831/2793948092","heartbeat_front_addr":"172.21.15.157:6829/2793948092","state":["exists","up"]},{"osd":3,"uuid":"f2331acc-aa37-433e-868e-3561b6039562","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6801","nonce":422758549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6803","nonce":422758549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6807","nonce":422758549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6805","nonce":422758549}]},"public_addr":"172.21.15.157:6801/422758549","cluster_addr":"172.21.15.157:6803/422758549","heartbeat_back_addr":"172.21.15.157:6807/422758549","heartbeat_front_addr":"172.21.15.157:6805/422758549","state":["exists","up"]},{"osd":4,"uuid":"1998854b-7398-43f5-ab7d-79fa0debcf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6801","nonce":321910099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6803","nonce":321910099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6807","nonce":321910099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6805","nonce":321910099}]},"public_addr":"172.21.15.169:6801/321910099","cluster_addr":"172.21.15.169:6803/321910099","heartbeat_back_addr":"172.21.15.169:6807/321910099","heartbeat_front_addr":"172.21.15.169:6805/321910099","state":["exists","up"]},{"osd":5,"uuid":"c37153ff-9b2b-48b9-a684-2c82b6f39f15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6825","nonce":243321711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6827","nonce":243321711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6831","nonce":243321711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6829","nonce":243321711}]},"public_addr":"172.21.15.169:6825/243321711","cluster_addr":"172.21.15.169:6827/243321711","heartbeat_back_addr":"172.21.15.169:6831/243321711","heartbeat_front_addr":"172.21.15.169:6829/243321711","state":["exists","up"]},{"osd":6,"uuid":"8db79e1c-f36b-49cb-8d8a-a20c2aec57c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6809","nonce":3441420785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6811","nonce":3441420785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6815","nonce":3441420785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6813","nonce":3441420785}]},"public_addr":"172.21.15.169:6809/3441420785","cluster_addr":"172.21.15.169:6811/3441420785","heartbeat_back_addr":"172.21.15.169:6815/3441420785","heartbeat_front_addr":"172.21.15.169:6813/3441420785","state":["exists","up"]},{"osd":7,"uuid":"928586e9-296c-4b8b-9b27-42f9f155dbdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6817","nonce":1663179020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6819","nonce":1663179020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6823","nonce":1663179020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6821","nonce":1663179020}]},"public_addr":"172.21.15.169:6817/1663179020","cluster_addr":"172.21.15.169:6819/1663179020","heartbeat_back_addr":"172.21.15.169:6823/1663179020","heartbeat_front_addr":"172.21.15.169:6821/1663179020","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.848039+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.826489+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.801120+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.727706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.824343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.791915+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.878073+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.810831+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:35:18.425 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.424+0000 7f40a1c9e640 1 -- 172.21.15.157:0/1224659667 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f408806be30 msgr2=0x7f408806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:18.425 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.424+0000 7f40a1c9e640 1 --2- 172.21.15.157:0/1224659667 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f408806be30 0x7f408806e2f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f408c034000 tx=0x7f408c02e000 comp rx=0 tx=0).stop 2024-04-24T23:35:18.425 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.424+0000 7f40a1c9e640 1 -- 172.21.15.157:0/1224659667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f409c1e9a20 msgr2=0x7f409c14a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:18.426 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.424+0000 7f40a1c9e640 1 --2- 172.21.15.157:0/1224659667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f409c1e9a20 0x7f409c14a190 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f409400c7a0 tx=0x7f409400cc60 comp rx=0 tx=0).stop 2024-04-24T23:35:18.426 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.424+0000 7f40a1c9e640 1 -- 172.21.15.157:0/1224659667 shutdown_connections 2024-04-24T23:35:18.426 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.424+0000 7f40a1c9e640 1 --2- 172.21.15.157:0/1224659667 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f408806be30 0x7f408806e2f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.426 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.424+0000 7f40a1c9e640 1 --2- 172.21.15.157:0/1224659667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f409c1e9a20 0x7f409c14a190 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.426 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.424+0000 7f40a1c9e640 1 --2- 172.21.15.157:0/1224659667 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f409c0a5c50 0x7f409c149c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.426 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.424+0000 7f40a1c9e640 1 --2- 172.21.15.157:0/1224659667 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f409c070860 0x7f409c1e94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.426 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.424+0000 7f40a1c9e640 1 -- 172.21.15.157:0/1224659667 >> 172.21.15.157:0/1224659667 conn(0x7f409c0a8720 msgr2=0x7f409c152780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:18.426 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.425+0000 7f40a1c9e640 1 -- 172.21.15.157:0/1224659667 shutdown_connections 2024-04-24T23:35:18.426 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.425+0000 7f40a1c9e640 1 -- 172.21.15.157:0/1224659667 wait complete. 2024-04-24T23:35:18.436 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-24T23:35:18.436 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:35:18.534 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.533+0000 7ffa7f55a640 1 Processor -- start 2024-04-24T23:35:18.534 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.533+0000 7ffa7f55a640 1 -- start start 2024-04-24T23:35:18.537 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.534+0000 7ffa7f55a640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa7806b3d0 0x7ffa7806b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.537 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.534+0000 7ffa7f55a640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ffa780c2110 0x7ffa780ab5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.537 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.534+0000 7ffa7f55a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ffa780c3820 0x7ffa780c0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.537 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.534+0000 7ffa7f55a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7ffa7806e030 con 0x7ffa780c3820 2024-04-24T23:35:18.537 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.534+0000 7ffa7f55a640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7ffa7814f7f0 con 0x7ffa780c2110 2024-04-24T23:35:18.537 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.534+0000 7ffa7f55a640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ffa78155d60 con 0x7ffa7806b3d0 2024-04-24T23:35:18.537 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.534+0000 7ffa7daf7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ffa780c3820 0x7ffa780c0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:18.537 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.534+0000 7ffa7daf7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ffa780c3820 0x7ffa780c0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42566/0 (socket says 172.21.15.157:42566) 2024-04-24T23:35:18.537 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.534+0000 7ffa7daf7640 1 -- 172.21.15.157:0/2359369900 learned_addr learned my addr 172.21.15.157:0/2359369900 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:18.537 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.535+0000 7ffa7caf5640 1 --2- 172.21.15.157:0/2359369900 >> v2:172.21.15.169:3300/0 conn(0x7ffa780c2110 0x7ffa780ab5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:18.537 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.535+0000 7ffa7daf7640 1 -- 172.21.15.157:0/2359369900 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa7806b3d0 msgr2=0x7ffa7806b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:18.537 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.535+0000 7ffa7daf7640 1 --2- 172.21.15.157:0/2359369900 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa7806b3d0 0x7ffa7806b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.538 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.535+0000 7ffa7daf7640 1 -- 172.21.15.157:0/2359369900 >> v2:172.21.15.169:3300/0 conn(0x7ffa780c2110 msgr2=0x7ffa780ab5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:18.538 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.535+0000 7ffa7daf7640 1 --2- 172.21.15.157:0/2359369900 >> v2:172.21.15.169:3300/0 conn(0x7ffa780c2110 0x7ffa780ab5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.538 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.535+0000 7ffa7daf7640 1 -- 172.21.15.157:0/2359369900 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa78156040 con 0x7ffa780c3820 2024-04-24T23:35:18.538 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.535+0000 7ffa7daf7640 1 --2- 172.21.15.157:0/2359369900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa780c3820 0x7ffa780c0620 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ffa7806cbc0 tx=0x7ffa7002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad177be40d323aa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:18.538 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.535+0000 7ffa6ffff640 1 -- 172.21.15.157:0/2359369900 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa70036070 con 0x7ffa780c3820 2024-04-24T23:35:18.538 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.535+0000 7ffa6ffff640 1 -- 172.21.15.157:0/2359369900 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa70031600 con 0x7ffa780c3820 2024-04-24T23:35:18.538 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.535+0000 7ffa6ffff640 1 -- 172.21.15.157:0/2359369900 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa700319d0 con 0x7ffa780c3820 2024-04-24T23:35:18.538 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.536+0000 7ffa7f55a640 1 -- 172.21.15.157:0/2359369900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa780c3820 msgr2=0x7ffa780c0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:18.538 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.536+0000 7ffa7f55a640 1 --2- 172.21.15.157:0/2359369900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa780c3820 0x7ffa780c0620 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ffa7806cbc0 tx=0x7ffa7002a6a0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.538 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.536+0000 7ffa7f55a640 1 -- 172.21.15.157:0/2359369900 shutdown_connections 2024-04-24T23:35:18.538 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.536+0000 7ffa7f55a640 1 --2- 172.21.15.157:0/2359369900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa780c3820 0x7ffa780c0620 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.538 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.536+0000 7ffa7f55a640 1 --2- 172.21.15.157:0/2359369900 >> v2:172.21.15.169:3300/0 conn(0x7ffa780c2110 0x7ffa780ab5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.538 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.536+0000 7ffa7f55a640 1 --2- 172.21.15.157:0/2359369900 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa7806b3d0 0x7ffa7806b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.539 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.536+0000 7ffa7f55a640 1 -- 172.21.15.157:0/2359369900 >> 172.21.15.157:0/2359369900 conn(0x7ffa780c71f0 msgr2=0x7ffa780c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:18.539 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.536+0000 7ffa7f55a640 1 -- 172.21.15.157:0/2359369900 shutdown_connections 2024-04-24T23:35:18.539 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.536+0000 7ffa7f55a640 1 -- 172.21.15.157:0/2359369900 wait complete. 2024-04-24T23:35:18.542 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.537+0000 7ffa7f55a640 1 Processor -- start 2024-04-24T23:35:18.542 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.537+0000 7ffa7f55a640 1 -- start start 2024-04-24T23:35:18.542 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.538+0000 7ffa7f55a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa7806b3d0 0x7ffa781eedc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.542 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.538+0000 7ffa7f55a640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa780c2110 0x7ffa780c6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.542 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.538+0000 7ffa7f55a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa780c3820 0x7ffa781eb500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.542 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.538+0000 7ffa7f55a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ffa780c1bf0 con 0x7ffa7806b3d0 2024-04-24T23:35:18.542 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.538+0000 7ffa7f55a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ffa780ac6b0 con 0x7ffa780c3820 2024-04-24T23:35:18.542 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.538+0000 7ffa7f55a640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ffa7806bcf0 con 0x7ffa780c2110 2024-04-24T23:35:18.542 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.538+0000 7ffa7d2f6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa7806b3d0 0x7ffa781eedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:18.542 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.538+0000 7ffa7d2f6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa7806b3d0 0x7ffa781eedc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42580/0 (socket says 172.21.15.157:42580) 2024-04-24T23:35:18.542 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.538+0000 7ffa7d2f6640 1 -- 172.21.15.157:0/3856851302 learned_addr learned my addr 172.21.15.157:0/3856851302 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:18.543 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.538+0000 7ffa7daf7640 1 --2- 172.21.15.157:0/3856851302 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa780c3820 0x7ffa781eb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:18.543 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.538+0000 7ffa7caf5640 1 --2- 172.21.15.157:0/3856851302 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa780c2110 0x7ffa780c6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:18.543 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.538+0000 7ffa7d2f6640 1 -- 172.21.15.157:0/3856851302 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa780c2110 msgr2=0x7ffa780c6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:18.543 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.538+0000 7ffa7d2f6640 1 --2- 172.21.15.157:0/3856851302 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa780c2110 0x7ffa780c6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.543 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.538+0000 7ffa7d2f6640 1 -- 172.21.15.157:0/3856851302 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa780c3820 msgr2=0x7ffa781eb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:18.543 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.538+0000 7ffa7d2f6640 1 --2- 172.21.15.157:0/3856851302 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa780c3820 0x7ffa781eb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.543 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.538+0000 7ffa7d2f6640 1 -- 172.21.15.157:0/3856851302 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa78072cc0 con 0x7ffa7806b3d0 2024-04-24T23:35:18.543 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.538+0000 7ffa7caf5640 1 --2- 172.21.15.157:0/3856851302 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa780c2110 0x7ffa780c6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:35:18.543 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.538+0000 7ffa7d2f6640 1 --2- 172.21.15.157:0/3856851302 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa7806b3d0 0x7ffa781eedc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ffa7400b4d0 tx=0x7ffa7400b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:18.543 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.538+0000 7ffa6dffb640 1 -- 172.21.15.157:0/3856851302 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa74014070 con 0x7ffa7806b3d0 2024-04-24T23:35:18.543 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.539+0000 7ffa6dffb640 1 -- 172.21.15.157:0/3856851302 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa74010870 con 0x7ffa7806b3d0 2024-04-24T23:35:18.543 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.539+0000 7ffa6dffb640 1 -- 172.21.15.157:0/3856851302 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa74010c60 con 0x7ffa7806b3d0 2024-04-24T23:35:18.543 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.539+0000 7ffa7f55a640 1 -- 172.21.15.157:0/3856851302 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa78072fa0 con 0x7ffa7806b3d0 2024-04-24T23:35:18.543 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.539+0000 7ffa7f55a640 1 -- 172.21.15.157:0/3856851302 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa781eafe0 con 0x7ffa7806b3d0 2024-04-24T23:35:18.544 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.540+0000 7ffa7f55a640 1 -- 172.21.15.157:0/3856851302 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa781eb1e0 con 0x7ffa7806b3d0 2024-04-24T23:35:18.544 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.540+0000 7ffa6dffb640 1 -- 172.21.15.157:0/3856851302 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffa74017070 con 0x7ffa7806b3d0 2024-04-24T23:35:18.544 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.540+0000 7ffa6dffb640 1 --2- 172.21.15.157:0/3856851302 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7ffa5c06be60 0x7ffa5c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.544 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.541+0000 7ffa6dffb640 1 -- 172.21.15.157:0/3856851302 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ffa7408af10 con 0x7ffa7806b3d0 2024-04-24T23:35:18.544 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.541+0000 7ffa7caf5640 1 --2- 172.21.15.157:0/3856851302 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7ffa5c06be60 0x7ffa5c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:18.544 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.541+0000 7ffa7caf5640 1 --2- 172.21.15.157:0/3856851302 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7ffa5c06be60 0x7ffa5c06e320 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ffa780c6c00 tx=0x7ffa68008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:18.545 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.545+0000 7ffa6dffb640 1 -- 172.21.15.157:0/3856851302 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ffa74059610 con 0x7ffa7806b3d0 2024-04-24T23:35:18.717 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.717+0000 7ffa7f55a640 1 -- 172.21.15.157:0/3856851302 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffa780681c0 con 0x7ffa7806b3d0 2024-04-24T23:35:18.719 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.718+0000 7ffa6dffb640 1 -- 172.21.15.157:0/3856851302 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18538 (secure 0 0 0) 0x7ffa74058f10 con 0x7ffa7806b3d0 2024-04-24T23:35:18.719 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:35:18.719 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":25,"fsid":"223685fb-5b64-46a5-9857-3791c9c07df2","created":"2024-04-24T23:34:37.504202+0000","modified":"2024-04-24T23:34:58.744500+0000","last_up_change":"2024-04-24T23:34:47.584963+0000","last_in_change":"2024-04-24T23:34:39.632954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:34:47.796655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:34:52.177208+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:34:52.667022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:34:56.395517+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"95d14ac8-fccf-4261-be8e-edafa746f629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6809","nonce":4003822160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6811","nonce":4003822160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6815","nonce":4003822160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6813","nonce":4003822160}]},"public_addr":"172.21.15.157:6809/4003822160","cluster_addr":"172.21.15.157:6811/4003822160","heartbeat_back_addr":"172.21.15.157:6815/4003822160","heartbeat_front_addr":"172.21.15.157:6813/4003822160","state":["exists","up"]},{"osd":1,"uuid":"7be79da9-8eec-4990-9178-71d602e39d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6817","nonce":1557575576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6819","nonce":1557575576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6823","nonce":1557575576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6821","nonce":1557575576}]},"public_addr":"172.21.15.157:6817/1557575576","cluster_addr":"172.21.15.157:6819/1557575576","heartbeat_back_addr":"172.21.15.157:6823/1557575576","heartbeat_front_addr":"172.21.15.157:6821/1557575576","state":["exists","up"]},{"osd":2,"uuid":"0127ed00-b7c3-466d-bfb0-01b0fd315f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6825","nonce":2793948092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6827","nonce":2793948092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6831","nonce":2793948092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6829","nonce":2793948092}]},"public_addr":"172.21.15.157:6825/2793948092","cluster_addr":"172.21.15.157:6827/2793948092","heartbeat_back_addr":"172.21.15.157:6831/2793948092","heartbeat_front_addr":"172.21.15.157:6829/2793948092","state":["exists","up"]},{"osd":3,"uuid":"f2331acc-aa37-433e-868e-3561b6039562","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6801","nonce":422758549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6803","nonce":422758549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6807","nonce":422758549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6805","nonce":422758549}]},"public_addr":"172.21.15.157:6801/422758549","cluster_addr":"172.21.15.157:6803/422758549","heartbeat_back_addr":"172.21.15.157:6807/422758549","heartbeat_front_addr":"172.21.15.157:6805/422758549","state":["exists","up"]},{"osd":4,"uuid":"1998854b-7398-43f5-ab7d-79fa0debcf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6801","nonce":321910099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6803","nonce":321910099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6807","nonce":321910099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6805","nonce":321910099}]},"public_addr":"172.21.15.169:6801/321910099","cluster_addr":"172.21.15.169:6803/321910099","heartbeat_back_addr":"172.21.15.169:6807/321910099","heartbeat_front_addr":"172.21.15.169:6805/321910099","state":["exists","up"]},{"osd":5,"uuid":"c37153ff-9b2b-48b9-a684-2c82b6f39f15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6825","nonce":243321711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6827","nonce":243321711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6831","nonce":243321711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6829","nonce":243321711}]},"public_addr":"172.21.15.169:6825/243321711","cluster_addr":"172.21.15.169:6827/243321711","heartbeat_back_addr":"172.21.15.169:6831/243321711","heartbeat_front_addr":"172.21.15.169:6829/243321711","state":["exists","up"]},{"osd":6,"uuid":"8db79e1c-f36b-49cb-8d8a-a20c2aec57c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6809","nonce":3441420785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6811","nonce":3441420785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6815","nonce":3441420785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6813","nonce":3441420785}]},"public_addr":"172.21.15.169:6809/3441420785","cluster_addr":"172.21.15.169:6811/3441420785","heartbeat_back_addr":"172.21.15.169:6815/3441420785","heartbeat_front_addr":"172.21.15.169:6813/3441420785","state":["exists","up"]},{"osd":7,"uuid":"928586e9-296c-4b8b-9b27-42f9f155dbdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6817","nonce":1663179020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6819","nonce":1663179020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6823","nonce":1663179020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6821","nonce":1663179020}]},"public_addr":"172.21.15.169:6817/1663179020","cluster_addr":"172.21.15.169:6819/1663179020","heartbeat_back_addr":"172.21.15.169:6823/1663179020","heartbeat_front_addr":"172.21.15.169:6821/1663179020","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.848039+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.826489+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.801120+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.727706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.824343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.791915+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.878073+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.810831+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:35:18.721 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.720+0000 7ffa7f55a640 1 -- 172.21.15.157:0/3856851302 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7ffa5c06be60 msgr2=0x7ffa5c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:18.721 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.720+0000 7ffa7f55a640 1 --2- 172.21.15.157:0/3856851302 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7ffa5c06be60 0x7ffa5c06e320 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ffa780c6c00 tx=0x7ffa68008040 comp rx=0 tx=0).stop 2024-04-24T23:35:18.722 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.720+0000 7ffa7f55a640 1 -- 172.21.15.157:0/3856851302 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa7806b3d0 msgr2=0x7ffa781eedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:18.722 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.720+0000 7ffa7f55a640 1 --2- 172.21.15.157:0/3856851302 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa7806b3d0 0x7ffa781eedc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ffa7400b4d0 tx=0x7ffa7400b990 comp rx=0 tx=0).stop 2024-04-24T23:35:18.722 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.720+0000 7ffa7f55a640 1 -- 172.21.15.157:0/3856851302 shutdown_connections 2024-04-24T23:35:18.722 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.720+0000 7ffa7f55a640 1 --2- 172.21.15.157:0/3856851302 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7ffa5c06be60 0x7ffa5c06e320 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.722 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.720+0000 7ffa7f55a640 1 --2- 172.21.15.157:0/3856851302 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa780c3820 0x7ffa781eb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.722 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.720+0000 7ffa7f55a640 1 --2- 172.21.15.157:0/3856851302 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa780c2110 0x7ffa780c6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.722 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.720+0000 7ffa7f55a640 1 --2- 172.21.15.157:0/3856851302 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa7806b3d0 0x7ffa781eedc0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.722 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.720+0000 7ffa7f55a640 1 -- 172.21.15.157:0/3856851302 >> 172.21.15.157:0/3856851302 conn(0x7ffa780c71f0 msgr2=0x7ffa7806f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:18.722 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.721+0000 7ffa7f55a640 1 -- 172.21.15.157:0/3856851302 shutdown_connections 2024-04-24T23:35:18.722 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.721+0000 7ffa7f55a640 1 -- 172.21.15.157:0/3856851302 wait complete. 2024-04-24T23:35:18.733 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-24T23:35:18.733 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-24T23:35:18.734 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-24T23:35:18.734 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-24T23:35:18.734 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-24T23:35:18.735 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-24T23:35:18.735 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-24T23:35:18.736 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-24T23:35:18.903 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.903+0000 7f7e21a4d640 1 Processor -- start 2024-04-24T23:35:18.904 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.903+0000 7f04bfd90640 1 Processor -- start 2024-04-24T23:35:18.904 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.903+0000 7f6f6fd1b640 1 Processor -- start 2024-04-24T23:35:18.904 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.903+0000 7f7e21a4d640 1 -- start start 2024-04-24T23:35:18.905 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.903+0000 7f04bfd90640 1 -- start start 2024-04-24T23:35:18.905 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.904+0000 7fd46a173640 1 Processor -- start 2024-04-24T23:35:18.905 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.904+0000 7f6f6fd1b640 1 -- start start 2024-04-24T23:35:18.906 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.904+0000 7fd46a173640 1 -- start start 2024-04-24T23:35:18.906 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.904+0000 7fe49ae29640 1 Processor -- start 2024-04-24T23:35:18.906 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.904+0000 7fe49ae29640 1 -- start start 2024-04-24T23:35:18.906 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.905+0000 7fa31a25d640 1 Processor -- start 2024-04-24T23:35:18.907 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.905+0000 7f04bfd90640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f04b0047f60 0x7f04b003f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.907 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.905+0000 7fa31a25d640 1 -- start start 2024-04-24T23:35:18.907 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.905+0000 7f6f6fd1b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6f6806d340 0x7f6f680c1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.908 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.905+0000 7fd46a173640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd4640c60c0 0x7fd46406ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.908 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.905+0000 7f6f6fd1b640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6f6806d720 0x7f6f68151ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.908 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.905+0000 7f7e21a4d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7e1c0c8510 0x7f7e1c153390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.908 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.905+0000 7fd46a173640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4640c2b60 0x7fd46415d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.908 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.905+0000 7fd46a173640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4640c64a0 con 0x7fd4640c2720 2024-04-24T23:35:18.909 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.905+0000 7fd46a173640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fd4640c2320 con 0x7fd4640c60c0 2024-04-24T23:35:18.909 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.905+0000 7fd46a173640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd46415a100 con 0x7fd4640c2b60 2024-04-24T23:35:18.909 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.906+0000 7f6f6fd1b640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f68072ec0 0x7f6f68152220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.909 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.906+0000 7f6f6fd1b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f6f680c2230 con 0x7f6f6806d340 2024-04-24T23:35:18.909 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.906+0000 7f6f6fd1b640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f6f680c2510 con 0x7f6f6806d720 2024-04-24T23:35:18.910 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.906+0000 7fe49ae29640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe494071230 0x7fe494150700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.910 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.906+0000 7f6f6fd1b640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6f680c27f0 con 0x7f6f68072ec0 2024-04-24T23:35:18.910 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.906+0000 7f6f6d2b6640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6f6806d720 0x7f6f68151ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:18.911 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.905+0000 7f7e21a4d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e1c0abb90 con 0x7f7e1c0c4740 2024-04-24T23:35:18.911 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.905+0000 7f7e21a4d640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e1c0ace60 con 0x7f7e1c0c4b20 2024-04-24T23:35:18.911 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.905+0000 7f7e21a4d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f7e1c0c4f50 con 0x7f7e1c0c8510 2024-04-24T23:35:18.911 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.906+0000 7fd468970640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fd4640c2720 0x7fd46415cbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:47640/0 (socket says 172.21.15.157:47640) 2024-04-24T23:35:18.911 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.906+0000 7fd468970640 1 -- 172.21.15.157:0/1677767136 learned_addr learned my addr 172.21.15.157:0/1677767136 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-24T23:35:18.911 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.906+0000 7fa31a25d640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa30c0d7b10 0x7fa30c0071e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.912 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.906+0000 7fe49ae29640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4940c2f30 con 0x7fe494150c40 2024-04-24T23:35:18.912 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.906+0000 7fe498bc5640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fe494072d60 0x7fe494073140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.157:42536/0 (socket says 172.21.15.157:42536) 2024-04-24T23:35:18.912 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.906+0000 7fa31a25d640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa30c04db10 0x7fa30c04ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.912 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.906+0000 7fe498bc5640 1 -- 172.21.15.157:0/450435316 learned_addr learned my addr 172.21.15.157:0/450435316 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-24T23:35:18.913 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.906+0000 7fa31a25d640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa30c00fa00 con 0x7fa30c04e020 2024-04-24T23:35:18.913 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.906+0000 7fa31a25d640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fa30c0da4e0 con 0x7fa30c04db10 2024-04-24T23:35:18.913 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.906+0000 7fa31a25d640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa30c0da7c0 con 0x7fa30c0d7b10 2024-04-24T23:35:18.913 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.906+0000 7fe493fff640 1 --2- 172.21.15.157:0/450435316 >> v2:172.21.15.169:3300/0 conn(0x7fe494071230 0x7fe494150700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:18.913 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.906+0000 7f7e1affd640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f7e1c0c4740 0x7f7e1c0c8980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:47642/0 (socket says 172.21.15.157:47642) 2024-04-24T23:35:18.913 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.906+0000 7f7e1affd640 1 -- 172.21.15.157:0/1508436491 learned_addr learned my addr 172.21.15.157:0/1508436491 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-24T23:35:18.914 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.906+0000 7f7e13fff640 1 --2- 172.21.15.157:0/1508436491 >> v2:172.21.15.157:3300/0 conn(0x7f7e1c0c8510 0x7f7e1c153390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:18.914 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.906+0000 7fe49ae29640 1 -- 172.21.15.157:0/450435316 --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4940c2c50 con 0x7fe494072d60 2024-04-24T23:35:18.914 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.906+0000 7fd469171640 1 --2- 172.21.15.157:0/1677767136 >> v2:172.21.15.169:3300/0 conn(0x7fd4640c60c0 0x7fd46406ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:18.914 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.906+0000 7fe49ae29640 1 -- 172.21.15.157:0/450435316 --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fe4940c15c0 con 0x7fe494071230 2024-04-24T23:35:18.915 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.906+0000 7fd469972640 1 --2- 172.21.15.157:0/1677767136 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4640c2b60 0x7fd46415d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:18.915 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.906+0000 7fa313fff640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa30c04db10 0x7fa30c04ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:18.915 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.906+0000 7fa313fff640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa30c04db10 0x7fa30c04ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.157:44866/0 (socket says 172.21.15.157:44866) 2024-04-24T23:35:18.915 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.906+0000 7fa313fff640 1 -- 172.21.15.157:0/1048643049 learned_addr learned my addr 172.21.15.157:0/1048643049 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:18.916 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.905+0000 7f04bfd90640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04b0049f50 0x7f04b004a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.916 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.905+0000 7f04bfd90640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04b004be20 con 0x7f04b000be70 2024-04-24T23:35:18.916 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.905+0000 7f04bfd90640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f04b003f940 con 0x7f04b0047f60 2024-04-24T23:35:18.916 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.905+0000 7f04bfd90640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f04b00de9e0 con 0x7f04b0049f50 2024-04-24T23:35:18.916 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.906+0000 7f04bd32b640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f04b0047f60 0x7f04b003f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:18.916 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.906+0000 7f6f6d2b6640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6f6806d720 0x7f6f68151ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.157:44840/0 (socket says 172.21.15.157:44840) 2024-04-24T23:35:18.916 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.906+0000 7f6f6d2b6640 1 -- 172.21.15.157:0/1891382917 learned_addr learned my addr 172.21.15.157:0/1891382917 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:18.916 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.906+0000 7f6f6dab7640 1 --2- 172.21.15.157:0/1891382917 >> v2:172.21.15.157:3300/0 conn(0x7f6f6806d340 0x7f6f680c1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:18.917 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.906+0000 7fe4937fe640 1 -- 172.21.15.157:0/450435316 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 515574411 0 0) 0x7fe4940c2c50 con 0x7fe494072d60 2024-04-24T23:35:18.917 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.906+0000 7fe4937fe640 1 -- 172.21.15.157:0/450435316 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe484003520 con 0x7fe494072d60 2024-04-24T23:35:18.917 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.907+0000 7f04bd32b640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f04b0047f60 0x7f04b003f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.157:44826/0 (socket says 172.21.15.157:44826) 2024-04-24T23:35:18.917 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.907+0000 7f04bd32b640 1 -- 172.21.15.157:0/1972152028 learned_addr learned my addr 172.21.15.157:0/1972152028 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:18.917 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.907+0000 7ff438b01640 1 Processor -- start 2024-04-24T23:35:18.918 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.907+0000 7f04be32d640 1 --2- 172.21.15.157:0/1972152028 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04b0049f50 0x7f04b004a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:18.918 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.907+0000 7f7e1a7fc640 1 -- 172.21.15.157:0/1508436491 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 647138203 0 0) 0x7f7e1c0abb90 con 0x7f7e1c0c4740 2024-04-24T23:35:18.918 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.907+0000 7f7e1a7fc640 1 -- 172.21.15.157:0/1508436491 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e08003520 con 0x7f7e1c0c4740 2024-04-24T23:35:18.919 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.907+0000 7fa3137fe640 1 --2- 172.21.15.157:0/1048643049 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa30c0d7b10 0x7fa30c0071e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:18.919 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.907+0000 7fe4937fe640 1 -- 172.21.15.157:0/450435316 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1147037363 0 0) 0x7fe484003520 con 0x7fe494072d60 2024-04-24T23:35:18.919 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.907+0000 7fe4937fe640 1 -- 172.21.15.157:0/450435316 >> v2:172.21.15.169:3300/0 conn(0x7fe494071230 msgr2=0x7fe494150700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:18.919 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.907+0000 7fe4937fe640 1 --2- 172.21.15.157:0/450435316 >> v2:172.21.15.169:3300/0 conn(0x7fe494071230 0x7fe494150700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.919 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.907+0000 7fe4937fe640 1 -- 172.21.15.157:0/450435316 >> v1:172.21.15.157:6789/0 conn(0x7fe494150c40 legacy=0x7fe4940c1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:18.919 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.907+0000 7fe4937fe640 1 -- 172.21.15.157:0/450435316 --> v1:172.21.15.169:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe494157330 con 0x7fe494072d60 2024-04-24T23:35:18.919 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.907+0000 7fe4937fe640 1 -- 172.21.15.157:0/450435316 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1480525340 0 0) 0x7fe488001670 con 0x7fe494072d60 2024-04-24T23:35:18.921 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.907+0000 7fe4937fe640 1 -- 172.21.15.157:0/450435316 >> v1:172.21.15.169:6789/0 conn(0x7fe494072d60 legacy=0x7fe494073140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:18.921 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.907+0000 7fe4937fe640 1 --2- 172.21.15.157:0/450435316 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe4840039b0 0x7fe484003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.921 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.907+0000 7fe4937fe640 1 --2- 172.21.15.157:0/450435316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe484004800 0x7fe484004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.921 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.907+0000 7fe4937fe640 1 --2- 172.21.15.157:0/450435316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4840051c0 0x7fe4840145c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.922 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.907+0000 7fe4937fe640 1 -- 172.21.15.157:0/450435316 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe4940c15c0 con 0x7fe4840051c0 2024-04-24T23:35:18.922 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.907+0000 7fe4937fe640 1 -- 172.21.15.157:0/450435316 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe484014b00 con 0x7fe484004800 2024-04-24T23:35:18.922 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.907+0000 7fe4937fe640 1 -- 172.21.15.157:0/450435316 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe484014c80 con 0x7fe4840039b0 2024-04-24T23:35:18.922 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.907+0000 7f04bd32b640 1 -- 172.21.15.157:0/1972152028 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04b0049f50 msgr2=0x7f04b004a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:18.922 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.907+0000 7f04bd32b640 1 --2- 172.21.15.157:0/1972152028 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04b0049f50 0x7f04b004a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.922 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.907+0000 7f04bd32b640 1 -- 172.21.15.157:0/1972152028 >> v1:172.21.15.169:6789/0 conn(0x7f04b000be70 legacy=0x7f04b0007b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:18.922 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.907+0000 7f04bd32b640 1 -- 172.21.15.157:0/1972152028 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04b00decc0 con 0x7f04b0047f60 2024-04-24T23:35:18.923 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.907+0000 7fd469171640 1 -- 172.21.15.157:0/1677767136 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4640c2b60 msgr2=0x7fd46415d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:18.923 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.907+0000 7fd469171640 1 --2- 172.21.15.157:0/1677767136 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4640c2b60 0x7fd46415d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.923 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.907+0000 7fd469171640 1 -- 172.21.15.157:0/1677767136 >> v1:172.21.15.157:6789/0 conn(0x7fd4640c2720 legacy=0x7fd46415cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:18.923 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.907+0000 7fd469171640 1 -- 172.21.15.157:0/1677767136 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4641597d0 con 0x7fd4640c60c0 2024-04-24T23:35:18.923 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.908+0000 7fd469171640 1 --2- 172.21.15.157:0/1677767136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4640c60c0 0x7fd46406ddb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd4600097a0 tx=0x7fd46002a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb97cc1407529b1b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:18.923 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.907+0000 7fa313fff640 1 -- 172.21.15.157:0/1048643049 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa30c0d7b10 msgr2=0x7fa30c0071e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:18.923 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.907+0000 7fa313fff640 1 --2- 172.21.15.157:0/1048643049 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa30c0d7b10 0x7fa30c0071e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.924 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.907+0000 7f6f6e2b8640 1 --2- 172.21.15.157:0/1891382917 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f68072ec0 0x7f6f68152220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:18.924 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.907+0000 7fa313fff640 1 -- 172.21.15.157:0/1048643049 >> v1:172.21.15.169:6789/0 conn(0x7fa30c04e020 legacy=0x7fa30c04e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:18.924 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.907+0000 7fa313fff640 1 -- 172.21.15.157:0/1048643049 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa30c0daaa0 con 0x7fa30c04db10 2024-04-24T23:35:18.924 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.908+0000 7f7e13fff640 1 -- 172.21.15.157:0/1508436491 >> v1:172.21.15.169:6789/0 conn(0x7f7e1c0c4b20 legacy=0x7f7e1c0a7a60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T23:35:18.924 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.908+0000 7f7e13fff640 1 -- 172.21.15.157:0/1508436491 >> v1:172.21.15.157:6789/0 conn(0x7f7e1c0c4740 legacy=0x7f7e1c0c8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:18.925 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.908+0000 7f7e13fff640 1 -- 172.21.15.157:0/1508436491 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e1c0ad140 con 0x7f7e1c0c8510 2024-04-24T23:35:18.925 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.907+0000 7f6f6d2b6640 1 -- 172.21.15.157:0/1891382917 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f68072ec0 msgr2=0x7f6f68152220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:18.926 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.907+0000 7ff438b01640 1 -- start start 2024-04-24T23:35:18.926 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.908+0000 7fd45bfff640 1 -- 172.21.15.157:0/1677767136 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd460037070 con 0x7fd4640c60c0 2024-04-24T23:35:18.927 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.907+0000 7f6f6d2b6640 1 --2- 172.21.15.157:0/1891382917 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f68072ec0 0x7f6f68152220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.927 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.907+0000 7f6f6d2b6640 1 -- 172.21.15.157:0/1891382917 >> v2:172.21.15.157:3300/0 conn(0x7f6f6806d340 msgr2=0x7f6f680c1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:18.927 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.907+0000 7f6f6d2b6640 1 --2- 172.21.15.157:0/1891382917 >> v2:172.21.15.157:3300/0 conn(0x7f6f6806d340 0x7f6f680c1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.928 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.907+0000 7f6f6d2b6640 1 -- 172.21.15.157:0/1891382917 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f680c2ad0 con 0x7f6f6806d720 2024-04-24T23:35:18.928 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.907+0000 7fe4993c6640 1 --2- 172.21.15.157:0/450435316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4840051c0 0x7fe4840145c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:18.928 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.908+0000 7fe498bc5640 1 --2- 172.21.15.157:0/450435316 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe4840039b0 0x7fe484003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:18.928 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.908+0000 7fa313fff640 1 --2- 172.21.15.157:0/1048643049 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa30c04db10 0x7fa30c04ef10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa31406a550 tx=0x7fa3140a5a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5eb51f800fa617b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:18.929 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.908+0000 7fe498bc5640 1 -- 172.21.15.157:0/450435316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe484004800 msgr2=0x7fe484004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:18.929 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.908+0000 7fe493fff640 1 --2- 172.21.15.157:0/450435316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe484004800 0x7fe484004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:18.929 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.908+0000 7fe498bc5640 1 --2- 172.21.15.157:0/450435316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe484004800 0x7fe484004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.929 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.908+0000 7fd45bfff640 1 -- 172.21.15.157:0/1677767136 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd460032440 con 0x7fd4640c60c0 2024-04-24T23:35:18.929 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.908+0000 7fd45bfff640 1 -- 172.21.15.157:0/1677767136 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd46002ae60 con 0x7fd4640c60c0 2024-04-24T23:35:18.930 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.908+0000 7f04bd32b640 1 --2- 172.21.15.157:0/1972152028 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04b0047f60 0x7f04b003f240 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f04a8009870 tx=0x7f04a802a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a680ec261a1e6cf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:18.930 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.908+0000 7fe498bc5640 1 -- 172.21.15.157:0/450435316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4840051c0 msgr2=0x7fe4840145c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:18.930 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.908+0000 7fe498bc5640 1 --2- 172.21.15.157:0/450435316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4840051c0 0x7fe4840145c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.930 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.908+0000 7fe498bc5640 1 -- 172.21.15.157:0/450435316 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe494157330 con 0x7fe4840039b0 2024-04-24T23:35:18.930 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.908+0000 7fe4993c6640 1 -- 172.21.15.157:0/450435316 reap_dead start 2024-04-24T23:35:18.931 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.909+0000 7f6f6d2b6640 1 --2- 172.21.15.157:0/1891382917 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f6806d720 0x7f6f68151ce0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6f680c44a0 tx=0x7f6f5802a0b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=543b668abea7ac2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:18.931 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.909+0000 7ff438b01640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff42c047490 0x7ff42c006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.931 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.909+0000 7ff438b01640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff42c0073d0 0x7ff42c00a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.931 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.909+0000 7ff438b01640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff42c00bb40 con 0x7ff42c0dbcd0 2024-04-24T23:35:18.931 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.909+0000 7ff438b01640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7ff42c00a410 con 0x7ff42c0073d0 2024-04-24T23:35:18.932 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.909+0000 7ff438b01640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7ff42c048050 con 0x7ff42c047490 2024-04-24T23:35:18.932 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.909+0000 7f04bcb2a640 1 -- 172.21.15.157:0/1972152028 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a8037070 con 0x7f04b0047f60 2024-04-24T23:35:18.932 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.909+0000 7f04bcb2a640 1 -- 172.21.15.157:0/1972152028 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04a8032440 con 0x7f04b0047f60 2024-04-24T23:35:18.932 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.909+0000 7ff432ffd640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff42c047490 0x7ff42c006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:18.932 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.909+0000 7ff432ffd640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff42c047490 0x7ff42c006e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.157:44880/0 (socket says 172.21.15.157:44880) 2024-04-24T23:35:18.932 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.909+0000 7ff432ffd640 1 -- 172.21.15.157:0/2526145494 learned_addr learned my addr 172.21.15.157:0/2526145494 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:18.933 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.909+0000 7f6f6cab5640 1 -- 172.21.15.157:0/1891382917 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f58037070 con 0x7f6f6806d720 2024-04-24T23:35:18.933 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.909+0000 7ff433fff640 1 --2- 172.21.15.157:0/2526145494 >> v2:172.21.15.157:3300/0 conn(0x7ff42c0073d0 0x7ff42c00a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:18.933 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.909+0000 7f7e13fff640 1 --2- 172.21.15.157:0/1508436491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e1c0c8510 0x7f7e1c153390 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c0097a0 tx=0x7f7e0c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd96d6df9d93aa1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:18.934 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.909+0000 7f6f6cab5640 1 -- 172.21.15.157:0/1891382917 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f5802ae00 con 0x7f6f6806d720 2024-04-24T23:35:18.934 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.909+0000 7f6f6cab5640 1 -- 172.21.15.157:0/1891382917 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f58032660 con 0x7f6f6806d720 2024-04-24T23:35:18.934 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.909+0000 7fe493fff640 1 --2- 172.21.15.157:0/450435316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe484004800 0x7fe484004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:35:18.934 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.909+0000 7fd46a173640 1 -- 172.21.15.157:0/1677767136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4640c60c0 msgr2=0x7fd46406ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:18.935 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.909+0000 7fa3127fc640 1 -- 172.21.15.157:0/1048643049 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31406d040 con 0x7fa30c04db10 2024-04-24T23:35:18.935 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.909+0000 7fd46a173640 1 --2- 172.21.15.157:0/1677767136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4640c60c0 0x7fd46406ddb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd4600097a0 tx=0x7fd46002a110 comp rx=0 tx=0).stop 2024-04-24T23:35:18.935 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.909+0000 7f04bcb2a640 1 -- 172.21.15.157:0/1972152028 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a8032850 con 0x7f04b0047f60 2024-04-24T23:35:18.935 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.909+0000 7fd46a173640 1 -- 172.21.15.157:0/1677767136 shutdown_connections 2024-04-24T23:35:18.936 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.909+0000 7fa3127fc640 1 -- 172.21.15.157:0/1048643049 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3140a9740 con 0x7fa30c04db10 2024-04-24T23:35:18.936 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.909+0000 7fd46a173640 1 --2- 172.21.15.157:0/1677767136 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4640c2b60 0x7fd46415d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.936 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.909+0000 7fd46a173640 1 --2- 172.21.15.157:0/1677767136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4640c60c0 0x7fd46406ddb0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.936 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.909+0000 7fd46a173640 1 -- 172.21.15.157:0/1677767136 >> 172.21.15.157:0/1677767136 conn(0x7fd4640c6800 msgr2=0x7fd4640c6c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:18.937 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.910+0000 7ff432ffd640 1 -- 172.21.15.157:0/2526145494 >> v2:172.21.15.157:3300/0 conn(0x7ff42c0073d0 msgr2=0x7ff42c00a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:18.937 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.910+0000 7ff432ffd640 1 --2- 172.21.15.157:0/2526145494 >> v2:172.21.15.157:3300/0 conn(0x7ff42c0073d0 0x7ff42c00a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.937 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.910+0000 7ff432ffd640 1 -- 172.21.15.157:0/2526145494 >> v1:172.21.15.157:6789/0 conn(0x7ff42c0dbcd0 legacy=0x7ff42c0df170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:18.937 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.910+0000 7ff432ffd640 1 -- 172.21.15.157:0/2526145494 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff42c00a0e0 con 0x7ff42c047490 2024-04-24T23:35:18.937 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.910+0000 7f7e1a7fc640 1 -- 172.21.15.157:0/1508436491 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c037070 con 0x7f7e1c0c8510 2024-04-24T23:35:18.937 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.910+0000 7f7e1a7fc640 1 -- 172.21.15.157:0/1508436491 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e0c032440 con 0x7f7e1c0c8510 2024-04-24T23:35:18.937 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.910+0000 7f7e1a7fc640 1 -- 172.21.15.157:0/1508436491 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c032810 con 0x7f7e1c0c8510 2024-04-24T23:35:18.938 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.910+0000 7f04bfd90640 1 -- 172.21.15.157:0/1972152028 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04b0047f60 msgr2=0x7f04b003f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:18.938 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.910+0000 7f04bfd90640 1 --2- 172.21.15.157:0/1972152028 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04b0047f60 0x7f04b003f240 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f04a8009870 tx=0x7f04a802a1a0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.938 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.910+0000 7fd46a173640 1 -- 172.21.15.157:0/1677767136 shutdown_connections 2024-04-24T23:35:18.938 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.910+0000 7f04bfd90640 1 -- 172.21.15.157:0/1972152028 shutdown_connections 2024-04-24T23:35:18.938 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.910+0000 7f04bfd90640 1 --2- 172.21.15.157:0/1972152028 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04b0049f50 0x7f04b004a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.939 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.910+0000 7f6f6fd1b640 1 -- 172.21.15.157:0/1891382917 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f6806d720 msgr2=0x7f6f68151ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:18.939 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.910+0000 7f6f6fd1b640 1 --2- 172.21.15.157:0/1891382917 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f6806d720 0x7f6f68151ce0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6f680c44a0 tx=0x7f6f5802a0b0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.939 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.910+0000 7fa31a25d640 1 -- 172.21.15.157:0/1048643049 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa30c04db10 msgr2=0x7fa30c04ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:18.940 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.910+0000 7f04bfd90640 1 --2- 172.21.15.157:0/1972152028 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04b0047f60 0x7f04b003f240 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.940 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.910+0000 7f04bfd90640 1 -- 172.21.15.157:0/1972152028 >> 172.21.15.157:0/1972152028 conn(0x7f04b002db80 msgr2=0x7f04b002df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:18.940 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.911+0000 7ff433fff640 1 --2- 172.21.15.157:0/2526145494 >> v2:172.21.15.157:3300/0 conn(0x7ff42c0073d0 0x7ff42c00a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:35:18.940 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.910+0000 7fe498bc5640 1 --2- 172.21.15.157:0/450435316 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe4840039b0 0x7fe484003dd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe48800c2c0 tx=0x7fe484005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c4b6c72264e0a67 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:18.940 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.910+0000 7fa31a25d640 1 --2- 172.21.15.157:0/1048643049 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa30c04db10 0x7fa30c04ef10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa31406a550 tx=0x7fa3140a5a90 comp rx=0 tx=0).stop 2024-04-24T23:35:18.941 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.911+0000 7fa31a25d640 1 -- 172.21.15.157:0/1048643049 shutdown_connections 2024-04-24T23:35:18.941 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.911+0000 7ff432ffd640 1 --2- 172.21.15.157:0/2526145494 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff42c047490 0x7ff42c006e90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff42c046bd0 tx=0x7ff42002a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13720ec17c859e08 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:18.941 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.911+0000 7f6f6fd1b640 1 -- 172.21.15.157:0/1891382917 shutdown_connections 2024-04-24T23:35:18.941 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.911+0000 7f6f6fd1b640 1 --2- 172.21.15.157:0/1891382917 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f68072ec0 0x7f6f68152220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.941 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.911+0000 7f6f6fd1b640 1 --2- 172.21.15.157:0/1891382917 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f6806d720 0x7f6f68151ce0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.941 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.911+0000 7f6f6fd1b640 1 --2- 172.21.15.157:0/1891382917 >> v2:172.21.15.157:3300/0 conn(0x7f6f6806d340 0x7f6f680c1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.942 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.911+0000 7f6f6fd1b640 1 -- 172.21.15.157:0/1891382917 >> 172.21.15.157:0/1891382917 conn(0x7f6f680a8690 msgr2=0x7f6f680a8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:18.942 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.911+0000 7fa31a25d640 1 --2- 172.21.15.157:0/1048643049 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa30c04db10 0x7fa30c04ef10 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.942 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.911+0000 7fa31a25d640 1 --2- 172.21.15.157:0/1048643049 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa30c0d7b10 0x7fa30c0071e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.942 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.911+0000 7f7e21a4d640 1 -- 172.21.15.157:0/1508436491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e1c0c8510 msgr2=0x7f7e1c153390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:18.942 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.911+0000 7f7e21a4d640 1 --2- 172.21.15.157:0/1508436491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e1c0c8510 0x7f7e1c153390 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c0097a0 tx=0x7f7e0c02a1e0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.943 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.911+0000 7f6f6fd1b640 1 -- 172.21.15.157:0/1891382917 shutdown_connections 2024-04-24T23:35:18.943 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.911+0000 7ff4327fc640 1 -- 172.21.15.157:0/2526145494 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff420037070 con 0x7ff42c047490 2024-04-24T23:35:18.943 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.911+0000 7f6f6fd1b640 1 -- 172.21.15.157:0/1891382917 wait complete. 2024-04-24T23:35:18.944 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.911+0000 7ff4327fc640 1 -- 172.21.15.157:0/2526145494 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff420032440 con 0x7ff42c047490 2024-04-24T23:35:18.944 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.911+0000 7ff4327fc640 1 -- 172.21.15.157:0/2526145494 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff420032850 con 0x7ff42c047490 2024-04-24T23:35:18.944 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.911+0000 7fa31a25d640 1 -- 172.21.15.157:0/1048643049 >> 172.21.15.157:0/1048643049 conn(0x7fa30c04c690 msgr2=0x7fa30c04ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:18.944 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.910+0000 7f04bfd90640 1 -- 172.21.15.157:0/1972152028 shutdown_connections 2024-04-24T23:35:18.945 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.910+0000 7f04bfd90640 1 -- 172.21.15.157:0/1972152028 wait complete. 2024-04-24T23:35:18.945 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.910+0000 7fd46a173640 1 -- 172.21.15.157:0/1677767136 wait complete. 2024-04-24T23:35:18.945 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.911+0000 7fa31a25d640 1 -- 172.21.15.157:0/1048643049 shutdown_connections 2024-04-24T23:35:18.946 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.911+0000 7fe4937fe640 1 -- 172.21.15.157:0/450435316 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe488007cf0 con 0x7fe4840039b0 2024-04-24T23:35:18.946 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.911+0000 7fe4937fe640 1 -- 172.21.15.157:0/450435316 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe488005af0 con 0x7fe4840039b0 2024-04-24T23:35:18.946 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.911+0000 7fe4937fe640 1 -- 172.21.15.157:0/450435316 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe488007070 con 0x7fe4840039b0 2024-04-24T23:35:18.946 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.912+0000 7fd46a173640 1 Processor -- start 2024-04-24T23:35:18.946 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.912+0000 7f04bfd90640 1 Processor -- start 2024-04-24T23:35:18.947 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.912+0000 7ff438b01640 1 -- 172.21.15.157:0/2526145494 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff42c047490 msgr2=0x7ff42c006e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:18.947 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.912+0000 7ff438b01640 1 --2- 172.21.15.157:0/2526145494 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff42c047490 0x7ff42c006e90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff42c046bd0 tx=0x7ff42002a210 comp rx=0 tx=0).stop 2024-04-24T23:35:18.947 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.912+0000 7ff438b01640 1 -- 172.21.15.157:0/2526145494 shutdown_connections 2024-04-24T23:35:18.947 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.912+0000 7ff438b01640 1 --2- 172.21.15.157:0/2526145494 >> v2:172.21.15.157:3300/0 conn(0x7ff42c0073d0 0x7ff42c00a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.947 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.912+0000 7ff438b01640 1 --2- 172.21.15.157:0/2526145494 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff42c047490 0x7ff42c006e90 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.947 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.912+0000 7ff438b01640 1 -- 172.21.15.157:0/2526145494 >> 172.21.15.157:0/2526145494 conn(0x7ff42c048240 msgr2=0x7ff42c048650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:18.947 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.912+0000 7f7e21a4d640 1 -- 172.21.15.157:0/1508436491 shutdown_connections 2024-04-24T23:35:18.947 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.912+0000 7f7e21a4d640 1 --2- 172.21.15.157:0/1508436491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e1c0c8510 0x7f7e1c153390 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.948 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.912+0000 7f7e21a4d640 1 -- 172.21.15.157:0/1508436491 >> 172.21.15.157:0/1508436491 conn(0x7f7e1c0aa7b0 msgr2=0x7f7e1c0aab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:18.949 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.912+0000 7ff438b01640 1 -- 172.21.15.157:0/2526145494 shutdown_connections 2024-04-24T23:35:18.949 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.912+0000 7f04bfd90640 1 -- start start 2024-04-24T23:35:18.950 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.912+0000 7fe49ae29640 1 -- 172.21.15.157:0/450435316 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe4840039b0 msgr2=0x7fe484003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:18.950 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.912+0000 7fe49ae29640 1 --2- 172.21.15.157:0/450435316 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe4840039b0 0x7fe484003dd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe48800c2c0 tx=0x7fe484005830 comp rx=0 tx=0).stop 2024-04-24T23:35:18.950 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.912+0000 7fe49ae29640 1 -- 172.21.15.157:0/450435316 shutdown_connections 2024-04-24T23:35:18.950 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.912+0000 7fe49ae29640 1 --2- 172.21.15.157:0/450435316 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe4840039b0 0x7fe484003dd0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.950 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.912+0000 7fe49ae29640 1 -- 172.21.15.157:0/450435316 >> 172.21.15.157:0/450435316 conn(0x7fe4940a8690 msgr2=0x7fe4940a8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:18.950 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.913+0000 7f8db759e640 1 Processor -- start 2024-04-24T23:35:18.951 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.913+0000 7f7e21a4d640 1 -- 172.21.15.157:0/1508436491 shutdown_connections 2024-04-24T23:35:18.951 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.912+0000 7fd46a173640 1 -- start start 2024-04-24T23:35:18.951 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.913+0000 7f8db759e640 1 -- start start 2024-04-24T23:35:18.952 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.913+0000 7f6f6fd1b640 1 Processor -- start 2024-04-24T23:35:18.952 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.913+0000 7f04bfd90640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04b000be70 0x7f04b0042260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.952 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.913+0000 7f04bfd90640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04b0047f60 0x7f04b01748c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.952 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.913+0000 7f04bfd90640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04b0049f50 0x7f04b0174e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.952 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.913+0000 7f04bfd90640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f04b00df7a0 con 0x7f04b0049f50 2024-04-24T23:35:18.952 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.913+0000 7f04bfd90640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f04b00df280 con 0x7f04b0047f60 2024-04-24T23:35:18.952 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.913+0000 7f04bfd90640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f04b004a920 con 0x7f04b000be70 2024-04-24T23:35:18.953 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.913+0000 7f6f6fd1b640 1 -- start start 2024-04-24T23:35:18.953 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.913+0000 7fd46a173640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4640c2720 0x7fd4641eb620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.953 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.913+0000 7fd46a173640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4640c2b60 0x7fd4640aae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.953 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.913+0000 7fd46a173640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4640c60c0 0x7fd4640ab3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.953 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.913+0000 7fd469171640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4640c2720 0x7fd4641eb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:18.953 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.913+0000 7fd46a173640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd464159d90 con 0x7fd4640c2b60 2024-04-24T23:35:18.954 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.913+0000 7f04bdb2c640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04b000be70 0x7f04b0042260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:18.954 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.913+0000 7f04bdb2c640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04b000be70 0x7f04b0042260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.157:52690/0 (socket says 172.21.15.157:52690) 2024-04-24T23:35:18.954 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.913+0000 7f04bdb2c640 1 -- 172.21.15.157:0/526758282 learned_addr learned my addr 172.21.15.157:0/526758282 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:18.954 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.913+0000 7f04be32d640 1 --2- 172.21.15.157:0/526758282 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04b0049f50 0x7f04b0174e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:18.954 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.911+0000 7fa31a25d640 1 -- 172.21.15.157:0/1048643049 wait complete. 2024-04-24T23:35:18.955 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.913+0000 7fd468970640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4640c2b60 0x7fd4640aae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:18.955 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.913+0000 7fd468970640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4640c2b60 0x7fd4640aae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42638/0 (socket says 172.21.15.157:42638) 2024-04-24T23:35:18.955 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.913+0000 7fd468970640 1 -- 172.21.15.157:0/2311295592 learned_addr learned my addr 172.21.15.157:0/2311295592 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:18.955 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.913+0000 7f04bd32b640 1 --2- 172.21.15.157:0/526758282 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04b0047f60 0x7f04b01748c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:18.955 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.913+0000 7fa31a25d640 1 Processor -- start 2024-04-24T23:35:18.956 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.913+0000 7fd46a173640 1 -- 172.21.15.157:0/2311295592 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd4640c2ff0 con 0x7fd4640c2720 2024-04-24T23:35:18.956 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.913+0000 7fd46a173640 1 -- 172.21.15.157:0/2311295592 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd46406d930 con 0x7fd4640c60c0 2024-04-24T23:35:18.956 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.913+0000 7fe49ae29640 1 -- 172.21.15.157:0/450435316 shutdown_connections 2024-04-24T23:35:18.956 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.914+0000 7fd469972640 1 --2- 172.21.15.157:0/2311295592 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4640c60c0 0x7fd4640ab3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:18.957 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.913+0000 7f04bdb2c640 1 -- 172.21.15.157:0/526758282 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04b0047f60 msgr2=0x7f04b01748c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:18.957 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.913+0000 7f04bdb2c640 1 --2- 172.21.15.157:0/526758282 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04b0047f60 0x7f04b01748c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.957 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.913+0000 7f04bdb2c640 1 -- 172.21.15.157:0/526758282 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04b0049f50 msgr2=0x7f04b0174e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:18.957 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.913+0000 7f04bdb2c640 1 --2- 172.21.15.157:0/526758282 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04b0049f50 0x7f04b0174e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.957 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.914+0000 7f04bdb2c640 1 -- 172.21.15.157:0/526758282 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04b0030db0 con 0x7f04b000be70 2024-04-24T23:35:18.957 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.912+0000 7ff438b01640 1 -- 172.21.15.157:0/2526145494 wait complete. 2024-04-24T23:35:18.958 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.914+0000 7ff438b01640 1 Processor -- start 2024-04-24T23:35:18.958 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.914+0000 7f04bdb2c640 1 --2- 172.21.15.157:0/526758282 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04b000be70 0x7f04b0042260 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f04b400b3f0 tx=0x7f04b400b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:18.958 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.914+0000 7f04ae7fc640 1 -- 172.21.15.157:0/526758282 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b4015020 con 0x7f04b000be70 2024-04-24T23:35:18.958 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.914+0000 7fd469972640 1 -- 172.21.15.157:0/2311295592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4640c2720 msgr2=0x7fd4641eb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:18.958 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.914+0000 7fd469972640 1 --2- 172.21.15.157:0/2311295592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4640c2720 0x7fd4641eb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.959 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.914+0000 7fd469972640 1 -- 172.21.15.157:0/2311295592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4640c2b60 msgr2=0x7fd4640aae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:18.959 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.914+0000 7fd469972640 1 --2- 172.21.15.157:0/2311295592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4640c2b60 0x7fd4640aae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.959 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.914+0000 7fd469972640 1 -- 172.21.15.157:0/2311295592 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd46415aa50 con 0x7fd4640c60c0 2024-04-24T23:35:18.959 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.914+0000 7f04ae7fc640 1 -- 172.21.15.157:0/526758282 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04b4011830 con 0x7f04b000be70 2024-04-24T23:35:18.959 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.914+0000 7f04ae7fc640 1 -- 172.21.15.157:0/526758282 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b4011c20 con 0x7f04b000be70 2024-04-24T23:35:18.959 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.914+0000 7fd468970640 1 --2- 172.21.15.157:0/2311295592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4640c2b60 0x7fd4640aae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:35:18.960 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.914+0000 7f8db759e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8db00d58d0 0x7f8db004baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.960 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.914+0000 7f8db759e640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8db004c030 0x7f8db004cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.960 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.914+0000 7f8db759e640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8db004e4c0 0x7f8db004d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.960 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.914+0000 7f8db759e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f8db00d87c0 con 0x7f8db00d58d0 2024-04-24T23:35:18.960 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.914+0000 7f8db759e640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f8db00d8aa0 con 0x7f8db004c030 2024-04-24T23:35:18.960 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.914+0000 7f8db759e640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8db00d8d80 con 0x7f8db004e4c0 2024-04-24T23:35:18.961 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.914+0000 7fd469972640 1 --2- 172.21.15.157:0/2311295592 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4640c60c0 0x7fd4640ab3b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd45c009d90 tx=0x7fd45c00c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:18.961 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.915+0000 7fd459ffb640 1 -- 172.21.15.157:0/2311295592 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45c018020 con 0x7fd4640c60c0 2024-04-24T23:35:18.961 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.915+0000 7fe49ae29640 1 -- 172.21.15.157:0/450435316 wait complete. 2024-04-24T23:35:18.961 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.915+0000 7fd459ffb640 1 -- 172.21.15.157:0/2311295592 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd45c014480 con 0x7fd4640c60c0 2024-04-24T23:35:18.961 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.915+0000 7fd459ffb640 1 -- 172.21.15.157:0/2311295592 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45c014870 con 0x7fd4640c60c0 2024-04-24T23:35:18.961 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.915+0000 7fd46a173640 1 -- 172.21.15.157:0/2311295592 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4640ab8f0 con 0x7fd4640c60c0 2024-04-24T23:35:18.962 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.915+0000 7f6f6fd1b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f6806d340 0x7f6f6814d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.962 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.915+0000 7f8db5d9b640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8db004c030 0x7f8db004cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:18.962 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.915+0000 7fd46a173640 1 -- 172.21.15.157:0/2311295592 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd46415acf0 con 0x7fd4640c60c0 2024-04-24T23:35:18.963 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.914+0000 7f04bfd90640 1 -- 172.21.15.157:0/526758282 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04b0031090 con 0x7f04b000be70 2024-04-24T23:35:18.963 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.914+0000 7f04bfd90640 1 -- 172.21.15.157:0/526758282 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f04b0031650 con 0x7f04b000be70 2024-04-24T23:35:18.963 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.915+0000 7f6f6fd1b640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f6806d720 0x7f6f68144bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.963 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.915+0000 7f8db5d9b640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8db004c030 0x7f8db004cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.157:44908/0 (socket says 172.21.15.157:44908) 2024-04-24T23:35:18.963 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.915+0000 7f8db5d9b640 1 -- 172.21.15.157:0/3727495400 learned_addr learned my addr 172.21.15.157:0/3727495400 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:18.964 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.914+0000 7f7e21a4d640 1 -- 172.21.15.157:0/1508436491 wait complete. 2024-04-24T23:35:18.964 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.915+0000 7f7e21a4d640 1 Processor -- start 2024-04-24T23:35:18.964 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.915+0000 7f6f6fd1b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f68072ec0 0x7f6f68145100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.964 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.915+0000 7f7e21a4d640 1 -- start start 2024-04-24T23:35:18.965 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.915+0000 7f8db659c640 1 --2- 172.21.15.157:0/3727495400 >> v2:172.21.15.157:3300/0 conn(0x7f8db00d58d0 0x7f8db004baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:18.965 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.915+0000 7f6f6fd1b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6806ebc0 con 0x7f6f68072ec0 2024-04-24T23:35:18.965 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.915+0000 7f8db6d9d640 1 --2- 172.21.15.157:0/3727495400 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8db004e4c0 0x7f8db004d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:18.966 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.913+0000 7fa31a25d640 1 -- start start 2024-04-24T23:35:18.966 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.915+0000 7fa31a25d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa30c04e020 0x7fa30c02f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.966 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.915+0000 7fa31a25d640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa30c0d7b10 0x7fa30c180930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.966 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.915+0000 7fa31a25d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa30c0d9be0 0x7fa30c16f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.966 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.914+0000 7ff438b01640 1 -- start start 2024-04-24T23:35:18.967 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.915+0000 7fa31a25d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa30c04e4e0 con 0x7fa30c04e020 2024-04-24T23:35:18.967 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.915+0000 7fa31a25d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa30c0db2a0 con 0x7fa30c0d9be0 2024-04-24T23:35:18.967 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.915+0000 7fa31a25d640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa30c00fdf0 con 0x7fa30c0d7b10 2024-04-24T23:35:18.967 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.915+0000 7ff438b01640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff42c0073d0 0x7ff42c03fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.967 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.915+0000 7ff438b01640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff42c047490 0x7ff42c02c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.968 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.916+0000 7f6f6e2b8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f68072ec0 0x7f6f68145100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:18.968 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.916+0000 7f6f6e2b8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f68072ec0 0x7f6f68145100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42672/0 (socket says 172.21.15.157:42672) 2024-04-24T23:35:18.968 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.916+0000 7f049bfff640 1 -- 172.21.15.157:0/526758282 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f04b0174490 con 0x7f04b000be70 2024-04-24T23:35:18.968 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.916+0000 7ff438b01640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff42c0dbcd0 0x7ff42c0dac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.968 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.916+0000 7f8db5d9b640 1 -- 172.21.15.157:0/3727495400 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8db004e4c0 msgr2=0x7f8db004d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:18.969 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.916+0000 7f8db5d9b640 1 --2- 172.21.15.157:0/3727495400 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8db004e4c0 0x7f8db004d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.969 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.916+0000 7ff4337fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff42c0073d0 0x7ff42c03fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:18.969 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.916+0000 7f8db5d9b640 1 -- 172.21.15.157:0/3727495400 >> v2:172.21.15.157:3300/0 conn(0x7f8db00d58d0 msgr2=0x7f8db004baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:18.969 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.916+0000 7f6f6e2b8640 1 -- 172.21.15.157:0/1751256345 learned_addr learned my addr 172.21.15.157:0/1751256345 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:18.970 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.916+0000 7f8db5d9b640 1 --2- 172.21.15.157:0/3727495400 >> v2:172.21.15.157:3300/0 conn(0x7f8db00d58d0 0x7f8db004baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.970 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.916+0000 7f8db5d9b640 1 -- 172.21.15.157:0/3727495400 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8db00d9060 con 0x7f8db004c030 2024-04-24T23:35:18.970 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.916+0000 7ff4337fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff42c0073d0 0x7ff42c03fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42686/0 (socket says 172.21.15.157:42686) 2024-04-24T23:35:18.970 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.916+0000 7ff432ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff42c047490 0x7ff42c02c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:18.970 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.916+0000 7ff432ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff42c047490 0x7ff42c02c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.157:44930/0 (socket says 172.21.15.157:44930) 2024-04-24T23:35:18.971 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.916+0000 7f8db659c640 1 --2- 172.21.15.157:0/3727495400 >> v2:172.21.15.157:3300/0 conn(0x7f8db00d58d0 0x7f8db004baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:35:18.971 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.916+0000 7ff438b01640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff42c0dc0b0 con 0x7ff42c0073d0 2024-04-24T23:35:18.971 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.916+0000 7ff438b01640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff42c047d70 con 0x7ff42c047490 2024-04-24T23:35:18.971 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.916+0000 7ff438b01640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff42c0dfbe0 con 0x7ff42c0dbcd0 2024-04-24T23:35:18.971 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.916+0000 7f6f6fd1b640 1 -- 172.21.15.157:0/1751256345 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6f680a6350 con 0x7f6f6806d340 2024-04-24T23:35:18.971 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.916+0000 7ff4337fe640 1 -- 172.21.15.157:0/2465463571 learned_addr learned my addr 172.21.15.157:0/2465463571 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:18.972 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.916+0000 7f6f6fd1b640 1 -- 172.21.15.157:0/1751256345 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6f6806ed40 con 0x7f6f6806d720 2024-04-24T23:35:18.972 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.917+0000 7fa3137fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa30c04e020 0x7fa30c02f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:18.972 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.916+0000 7ff432ffd640 1 -- 172.21.15.157:0/2465463571 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff42c0dbcd0 msgr2=0x7ff42c0dac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:18.972 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.917+0000 7ff432ffd640 1 --2- 172.21.15.157:0/2465463571 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff42c0dbcd0 0x7ff42c0dac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.972 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.917+0000 7ff432ffd640 1 -- 172.21.15.157:0/2465463571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff42c0073d0 msgr2=0x7ff42c03fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:18.973 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.917+0000 7ff432ffd640 1 --2- 172.21.15.157:0/2465463571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff42c0073d0 0x7ff42c03fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.973 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.917+0000 7ff432ffd640 1 -- 172.21.15.157:0/2465463571 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff42c03ffd0 con 0x7ff42c047490 2024-04-24T23:35:18.973 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.917+0000 7f7e21a4d640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e1c0c4740 0x7f7e1c1e97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.973 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.917+0000 7f7e21a4d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e1c0c4b20 0x7f7e1c1ecbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.973 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.917+0000 7f7e21a4d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e1c0c8510 0x7f7e1c1530c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.973 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.917+0000 7f6f6e2b8640 1 -- 172.21.15.157:0/1751256345 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f6806d720 msgr2=0x7f6f68144bc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-24T23:35:18.974 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.917+0000 7ff4337fe640 1 --2- 172.21.15.157:0/2465463571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff42c0073d0 0x7ff42c03fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:35:18.974 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.917+0000 7f7e21a4d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7e1c0ad700 con 0x7f7e1c0c4b20 2024-04-24T23:35:18.974 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.917+0000 7fa3137fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa30c04e020 0x7fa30c02f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42662/0 (socket says 172.21.15.157:42662) 2024-04-24T23:35:18.974 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.917+0000 7fa3137fe640 1 -- 172.21.15.157:0/1974679134 learned_addr learned my addr 172.21.15.157:0/1974679134 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:18.975 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.917+0000 7f7e21a4d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7e1c0ada70 con 0x7f7e1c0c8510 2024-04-24T23:35:18.975 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.917+0000 7f7e21a4d640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7e1c0acaf0 con 0x7f7e1c0c4740 2024-04-24T23:35:18.975 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.917+0000 7f04ae7fc640 1 -- 172.21.15.157:0/526758282 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f04b4018020 con 0x7f04b000be70 2024-04-24T23:35:18.975 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.917+0000 7fa312ffd640 1 --2- 172.21.15.157:0/1974679134 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa30c0d7b10 0x7fa30c180930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:18.976 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.917+0000 7f7e1b7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e1c0c8510 0x7f7e1c1530c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:18.976 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.917+0000 7f6f6e2b8640 1 --2- 172.21.15.157:0/1751256345 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f6806d720 0x7f6f68144bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.976 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.917+0000 7f6f6e2b8640 1 -- 172.21.15.157:0/1751256345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f6806d340 msgr2=0x7f6f6814d6e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-24T23:35:18.976 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.917+0000 7f6f6e2b8640 1 --2- 172.21.15.157:0/1751256345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f6806d340 0x7f6f6814d6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.977 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.917+0000 7ff432ffd640 1 --2- 172.21.15.157:0/2465463571 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff42c047490 0x7ff42c02c980 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff420031000 tx=0x7ff4200046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:18.977 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.917+0000 7fd46a173640 1 -- 172.21.15.157:0/2311295592 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd444001030 con 0x7fd4640c60c0 2024-04-24T23:35:18.977 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.917+0000 7f6f6e2b8640 1 -- 172.21.15.157:0/1751256345 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f680acb90 con 0x7f6f68072ec0 2024-04-24T23:35:18.977 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.917+0000 7f7e1b7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e1c0c8510 0x7f7e1c1530c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.157:44958/0 (socket says 172.21.15.157:44958) 2024-04-24T23:35:18.978 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.917+0000 7f7e1b7fe640 1 -- 172.21.15.157:0/556559919 learned_addr learned my addr 172.21.15.157:0/556559919 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:18.978 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.917+0000 7fa3137fe640 1 -- 172.21.15.157:0/1974679134 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa30c0d7b10 msgr2=0x7fa30c180930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:18.978 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.917+0000 7fa3137fe640 1 --2- 172.21.15.157:0/1974679134 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa30c0d7b10 0x7fa30c180930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.978 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.917+0000 7fa3137fe640 1 -- 172.21.15.157:0/1974679134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa30c0d9be0 msgr2=0x7fa30c16f430 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-24T23:35:18.978 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.917+0000 7fa3137fe640 1 --2- 172.21.15.157:0/1974679134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa30c0d9be0 0x7fa30c16f430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.978 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.917+0000 7fa3137fe640 1 -- 172.21.15.157:0/1974679134 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa30c006e10 con 0x7fa30c04e020 2024-04-24T23:35:18.978 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.917+0000 7ff41ffff640 1 -- 172.21.15.157:0/2465463571 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff420041020 con 0x7ff42c047490 2024-04-24T23:35:18.979 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.917+0000 7fd459ffb640 1 -- 172.21.15.157:0/2311295592 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd45c005d10 con 0x7fd4640c60c0 2024-04-24T23:35:18.979 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.917+0000 7f04ae7fc640 1 --2- 172.21.15.157:0/526758282 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f049c06bf00 0x7f049c06e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.979 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.917+0000 7ff438b01640 1 -- 172.21.15.157:0/2465463571 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff42c03f4d0 con 0x7ff42c047490 2024-04-24T23:35:18.979 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.918+0000 7f6f6e2b8640 1 --2- 172.21.15.157:0/1751256345 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f68072ec0 0x7f6f68145100 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6f60007e60 tx=0x7f6f6000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:18.980 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.918+0000 7f04ae7fc640 1 -- 172.21.15.157:0/526758282 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f04b408af30 con 0x7f04b000be70 2024-04-24T23:35:18.980 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.917+0000 7f8db5d9b640 1 --2- 172.21.15.157:0/3727495400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8db004c030 0x7f8db004cc60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8db000bb90 tx=0x7f8dac02a1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=800e5bbadd19424e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:18.980 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.918+0000 7ff438b01640 1 -- 172.21.15.157:0/2465463571 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff42c030f00 con 0x7ff42c047490 2024-04-24T23:35:18.981 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.918+0000 7fa3137fe640 1 --2- 172.21.15.157:0/1974679134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa30c04e020 0x7fa30c02f590 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa30800e870 tx=0x7fa30800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:18.981 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.918+0000 7f04ae7fc640 1 --2- 172.21.15.157:0/526758282 >> [v2:172.21.15.169:6824/243321711,v1:172.21.15.169:6825/243321711] conn(0x7f049c075600 0x7f049c077a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.981 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.918+0000 7f04ae7fc640 1 -- 172.21.15.157:0/526758282 --> [v2:172.21.15.169:6824/243321711,v1:172.21.15.169:6825/243321711] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f049c078110 con 0x7f049c075600 2024-04-24T23:35:18.981 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.918+0000 7f04ae7fc640 1 -- 172.21.15.157:0/526758282 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f04b4055260 con 0x7f04b000be70 2024-04-24T23:35:18.981 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.918+0000 7fe49ae29640 1 Processor -- start 2024-04-24T23:35:18.982 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.918+0000 7f7e1b7fe640 1 -- 172.21.15.157:0/556559919 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e1c0c4740 msgr2=0x7f7e1c1e97f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:35:18.982 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.918+0000 7f7e1b7fe640 1 --2- 172.21.15.157:0/556559919 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e1c0c4740 0x7f7e1c1e97f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.982 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.918+0000 7f7e1b7fe640 1 -- 172.21.15.157:0/556559919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e1c0c4b20 msgr2=0x7f7e1c1ecbc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:35:18.982 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.918+0000 7f7e1b7fe640 1 --2- 172.21.15.157:0/556559919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e1c0c4b20 0x7f7e1c1ecbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.982 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.918+0000 7f7e1b7fe640 1 -- 172.21.15.157:0/556559919 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e1c156c30 con 0x7f7e1c0c8510 2024-04-24T23:35:18.983 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.918+0000 7f04be32d640 1 --2- 172.21.15.157:0/526758282 >> [v2:172.21.15.169:6824/243321711,v1:172.21.15.169:6825/243321711] conn(0x7f049c075600 0x7f049c077a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:18.983 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.918+0000 7fa2fffff640 1 -- 172.21.15.157:0/1974679134 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa308017070 con 0x7fa30c04e020 2024-04-24T23:35:18.983 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.918+0000 7fa2fffff640 1 -- 172.21.15.157:0/1974679134 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa30800ee60 con 0x7fa30c04e020 2024-04-24T23:35:18.983 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.918+0000 7fa2fffff640 1 -- 172.21.15.157:0/1974679134 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30800cb80 con 0x7fa30c04e020 2024-04-24T23:35:18.983 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.918+0000 7fa31a25d640 1 -- 172.21.15.157:0/1974679134 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa30c0070f0 con 0x7fa30c04e020 2024-04-24T23:35:18.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.918+0000 7f7e1b7fe640 1 --2- 172.21.15.157:0/556559919 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e1c0c8510 0x7f7e1c1530c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7e1400b990 tx=0x7f7e1400be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:18.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.918+0000 7fd459ffb640 1 --2- 172.21.15.157:0/2311295592 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fd44806beb0 0x7fd44806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.918+0000 7fe49ae29640 1 -- start start 2024-04-24T23:35:18.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.918+0000 7f6f5e7fc640 1 -- 172.21.15.157:0/1751256345 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f60014620 con 0x7f6f68072ec0 2024-04-24T23:35:18.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.918+0000 7f6f5e7fc640 1 -- 172.21.15.157:0/1751256345 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f60019070 con 0x7f6f68072ec0 2024-04-24T23:35:18.985 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.918+0000 7fa31a25d640 1 -- 172.21.15.157:0/1974679134 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa30c16f970 con 0x7fa30c04e020 2024-04-24T23:35:18.985 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.918+0000 7fd459ffb640 1 -- 172.21.15.157:0/2311295592 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd45c08bcf0 con 0x7fd4640c60c0 2024-04-24T23:35:18.985 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.918+0000 7ff41ffff640 1 -- 172.21.15.157:0/2465463571 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff420030970 con 0x7ff42c047490 2024-04-24T23:35:18.985 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.919+0000 7ff41ffff640 1 -- 172.21.15.157:0/2465463571 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff420037040 con 0x7ff42c047490 2024-04-24T23:35:18.986 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.918+0000 7fd459ffb640 1 --2- 172.21.15.157:0/2311295592 >> [v2:172.21.15.157:6824/2793948092,v1:172.21.15.157:6825/2793948092] conn(0x7fd4480755a0 0x7fd4480779e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.986 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.919+0000 7ff438b01640 1 -- 172.21.15.157:0/2465463571 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff42c046ce0 con 0x7ff42c047490 2024-04-24T23:35:18.986 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.918+0000 7fd459ffb640 1 -- 172.21.15.157:0/2311295592 --> [v2:172.21.15.157:6824/2793948092,v1:172.21.15.157:6825/2793948092] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd45c0020b0 con 0x7fd4480755a0 2024-04-24T23:35:18.987 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.919+0000 7f7e18ff9640 1 -- 172.21.15.157:0/556559919 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e14019070 con 0x7f7e1c0c8510 2024-04-24T23:35:18.987 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.918+0000 7fd469171640 1 --2- 172.21.15.157:0/2311295592 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fd44806beb0 0x7fd44806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:18.987 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.919+0000 7f7e18ff9640 1 -- 172.21.15.157:0/556559919 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e1400cba0 con 0x7f7e1c0c8510 2024-04-24T23:35:18.987 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.919+0000 7fd459ffb640 1 -- 172.21.15.157:0/2311295592 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd45c056070 con 0x7fd4640c60c0 2024-04-24T23:35:18.988 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.919+0000 7f7e21a4d640 1 -- 172.21.15.157:0/556559919 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e1c1495d0 con 0x7f7e1c0c8510 2024-04-24T23:35:18.988 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.919+0000 7f7e18ff9640 1 -- 172.21.15.157:0/556559919 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e14014440 con 0x7f7e1c0c8510 2024-04-24T23:35:18.988 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.919+0000 7fe49ae29640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4840039b0 0x7fe49414da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.988 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.919+0000 7f7e21a4d640 1 -- 172.21.15.157:0/556559919 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e1c1492f0 con 0x7f7e1c0c8510 2024-04-24T23:35:18.989 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.919+0000 7fe49ae29640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4941537d0 0x7fe494155820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.989 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.919+0000 7f6f5e7fc640 1 -- 172.21.15.157:0/1751256345 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f60014a00 con 0x7f6f68072ec0 2024-04-24T23:35:18.989 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.919+0000 7f04bd32b640 1 --2- 172.21.15.157:0/526758282 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f049c06bf00 0x7f049c06e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:18.989 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.919+0000 7fe498bc5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4840039b0 0x7fe49414da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:18.989 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.919+0000 7fe498bc5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4840039b0 0x7fe49414da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.157:44960/0 (socket says 172.21.15.157:44960) 2024-04-24T23:35:18.990 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.919+0000 7f6f6fd1b640 1 -- 172.21.15.157:0/1751256345 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f681ea010 con 0x7f6f68072ec0 2024-04-24T23:35:18.990 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.919+0000 7fe498bc5640 1 -- 172.21.15.157:0/3260198839 learned_addr learned my addr 172.21.15.157:0/3260198839 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:18.990 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.920+0000 7f6f6fd1b640 1 -- 172.21.15.157:0/1751256345 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f681ea5d0 con 0x7f6f68072ec0 2024-04-24T23:35:18.991 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.920+0000 7f04be32d640 1 --2- 172.21.15.157:0/526758282 >> [v2:172.21.15.169:6824/243321711,v1:172.21.15.169:6825/243321711] conn(0x7f049c075600 0x7f049c077a40 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:18.991 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.920+0000 7fa31a25d640 1 -- 172.21.15.157:0/1974679134 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa2f4001030 con 0x7fa30c04e020 2024-04-24T23:35:18.991 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.920+0000 7fd468970640 1 --2- 172.21.15.157:0/2311295592 >> [v2:172.21.15.157:6824/2793948092,v1:172.21.15.157:6825/2793948092] conn(0x7fd4480755a0 0x7fd4480779e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:18.992 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.920+0000 7fe49ae29640 1 --2- 172.21.15.157:0/3260198839 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe49414f750 0x7fe4941e7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.992 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.920+0000 7fe49ae29640 1 -- 172.21.15.157:0/3260198839 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe49406eca0 con 0x7fe4941537d0 2024-04-24T23:35:18.992 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.920+0000 7fe49ae29640 1 -- 172.21.15.157:0/3260198839 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe4940c1810 con 0x7fe4840039b0 2024-04-24T23:35:18.992 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.920+0000 7fe49ae29640 1 -- 172.21.15.157:0/3260198839 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe4940c47e0 con 0x7fe49414f750 2024-04-24T23:35:18.992 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.920+0000 7fe4993c6640 1 --2- 172.21.15.157:0/3260198839 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe49414f750 0x7fe4941e7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:18.992 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.920+0000 7fe493fff640 1 --2- 172.21.15.157:0/3260198839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4941537d0 0x7fe494155820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:18.992 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.920+0000 7f8db559a640 1 -- 172.21.15.157:0/3727495400 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dac037070 con 0x7f8db004c030 2024-04-24T23:35:18.993 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.921+0000 7f7e21a4d640 1 -- 172.21.15.157:0/556559919 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7e1c0681c0 con 0x7f7e1c0c8510 2024-04-24T23:35:18.993 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.920+0000 7f8db559a640 1 -- 172.21.15.157:0/3727495400 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dac032440 con 0x7f8db004c030 2024-04-24T23:35:18.993 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.920+0000 7f8db559a640 1 -- 172.21.15.157:0/3727495400 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dac032810 con 0x7f8db004c030 2024-04-24T23:35:18.993 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.920+0000 7fd468970640 1 --2- 172.21.15.157:0/2311295592 >> [v2:172.21.15.157:6824/2793948092,v1:172.21.15.157:6825/2793948092] conn(0x7fd4480755a0 0x7fd4480779e0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:18.993 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.920+0000 7f04bd32b640 1 --2- 172.21.15.157:0/526758282 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f049c06bf00 0x7f049c06e3c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f04a8034000 tx=0x7f04a802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:18.994 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.921+0000 7ff41ffff640 1 -- 172.21.15.157:0/2465463571 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff420044020 con 0x7ff42c047490 2024-04-24T23:35:18.994 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.921+0000 7fd469171640 1 --2- 172.21.15.157:0/2311295592 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fd44806beb0 0x7fd44806e370 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd460034000 tx=0x7fd46002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:18.994 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.921+0000 7fe4993c6640 1 -- 172.21.15.157:0/3260198839 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4840039b0 msgr2=0x7fe49414da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:18.995 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.921+0000 7fe4993c6640 1 --2- 172.21.15.157:0/3260198839 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4840039b0 0x7fe49414da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.995 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.921+0000 7fe4993c6640 1 -- 172.21.15.157:0/3260198839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4941537d0 msgr2=0x7fe494155820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:18.995 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.921+0000 7fe4993c6640 1 --2- 172.21.15.157:0/3260198839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4941537d0 0x7fe494155820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.995 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.921+0000 7fe4993c6640 1 -- 172.21.15.157:0/3260198839 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4941511e0 con 0x7fe49414f750 2024-04-24T23:35:18.995 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.921+0000 7fa2fffff640 1 -- 172.21.15.157:0/1974679134 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa30801a070 con 0x7fa30c04e020 2024-04-24T23:35:18.995 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.921+0000 7f6f6fd1b640 1 -- 172.21.15.157:0/1751256345 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6f680681c0 con 0x7f6f68072ec0 2024-04-24T23:35:18.996 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.921+0000 7fa2fffff640 1 --2- 172.21.15.157:0/1974679134 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fa2f806bf00 0x7fa2f806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.996 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.921+0000 7fa2fffff640 1 -- 172.21.15.157:0/1974679134 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa30808b280 con 0x7fa30c04e020 2024-04-24T23:35:18.996 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.922+0000 7fa2fffff640 1 --2- 172.21.15.157:0/1974679134 >> [v2:172.21.15.157:6808/4003822160,v1:172.21.15.157:6809/4003822160] conn(0x7fa2f80755f0 0x7fa2f8077a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.996 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.922+0000 7fa2fffff640 1 -- 172.21.15.157:0/1974679134 --> [v2:172.21.15.157:6808/4003822160,v1:172.21.15.157:6809/4003822160] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa308016ec0 con 0x7fa2f80755f0 2024-04-24T23:35:18.996 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.921+0000 7ff41ffff640 1 --2- 172.21.15.157:0/2465463571 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7ff41806bf00 0x7ff41806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.996 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.921+0000 7ff41ffff640 1 -- 172.21.15.157:0/2465463571 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ff4200abff0 con 0x7ff42c047490 2024-04-24T23:35:18.996 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.922+0000 7ff41ffff640 1 --2- 172.21.15.157:0/2465463571 >> [v2:172.21.15.157:6816/1557575576,v1:172.21.15.157:6817/1557575576] conn(0x7ff418075680 0x7ff418077ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:18.996 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.922+0000 7ff41ffff640 1 -- 172.21.15.157:0/2465463571 --> [v2:172.21.15.157:6816/1557575576,v1:172.21.15.157:6817/1557575576] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff420031ec0 con 0x7ff418075680 2024-04-24T23:35:18.996 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.922+0000 7ff41ffff640 1 -- 172.21.15.157:0/2465463571 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff4200ac3e0 con 0x7ff42c047490 2024-04-24T23:35:18.997 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.921+0000 7fd459ffb640 1 -- 172.21.15.157:0/2311295592 <== osd.2 v2:172.21.15.157:6824/2793948092 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd45c0020b0 con 0x7fd4480755a0 2024-04-24T23:35:18.997 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.922+0000 7f7e18ff9640 1 -- 172.21.15.157:0/556559919 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7e1401c070 con 0x7f7e1c0c8510 2024-04-24T23:35:18.997 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.921+0000 7fe4993c6640 1 --2- 172.21.15.157:0/3260198839 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe49414f750 0x7fe4941e7920 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe494151020 tx=0x7fe48c00fc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:18.997 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.921+0000 7fe4917fa640 1 -- 172.21.15.157:0/3260198839 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48c01a070 con 0x7fe49414f750 2024-04-24T23:35:18.997 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.922+0000 7fe4917fa640 1 -- 172.21.15.157:0/3260198839 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe48c006d60 con 0x7fe49414f750 2024-04-24T23:35:18.998 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.922+0000 7fe49ae29640 1 -- 172.21.15.157:0/3260198839 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe494151480 con 0x7fe49414f750 2024-04-24T23:35:18.998 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.922+0000 7fe4917fa640 1 -- 172.21.15.157:0/3260198839 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48c003bb0 con 0x7fe49414f750 2024-04-24T23:35:18.998 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.922+0000 7fe49ae29640 1 -- 172.21.15.157:0/3260198839 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe494151760 con 0x7fe49414f750 2024-04-24T23:35:18.998 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.922+0000 7fa2fffff640 1 -- 172.21.15.157:0/1974679134 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa308092050 con 0x7fa30c04e020 2024-04-24T23:35:18.998 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.922+0000 7f8db759e640 1 -- 172.21.15.157:0/3727495400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8db004c030 msgr2=0x7f8db004cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:18.999 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.922+0000 7f8db759e640 1 --2- 172.21.15.157:0/3727495400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8db004c030 0x7f8db004cc60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8db000bb90 tx=0x7f8dac02a1e0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.999 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.922+0000 7f8db759e640 1 -- 172.21.15.157:0/3727495400 shutdown_connections 2024-04-24T23:35:18.999 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.922+0000 7f8db759e640 1 --2- 172.21.15.157:0/3727495400 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8db004e4c0 0x7f8db004d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.999 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.922+0000 7f8db759e640 1 --2- 172.21.15.157:0/3727495400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8db004c030 0x7f8db004cc60 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.999 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.922+0000 7f8db759e640 1 --2- 172.21.15.157:0/3727495400 >> v2:172.21.15.157:3300/0 conn(0x7f8db00d58d0 0x7f8db004baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:18.999 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.922+0000 7f8db759e640 1 -- 172.21.15.157:0/3727495400 >> 172.21.15.157:0/3727495400 conn(0x7f8db004d9d0 msgr2=0x7f8db004ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:18.999 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.922+0000 7fa312ffd640 1 --2- 172.21.15.157:0/1974679134 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fa2f806bf00 0x7fa2f806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:18.999 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.922+0000 7ff433fff640 1 --2- 172.21.15.157:0/2465463571 >> [v2:172.21.15.157:6816/1557575576,v1:172.21.15.157:6817/1557575576] conn(0x7ff418075680 0x7ff418077ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.000 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.922+0000 7f7e18ff9640 1 --2- 172.21.15.157:0/556559919 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f7dfc06be30 0x7f7dfc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.000 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.922+0000 7f8db759e640 1 -- 172.21.15.157:0/3727495400 shutdown_connections 2024-04-24T23:35:19.000 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.922+0000 7f8db759e640 1 -- 172.21.15.157:0/3727495400 wait complete. 2024-04-24T23:35:19.000 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.923+0000 7f7e1affd640 1 --2- 172.21.15.157:0/556559919 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f7dfc06be30 0x7f7dfc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.001 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.923+0000 7fe49ae29640 1 -- 172.21.15.157:0/3260198839 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe49414df60 con 0x7fe49414f750 2024-04-24T23:35:19.001 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.923+0000 7f6f5e7fc640 1 -- 172.21.15.157:0/1751256345 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6f60014ba0 con 0x7f6f68072ec0 2024-04-24T23:35:19.001 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.923+0000 7f04ae7fc640 1 -- 172.21.15.157:0/526758282 <== osd.5 v2:172.21.15.169:6824/243321711 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f049c078110 con 0x7f049c075600 2024-04-24T23:35:19.001 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.923+0000 7f6f5e7fc640 1 --2- 172.21.15.157:0/1751256345 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f6f4c06be80 0x7f6f4c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.002 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.924+0000 7fe4917fa640 1 -- 172.21.15.157:0/3260198839 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe48c005110 con 0x7fe49414f750 2024-04-24T23:35:19.002 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.924+0000 7f6f5e7fc640 1 -- 172.21.15.157:0/1751256345 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f6f6008c2a0 con 0x7f6f68072ec0 2024-04-24T23:35:19.002 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.924+0000 7f7e18ff9640 1 -- 172.21.15.157:0/556559919 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f7e1408b2a0 con 0x7f7e1c0c8510 2024-04-24T23:35:19.003 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.924+0000 7f6f6dab7640 1 --2- 172.21.15.157:0/1751256345 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f6f4c06be80 0x7f6f4c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.003 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.924+0000 7f6f5e7fc640 1 --2- 172.21.15.157:0/1751256345 >> [v2:172.21.15.169:6800/321910099,v1:172.21.15.169:6801/321910099] conn(0x7f6f4c075580 0x7f6f4c0779c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.003 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.924+0000 7f8db759e640 1 Processor -- start 2024-04-24T23:35:19.003 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.924+0000 7f7e1affd640 1 --2- 172.21.15.157:0/556559919 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f7dfc06be30 0x7f7dfc06e2f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7e04001590 tx=0x7f7e040078f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:19.004 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.924+0000 7f6f5e7fc640 1 -- 172.21.15.157:0/1751256345 --> [v2:172.21.15.169:6800/321910099,v1:172.21.15.169:6801/321910099] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6f60015ec0 con 0x7f6f4c075580 2024-04-24T23:35:19.004 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.924+0000 7f7e18ff9640 1 --2- 172.21.15.157:0/556559919 >> [v2:172.21.15.157:6800/422758549,v1:172.21.15.157:6801/422758549] conn(0x7f7dfc0755a0 0x7f7dfc0779e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.004 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.924+0000 7fa312ffd640 1 --2- 172.21.15.157:0/1974679134 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fa2f806bf00 0x7fa2f806e3c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa30c035a00 tx=0x7fa300009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:19.004 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.924+0000 7f8db759e640 1 -- start start 2024-04-24T23:35:19.005 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.924+0000 7f6f6d2b6640 1 --2- 172.21.15.157:0/1751256345 >> [v2:172.21.15.169:6800/321910099,v1:172.21.15.169:6801/321910099] conn(0x7f6f4c075580 0x7f6f4c0779c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.005 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.924+0000 7fe4917fa640 1 --2- 172.21.15.157:0/3260198839 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fe47806bf30 0x7fe47806e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.005 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.924+0000 7fe4917fa640 1 -- 172.21.15.157:0/3260198839 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe48c08a7e0 con 0x7fe49414f750 2024-04-24T23:35:19.005 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.924+0000 7fe4917fa640 1 --2- 172.21.15.157:0/3260198839 >> [v2:172.21.15.169:6808/3441420785,v1:172.21.15.169:6809/3441420785] conn(0x7fe4780756b0 0x7fe478077af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.005 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.924+0000 7fe4917fa640 1 -- 172.21.15.157:0/3260198839 --> [v2:172.21.15.169:6808/3441420785,v1:172.21.15.169:6809/3441420785] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe48c021e50 con 0x7fe4780756b0 2024-04-24T23:35:19.005 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.924+0000 7fe4917fa640 1 -- 172.21.15.157:0/3260198839 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe48c091050 con 0x7fe49414f750 2024-04-24T23:35:19.006 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.924+0000 7ff433fff640 1 --2- 172.21.15.157:0/2465463571 >> [v2:172.21.15.157:6816/1557575576,v1:172.21.15.157:6817/1557575576] conn(0x7ff418075680 0x7ff418077ac0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:19.006 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.924+0000 7ff4337fe640 1 --2- 172.21.15.157:0/2465463571 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7ff41806bf00 0x7ff41806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.006 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.924+0000 7f6f6dab7640 1 --2- 172.21.15.157:0/1751256345 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f6f4c06be80 0x7f6f4c06e340 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6f6400a750 tx=0x7f6f64009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:19.006 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.925+0000 7f6f5e7fc640 1 -- 172.21.15.157:0/1751256345 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6f60056450 con 0x7f6f68072ec0 2024-04-24T23:35:19.006 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.925+0000 7ff41ffff640 1 -- 172.21.15.157:0/2465463571 <== osd.1 v2:172.21.15.157:6816/1557575576 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff420031ec0 con 0x7ff418075680 2024-04-24T23:35:19.007 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.925+0000 7f7e13fff640 1 --2- 172.21.15.157:0/556559919 >> [v2:172.21.15.157:6800/422758549,v1:172.21.15.157:6801/422758549] conn(0x7f7dfc0755a0 0x7f7dfc0779e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.007 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.925+0000 7fe498bc5640 1 --2- 172.21.15.157:0/3260198839 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fe47806bf30 0x7fe47806e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.007 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.925+0000 7fe493fff640 1 --2- 172.21.15.157:0/3260198839 >> [v2:172.21.15.169:6808/3441420785,v1:172.21.15.169:6809/3441420785] conn(0x7fe4780756b0 0x7fe478077af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.007 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.925+0000 7f7e18ff9640 1 -- 172.21.15.157:0/556559919 --> [v2:172.21.15.157:6800/422758549,v1:172.21.15.157:6801/422758549] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7e14038ec0 con 0x7f7dfc0755a0 2024-04-24T23:35:19.008 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.925+0000 7fa313fff640 1 --2- 172.21.15.157:0/1974679134 >> [v2:172.21.15.157:6808/4003822160,v1:172.21.15.157:6809/4003822160] conn(0x7fa2f80755f0 0x7fa2f8077a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.008 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.925+0000 7f6f6d2b6640 1 --2- 172.21.15.157:0/1751256345 >> [v2:172.21.15.169:6800/321910099,v1:172.21.15.169:6801/321910099] conn(0x7f6f4c075580 0x7f6f4c0779c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:19.008 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.925+0000 7ff4337fe640 1 --2- 172.21.15.157:0/2465463571 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7ff41806bf00 0x7ff41806e3c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff42c048bb0 tx=0x7ff428009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:19.008 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.925+0000 7f8db759e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8db004c030 0x7f8db003bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.009 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.925+0000 7fe493fff640 1 --2- 172.21.15.157:0/3260198839 >> [v2:172.21.15.169:6808/3441420785,v1:172.21.15.169:6809/3441420785] conn(0x7fe4780756b0 0x7fe478077af0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:19.009 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.925+0000 7fe498bc5640 1 --2- 172.21.15.157:0/3260198839 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fe47806bf30 0x7fe47806e3f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe488001eb0 tx=0x7fe48802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:19.009 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.925+0000 7f8db759e640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8db004e4c0 0x7f8db003c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.009 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.925+0000 7fa313fff640 1 --2- 172.21.15.157:0/1974679134 >> [v2:172.21.15.157:6808/4003822160,v1:172.21.15.157:6809/4003822160] conn(0x7fa2f80755f0 0x7fa2f8077a30 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:19.010 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.925+0000 7f7e13fff640 1 --2- 172.21.15.157:0/556559919 >> [v2:172.21.15.157:6800/422758549,v1:172.21.15.157:6801/422758549] conn(0x7f7dfc0755a0 0x7f7dfc0779e0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:19.010 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.925+0000 7f8db759e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8db00d58d0 0x7f8db00c16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.010 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.925+0000 7f8db759e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8db000c3b0 con 0x7f8db00d58d0 2024-04-24T23:35:19.011 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.925+0000 7f8db759e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8db002d1b0 con 0x7f8db004c030 2024-04-24T23:35:19.011 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.925+0000 7f8db759e640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8db00d8200 con 0x7f8db004e4c0 2024-04-24T23:35:19.011 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.926+0000 7f8db659c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8db004c030 0x7f8db003bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.011 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.926+0000 7f8db659c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8db004c030 0x7f8db003bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.157:44976/0 (socket says 172.21.15.157:44976) 2024-04-24T23:35:19.011 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.926+0000 7f8db659c640 1 -- 172.21.15.157:0/876746712 learned_addr learned my addr 172.21.15.157:0/876746712 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:19.011 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.926+0000 7f8db5d9b640 1 --2- 172.21.15.157:0/876746712 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8db004e4c0 0x7f8db003c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.011 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.926+0000 7f8db6d9d640 1 --2- 172.21.15.157:0/876746712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8db00d58d0 0x7f8db00c16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.011 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.926+0000 7f8db659c640 1 -- 172.21.15.157:0/876746712 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8db004e4c0 msgr2=0x7f8db003c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.011 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.926+0000 7f6f5e7fc640 1 -- 172.21.15.157:0/1751256345 <== osd.4 v2:172.21.15.169:6800/321910099 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6f60015ec0 con 0x7f6f4c075580 2024-04-24T23:35:19.012 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.926+0000 7f8db659c640 1 --2- 172.21.15.157:0/876746712 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8db004e4c0 0x7f8db003c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.012 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.926+0000 7f8db659c640 1 -- 172.21.15.157:0/876746712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8db00d58d0 msgr2=0x7f8db00c16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.012 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.926+0000 7f8db659c640 1 --2- 172.21.15.157:0/876746712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8db00d58d0 0x7f8db00c16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.012 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.926+0000 7f8db659c640 1 -- 172.21.15.157:0/876746712 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8db00ef240 con 0x7f8db004c030 2024-04-24T23:35:19.012 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.926+0000 7f8db6d9d640 1 --2- 172.21.15.157:0/876746712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8db00d58d0 0x7f8db00c16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:35:19.012 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.926+0000 7fe4917fa640 1 -- 172.21.15.157:0/3260198839 <== osd.6 v2:172.21.15.169:6808/3441420785 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe48c021e50 con 0x7fe4780756b0 2024-04-24T23:35:19.013 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.926+0000 7f8db659c640 1 --2- 172.21.15.157:0/876746712 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8db004c030 0x7f8db003bbd0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8da800d800 tx=0x7f8da800dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:19.013 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.926+0000 7fa2fffff640 1 -- 172.21.15.157:0/1974679134 <== osd.0 v2:172.21.15.157:6808/4003822160 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa308016ec0 con 0x7fa2f80755f0 2024-04-24T23:35:19.013 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.927+0000 7f8da6ffd640 1 -- 172.21.15.157:0/876746712 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da8004030 con 0x7f8db004c030 2024-04-24T23:35:19.013 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.927+0000 7f8da6ffd640 1 -- 172.21.15.157:0/876746712 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8da80041d0 con 0x7f8db004c030 2024-04-24T23:35:19.013 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.927+0000 7f8da6ffd640 1 -- 172.21.15.157:0/876746712 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da800b660 con 0x7f8db004c030 2024-04-24T23:35:19.014 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.926+0000 7f7e18ff9640 1 -- 172.21.15.157:0/556559919 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7e1408b690 con 0x7f7e1c0c8510 2024-04-24T23:35:19.014 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.927+0000 7f7e18ff9640 1 -- 172.21.15.157:0/556559919 <== osd.3 v2:172.21.15.157:6800/422758549 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7e14038ec0 con 0x7f7dfc0755a0 2024-04-24T23:35:19.015 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.927+0000 7f8db759e640 1 -- 172.21.15.157:0/876746712 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8db00096d0 con 0x7f8db004c030 2024-04-24T23:35:19.015 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.927+0000 7f8db759e640 1 -- 172.21.15.157:0/876746712 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8db003c830 con 0x7f8db004c030 2024-04-24T23:35:19.015 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.928+0000 7f8db759e640 1 -- 172.21.15.157:0/876746712 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8db003c650 con 0x7f8db004c030 2024-04-24T23:35:19.016 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.929+0000 7f8da6ffd640 1 -- 172.21.15.157:0/876746712 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8da800b800 con 0x7f8db004c030 2024-04-24T23:35:19.016 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.929+0000 7f8da6ffd640 1 --2- 172.21.15.157:0/876746712 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f8d9806bf30 0x7f8d9806e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.016 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.930+0000 7f8da6ffd640 1 -- 172.21.15.157:0/876746712 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f8da808b2e0 con 0x7f8db004c030 2024-04-24T23:35:19.016 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.930+0000 7f8da6ffd640 1 --2- 172.21.15.157:0/876746712 >> [v2:172.21.15.169:6816/1663179020,v1:172.21.15.169:6817/1663179020] conn(0x7f8d98075620 0x7f8d98077a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.016 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.930+0000 7f8da6ffd640 1 -- 172.21.15.157:0/876746712 --> [v2:172.21.15.169:6816/1663179020,v1:172.21.15.169:6817/1663179020] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8d98078130 con 0x7f8d98075620 2024-04-24T23:35:19.016 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.930+0000 7f8db6d9d640 1 --2- 172.21.15.157:0/876746712 >> [v2:172.21.15.169:6816/1663179020,v1:172.21.15.169:6817/1663179020] conn(0x7f8d98075620 0x7f8d98077a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.016 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.931+0000 7f8db5d9b640 1 --2- 172.21.15.157:0/876746712 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f8d9806bf30 0x7f8d9806e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.016 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.931+0000 7f8da6ffd640 1 -- 172.21.15.157:0/876746712 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8da8092050 con 0x7f8db004c030 2024-04-24T23:35:19.016 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.931+0000 7f8db6d9d640 1 --2- 172.21.15.157:0/876746712 >> [v2:172.21.15.169:6816/1663179020,v1:172.21.15.169:6817/1663179020] conn(0x7f8d98075620 0x7f8d98077a60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:19.016 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.931+0000 7f8db5d9b640 1 --2- 172.21.15.157:0/876746712 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f8d9806bf30 0x7f8d9806e3f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8dac034000 tx=0x7f8dac02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:19.016 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.932+0000 7f8da6ffd640 1 -- 172.21.15.157:0/876746712 <== osd.7 v2:172.21.15.169:6816/1663179020 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8d98078130 con 0x7f8d98075620 2024-04-24T23:35:19.017 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.954+0000 7fd46a173640 1 -- 172.21.15.157:0/2311295592 --> [v2:172.21.15.157:6824/2793948092,v1:172.21.15.157:6825/2793948092] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd444002dd0 con 0x7fd4480755a0 2024-04-24T23:35:19.017 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.955+0000 7fd459ffb640 1 -- 172.21.15.157:0/2311295592 <== osd.2 v2:172.21.15.157:6824/2793948092 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd444002dd0 con 0x7fd4480755a0 2024-04-24T23:35:19.017 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.955+0000 7fd44f7fe640 1 -- 172.21.15.157:0/2311295592 >> [v2:172.21.15.157:6824/2793948092,v1:172.21.15.157:6825/2793948092] conn(0x7fd4480755a0 msgr2=0x7fd4480779e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.017 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.955+0000 7fd44f7fe640 1 --2- 172.21.15.157:0/2311295592 >> [v2:172.21.15.157:6824/2793948092,v1:172.21.15.157:6825/2793948092] conn(0x7fd4480755a0 0x7fd4480779e0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.017 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.956+0000 7fd44f7fe640 1 -- 172.21.15.157:0/2311295592 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fd44806beb0 msgr2=0x7fd44806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.017 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.956+0000 7fd44f7fe640 1 --2- 172.21.15.157:0/2311295592 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fd44806beb0 0x7fd44806e370 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd460034000 tx=0x7fd46002e000 comp rx=0 tx=0).stop 2024-04-24T23:35:19.018 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.956+0000 7fd44f7fe640 1 -- 172.21.15.157:0/2311295592 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4640c60c0 msgr2=0x7fd4640ab3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.018 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.956+0000 7fd44f7fe640 1 --2- 172.21.15.157:0/2311295592 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4640c60c0 0x7fd4640ab3b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd45c009d90 tx=0x7fd45c00c490 comp rx=0 tx=0).stop 2024-04-24T23:35:19.018 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.956+0000 7f049bfff640 1 -- 172.21.15.157:0/526758282 --> [v2:172.21.15.169:6824/243321711,v1:172.21.15.169:6825/243321711] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f04b00057d0 con 0x7f049c075600 2024-04-24T23:35:19.019 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.956+0000 7fd469972640 1 -- 172.21.15.157:0/2311295592 reap_dead start 2024-04-24T23:35:19.019 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.956+0000 7fd44f7fe640 1 -- 172.21.15.157:0/2311295592 shutdown_connections 2024-04-24T23:35:19.019 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.956+0000 7fd44f7fe640 1 -- 172.21.15.157:0/2311295592 >> 172.21.15.157:0/2311295592 conn(0x7fd4640c6800 msgr2=0x7fd4640c51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:19.019 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.956+0000 7fd44f7fe640 1 -- 172.21.15.157:0/2311295592 shutdown_connections 2024-04-24T23:35:19.019 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.956+0000 7fd44f7fe640 1 -- 172.21.15.157:0/2311295592 wait complete. 2024-04-24T23:35:19.019 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.957+0000 7f04ae7fc640 1 -- 172.21.15.157:0/526758282 <== osd.5 v2:172.21.15.169:6824/243321711 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f04b00057d0 con 0x7f049c075600 2024-04-24T23:35:19.020 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.957+0000 7f049bfff640 1 -- 172.21.15.157:0/526758282 >> [v2:172.21.15.169:6824/243321711,v1:172.21.15.169:6825/243321711] conn(0x7f049c075600 msgr2=0x7f049c077a40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.020 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.957+0000 7f049bfff640 1 --2- 172.21.15.157:0/526758282 >> [v2:172.21.15.169:6824/243321711,v1:172.21.15.169:6825/243321711] conn(0x7f049c075600 0x7f049c077a40 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.021 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.957+0000 7f049bfff640 1 -- 172.21.15.157:0/526758282 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f049c06bf00 msgr2=0x7f049c06e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.021 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.957+0000 7f049bfff640 1 --2- 172.21.15.157:0/526758282 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f049c06bf00 0x7f049c06e3c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f04a8034000 tx=0x7f04a802e000 comp rx=0 tx=0).stop 2024-04-24T23:35:19.021 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.957+0000 7f049bfff640 1 -- 172.21.15.157:0/526758282 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04b000be70 msgr2=0x7f04b0042260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.021 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.957+0000 7f049bfff640 1 --2- 172.21.15.157:0/526758282 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04b000be70 0x7f04b0042260 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f04b400b3f0 tx=0x7f04b400b8b0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.021 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.958+0000 7f04be32d640 1 -- 172.21.15.157:0/526758282 reap_dead start 2024-04-24T23:35:19.021 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.958+0000 7f049bfff640 1 -- 172.21.15.157:0/526758282 shutdown_connections 2024-04-24T23:35:19.021 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.958+0000 7f049bfff640 1 -- 172.21.15.157:0/526758282 >> 172.21.15.157:0/526758282 conn(0x7f04b002db80 msgr2=0x7f04b002c500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:19.021 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.958+0000 7ff438b01640 1 -- 172.21.15.157:0/2465463571 --> [v2:172.21.15.157:6816/1557575576,v1:172.21.15.157:6817/1557575576] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff42c005380 con 0x7ff418075680 2024-04-24T23:35:19.021 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.958+0000 7ff41ffff640 1 -- 172.21.15.157:0/2465463571 <== osd.1 v2:172.21.15.157:6816/1557575576 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff42c005380 con 0x7ff418075680 2024-04-24T23:35:19.022 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.958+0000 7f049bfff640 1 -- 172.21.15.157:0/526758282 shutdown_connections 2024-04-24T23:35:19.022 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.958+0000 7f049bfff640 1 -- 172.21.15.157:0/526758282 wait complete. 2024-04-24T23:35:19.022 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.959+0000 7f6f6fd1b640 1 -- 172.21.15.157:0/1751256345 --> [v2:172.21.15.169:6800/321910099,v1:172.21.15.169:6801/321910099] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6f680c1a30 con 0x7f6f4c075580 2024-04-24T23:35:19.022 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.959+0000 7ff438b01640 1 -- 172.21.15.157:0/2465463571 >> [v2:172.21.15.157:6816/1557575576,v1:172.21.15.157:6817/1557575576] conn(0x7ff418075680 msgr2=0x7ff418077ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.023 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.959+0000 7ff438b01640 1 --2- 172.21.15.157:0/2465463571 >> [v2:172.21.15.157:6816/1557575576,v1:172.21.15.157:6817/1557575576] conn(0x7ff418075680 0x7ff418077ac0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.023 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.959+0000 7ff438b01640 1 -- 172.21.15.157:0/2465463571 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7ff41806bf00 msgr2=0x7ff41806e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.023 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.959+0000 7ff438b01640 1 --2- 172.21.15.157:0/2465463571 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7ff41806bf00 0x7ff41806e3c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff42c048bb0 tx=0x7ff428009040 comp rx=0 tx=0).stop 2024-04-24T23:35:19.023 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.959+0000 7ff438b01640 1 -- 172.21.15.157:0/2465463571 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff42c047490 msgr2=0x7ff42c02c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.023 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.959+0000 7ff438b01640 1 --2- 172.21.15.157:0/2465463571 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff42c047490 0x7ff42c02c980 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff420031000 tx=0x7ff4200046c0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.023 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.959+0000 7fe49ae29640 1 -- 172.21.15.157:0/3260198839 --> [v2:172.21.15.169:6808/3441420785,v1:172.21.15.169:6809/3441420785] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe4940681c0 con 0x7fe4780756b0 2024-04-24T23:35:19.023 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.959+0000 7fa31a25d640 1 -- 172.21.15.157:0/1974679134 --> [v2:172.21.15.157:6808/4003822160,v1:172.21.15.157:6809/4003822160] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa2f4002dd0 con 0x7fa2f80755f0 2024-04-24T23:35:19.024 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.959+0000 7f6f5e7fc640 1 -- 172.21.15.157:0/1751256345 <== osd.4 v2:172.21.15.169:6800/321910099 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6f680c1a30 con 0x7f6f4c075580 2024-04-24T23:35:19.024 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.959+0000 7ff433fff640 1 -- 172.21.15.157:0/2465463571 reap_dead start 2024-04-24T23:35:19.024 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.959+0000 7ff438b01640 1 -- 172.21.15.157:0/2465463571 shutdown_connections 2024-04-24T23:35:19.024 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.959+0000 7ff438b01640 1 -- 172.21.15.157:0/2465463571 >> 172.21.15.157:0/2465463571 conn(0x7ff42c048240 msgr2=0x7ff42c0dc560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:19.025 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.960+0000 7ff438b01640 1 -- 172.21.15.157:0/2465463571 shutdown_connections 2024-04-24T23:35:19.025 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.960+0000 7ff438b01640 1 -- 172.21.15.157:0/2465463571 wait complete. 2024-04-24T23:35:19.025 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.960+0000 7f7e21a4d640 1 -- 172.21.15.157:0/556559919 --> [v2:172.21.15.157:6800/422758549,v1:172.21.15.157:6801/422758549] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7e1c0c7f60 con 0x7f7dfc0755a0 2024-04-24T23:35:19.026 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.960+0000 7fa2fffff640 1 -- 172.21.15.157:0/1974679134 <== osd.0 v2:172.21.15.157:6808/4003822160 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa2f4002dd0 con 0x7fa2f80755f0 2024-04-24T23:35:19.026 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.960+0000 7fe4917fa640 1 -- 172.21.15.157:0/3260198839 <== osd.6 v2:172.21.15.169:6808/3441420785 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe4940681c0 con 0x7fe4780756b0 2024-04-24T23:35:19.026 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.960+0000 7f6f4bfff640 1 -- 172.21.15.157:0/1751256345 >> [v2:172.21.15.169:6800/321910099,v1:172.21.15.169:6801/321910099] conn(0x7f6f4c075580 msgr2=0x7f6f4c0779c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.026 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.960+0000 7f6f4bfff640 1 --2- 172.21.15.157:0/1751256345 >> [v2:172.21.15.169:6800/321910099,v1:172.21.15.169:6801/321910099] conn(0x7f6f4c075580 0x7f6f4c0779c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.027 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.960+0000 7f7e18ff9640 1 -- 172.21.15.157:0/556559919 <== osd.3 v2:172.21.15.157:6800/422758549 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7e1c0c7f60 con 0x7f7dfc0755a0 2024-04-24T23:35:19.027 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.961+0000 7fa31a25d640 1 -- 172.21.15.157:0/1974679134 >> [v2:172.21.15.157:6808/4003822160,v1:172.21.15.157:6809/4003822160] conn(0x7fa2f80755f0 msgr2=0x7fa2f8077a30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.027 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.961+0000 7fa31a25d640 1 --2- 172.21.15.157:0/1974679134 >> [v2:172.21.15.157:6808/4003822160,v1:172.21.15.157:6809/4003822160] conn(0x7fa2f80755f0 0x7fa2f8077a30 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.028 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.961+0000 7fe49ae29640 1 -- 172.21.15.157:0/3260198839 >> [v2:172.21.15.169:6808/3441420785,v1:172.21.15.169:6809/3441420785] conn(0x7fe4780756b0 msgr2=0x7fe478077af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.028 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.961+0000 7fe49ae29640 1 --2- 172.21.15.157:0/3260198839 >> [v2:172.21.15.169:6808/3441420785,v1:172.21.15.169:6809/3441420785] conn(0x7fe4780756b0 0x7fe478077af0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.028 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.961+0000 7f7e11ffb640 1 -- 172.21.15.157:0/556559919 >> [v2:172.21.15.157:6800/422758549,v1:172.21.15.157:6801/422758549] conn(0x7f7dfc0755a0 msgr2=0x7f7dfc0779e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.028 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.961+0000 7f7e11ffb640 1 --2- 172.21.15.157:0/556559919 >> [v2:172.21.15.157:6800/422758549,v1:172.21.15.157:6801/422758549] conn(0x7f7dfc0755a0 0x7f7dfc0779e0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.028 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.961+0000 7f6f4bfff640 1 -- 172.21.15.157:0/1751256345 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f6f4c06be80 msgr2=0x7f6f4c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.028 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.961+0000 7f6f4bfff640 1 --2- 172.21.15.157:0/1751256345 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f6f4c06be80 0x7f6f4c06e340 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6f6400a750 tx=0x7f6f64009040 comp rx=0 tx=0).stop 2024-04-24T23:35:19.029 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.961+0000 7f6f4bfff640 1 -- 172.21.15.157:0/1751256345 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f68072ec0 msgr2=0x7f6f68145100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.029 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.961+0000 7f6f4bfff640 1 --2- 172.21.15.157:0/1751256345 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f68072ec0 0x7f6f68145100 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6f60007e60 tx=0x7f6f6000e4b0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.029 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.961+0000 7f6f6e2b8640 1 -- 172.21.15.157:0/1751256345 reap_dead start 2024-04-24T23:35:19.029 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.961+0000 7fe49ae29640 1 -- 172.21.15.157:0/3260198839 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fe47806bf30 msgr2=0x7fe47806e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.029 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.961+0000 7fe49ae29640 1 --2- 172.21.15.157:0/3260198839 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fe47806bf30 0x7fe47806e3f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe488001eb0 tx=0x7fe48802e000 comp rx=0 tx=0).stop 2024-04-24T23:35:19.029 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.961+0000 7fe49ae29640 1 -- 172.21.15.157:0/3260198839 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe49414f750 msgr2=0x7fe4941e7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.029 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.961+0000 7fe49ae29640 1 --2- 172.21.15.157:0/3260198839 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe49414f750 0x7fe4941e7920 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe494151020 tx=0x7fe48c00fc90 comp rx=0 tx=0).stop 2024-04-24T23:35:19.029 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.961+0000 7fe4993c6640 1 -- 172.21.15.157:0/3260198839 reap_dead start 2024-04-24T23:35:19.029 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.961+0000 7fe49ae29640 1 -- 172.21.15.157:0/3260198839 shutdown_connections 2024-04-24T23:35:19.029 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.961+0000 7fe49ae29640 1 -- 172.21.15.157:0/3260198839 >> 172.21.15.157:0/3260198839 conn(0x7fe4940a8690 msgr2=0x7fe4940a7410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:19.030 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.961+0000 7fa31a25d640 1 -- 172.21.15.157:0/1974679134 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fa2f806bf00 msgr2=0x7fa2f806e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.030 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.961+0000 7fa31a25d640 1 --2- 172.21.15.157:0/1974679134 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fa2f806bf00 0x7fa2f806e3c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa30c035a00 tx=0x7fa300009040 comp rx=0 tx=0).stop 2024-04-24T23:35:19.030 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.961+0000 7fa31a25d640 1 -- 172.21.15.157:0/1974679134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa30c04e020 msgr2=0x7fa30c02f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.030 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.961+0000 7fa31a25d640 1 --2- 172.21.15.157:0/1974679134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa30c04e020 0x7fa30c02f590 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa30800e870 tx=0x7fa30800ed30 comp rx=0 tx=0).stop 2024-04-24T23:35:19.030 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.961+0000 7fa313fff640 1 -- 172.21.15.157:0/1974679134 reap_dead start 2024-04-24T23:35:19.030 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.961+0000 7fa31a25d640 1 -- 172.21.15.157:0/1974679134 shutdown_connections 2024-04-24T23:35:19.030 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.961+0000 7fa31a25d640 1 -- 172.21.15.157:0/1974679134 >> 172.21.15.157:0/1974679134 conn(0x7fa30c04c690 msgr2=0x7fa30c0d33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:19.031 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.962+0000 7f6f4bfff640 1 -- 172.21.15.157:0/1751256345 shutdown_connections 2024-04-24T23:35:19.031 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.962+0000 7f6f4bfff640 1 -- 172.21.15.157:0/1751256345 >> 172.21.15.157:0/1751256345 conn(0x7f6f680a8690 msgr2=0x7f6f68072160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:19.031 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.962+0000 7fa31a25d640 1 -- 172.21.15.157:0/1974679134 shutdown_connections 2024-04-24T23:35:19.031 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.962+0000 7fa31a25d640 1 -- 172.21.15.157:0/1974679134 wait complete. 2024-04-24T23:35:19.032 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.962+0000 7f7e11ffb640 1 -- 172.21.15.157:0/556559919 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f7dfc06be30 msgr2=0x7f7dfc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.032 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.962+0000 7f7e11ffb640 1 --2- 172.21.15.157:0/556559919 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f7dfc06be30 0x7f7dfc06e2f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7e04001590 tx=0x7f7e040078f0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.032 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.962+0000 7f7e11ffb640 1 -- 172.21.15.157:0/556559919 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e1c0c8510 msgr2=0x7f7e1c1530c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.032 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.962+0000 7f7e11ffb640 1 --2- 172.21.15.157:0/556559919 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e1c0c8510 0x7f7e1c1530c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7e1400b990 tx=0x7f7e1400be50 comp rx=0 tx=0).stop 2024-04-24T23:35:19.032 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.962+0000 7f7e1b7fe640 1 -- 172.21.15.157:0/556559919 reap_dead start 2024-04-24T23:35:19.032 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.962+0000 7f6f4bfff640 1 -- 172.21.15.157:0/1751256345 shutdown_connections 2024-04-24T23:35:19.032 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.962+0000 7f6f4bfff640 1 -- 172.21.15.157:0/1751256345 wait complete. 2024-04-24T23:35:19.033 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.962+0000 7fe49ae29640 1 -- 172.21.15.157:0/3260198839 shutdown_connections 2024-04-24T23:35:19.033 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.962+0000 7fe49ae29640 1 -- 172.21.15.157:0/3260198839 wait complete. 2024-04-24T23:35:19.033 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.962+0000 7f7e11ffb640 1 -- 172.21.15.157:0/556559919 shutdown_connections 2024-04-24T23:35:19.033 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.962+0000 7f7e11ffb640 1 -- 172.21.15.157:0/556559919 >> 172.21.15.157:0/556559919 conn(0x7f7e1c0aa7b0 msgr2=0x7f7e1c14c6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:19.033 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.962+0000 7f7e11ffb640 1 -- 172.21.15.157:0/556559919 shutdown_connections 2024-04-24T23:35:19.033 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.962+0000 7f7e11ffb640 1 -- 172.21.15.157:0/556559919 wait complete. 2024-04-24T23:35:19.034 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.965+0000 7f8db759e640 1 -- 172.21.15.157:0/876746712 --> [v2:172.21.15.169:6816/1663179020,v1:172.21.15.169:6817/1663179020] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8db00057d0 con 0x7f8d98075620 2024-04-24T23:35:19.034 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.966+0000 7f8da6ffd640 1 -- 172.21.15.157:0/876746712 <== osd.7 v2:172.21.15.169:6816/1663179020 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8db00057d0 con 0x7f8d98075620 2024-04-24T23:35:19.035 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.967+0000 7f8da4ff9640 1 -- 172.21.15.157:0/876746712 >> [v2:172.21.15.169:6816/1663179020,v1:172.21.15.169:6817/1663179020] conn(0x7f8d98075620 msgr2=0x7f8d98077a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.035 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.967+0000 7f8da4ff9640 1 --2- 172.21.15.157:0/876746712 >> [v2:172.21.15.169:6816/1663179020,v1:172.21.15.169:6817/1663179020] conn(0x7f8d98075620 0x7f8d98077a60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.035 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.967+0000 7f8da4ff9640 1 -- 172.21.15.157:0/876746712 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f8d9806bf30 msgr2=0x7f8d9806e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.035 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.967+0000 7f8da4ff9640 1 --2- 172.21.15.157:0/876746712 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f8d9806bf30 0x7f8d9806e3f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8dac034000 tx=0x7f8dac02e000 comp rx=0 tx=0).stop 2024-04-24T23:35:19.035 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.967+0000 7f8da4ff9640 1 -- 172.21.15.157:0/876746712 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8db004c030 msgr2=0x7f8db003bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.035 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.967+0000 7f8da4ff9640 1 --2- 172.21.15.157:0/876746712 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8db004c030 0x7f8db003bbd0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8da800d800 tx=0x7f8da800dcc0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.036 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.967+0000 7f8db6d9d640 1 -- 172.21.15.157:0/876746712 reap_dead start 2024-04-24T23:35:19.036 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.967+0000 7f8da4ff9640 1 -- 172.21.15.157:0/876746712 shutdown_connections 2024-04-24T23:35:19.036 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.967+0000 7f8da4ff9640 1 -- 172.21.15.157:0/876746712 >> 172.21.15.157:0/876746712 conn(0x7f8db004d9d0 msgr2=0x7f8db0048bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:19.036 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.968+0000 7f8da4ff9640 1 -- 172.21.15.157:0/876746712 shutdown_connections 2024-04-24T23:35:19.036 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:18.968+0000 7f8da4ff9640 1 -- 172.21.15.157:0/876746712 wait complete. 2024-04-24T23:35:19.037 INFO:teuthology.orchestra.run.smithi157.stdout:60129542152 2024-04-24T23:35:19.037 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-24T23:35:19.038 INFO:teuthology.orchestra.run.smithi157.stdout:60129542152 2024-04-24T23:35:19.038 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-24T23:35:19.039 INFO:teuthology.orchestra.run.smithi157.stdout:60129542152 2024-04-24T23:35:19.039 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-24T23:35:19.040 INFO:teuthology.orchestra.run.smithi157.stdout:60129542152 2024-04-24T23:35:19.040 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-24T23:35:19.041 INFO:teuthology.orchestra.run.smithi157.stdout:60129542152 2024-04-24T23:35:19.041 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-24T23:35:19.042 INFO:teuthology.orchestra.run.smithi157.stdout:60129542153 2024-04-24T23:35:19.042 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-24T23:35:19.043 INFO:teuthology.orchestra.run.smithi157.stdout:55834574857 2024-04-24T23:35:19.043 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-24T23:35:19.044 INFO:teuthology.orchestra.run.smithi157.stdout:60129542152 2024-04-24T23:35:19.044 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-24T23:35:19.245 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.244+0000 7fd0675ce640 1 Processor -- start 2024-04-24T23:35:19.245 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.244+0000 7f1933d1b640 1 Processor -- start 2024-04-24T23:35:19.245 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.244+0000 7f1933d1b640 1 -- start start 2024-04-24T23:35:19.246 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.245+0000 7f947dcc7640 1 Processor -- start 2024-04-24T23:35:19.246 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.245+0000 7fc4de8a7640 1 Processor -- start 2024-04-24T23:35:19.246 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.245+0000 7fc4de8a7640 1 -- start start 2024-04-24T23:35:19.247 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.246+0000 7f947dcc7640 1 -- start start 2024-04-24T23:35:19.247 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.246+0000 7fcc2dc53640 1 Processor -- start 2024-04-24T23:35:19.247 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.246+0000 7fcc2dc53640 1 -- start start 2024-04-24T23:35:19.248 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.246+0000 7f1c04e12640 1 Processor -- start 2024-04-24T23:35:19.248 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.246+0000 7f1933d1b640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f192c15a5d0 0x7f192c0c95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.248 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.246+0000 7f1c04e12640 1 -- start start 2024-04-24T23:35:19.249 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.246+0000 7f1933d1b640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f192c0c58a0 0x7f192c0c9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.249 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.246+0000 7f1933d1b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f192c0c6800 0x7f192c0c6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.249 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.246+0000 7f1933d1b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f192c15a9b0 con 0x7f192c0c6800 2024-04-24T23:35:19.249 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.246+0000 7f1933d1b640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f192c06bbf0 con 0x7f192c15a5d0 2024-04-24T23:35:19.250 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.246+0000 7f1933d1b640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f192c06bed0 con 0x7f192c0c58a0 2024-04-24T23:35:19.250 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.246+0000 7f1931ab7640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f192c15a5d0 0x7f192c0c95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.250 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.247+0000 7f433759e640 1 Processor -- start 2024-04-24T23:35:19.251 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.246+0000 7f1931ab7640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f192c15a5d0 0x7f192c0c95b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.157:44984/0 (socket says 172.21.15.157:44984) 2024-04-24T23:35:19.251 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.246+0000 7f1931ab7640 1 -- 172.21.15.157:0/258337576 learned_addr learned my addr 172.21.15.157:0/258337576 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:19.251 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.247+0000 7f433759e640 1 -- start start 2024-04-24T23:35:19.252 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.247+0000 7f947dcc7640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94700d4800 0x7f9470047260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.252 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.247+0000 7f947dcc7640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f947004a7e0 0x7f947004bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.252 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.247+0000 7f947dcc7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f94700477a0 0x7f947004c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.252 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.247+0000 7f947dcc7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f947004acf0 con 0x7f94700477a0 2024-04-24T23:35:19.252 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.247+0000 7f947dcc7640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f947004b700 con 0x7f947004a7e0 2024-04-24T23:35:19.252 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.247+0000 7f947dcc7640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f947004c610 con 0x7f94700d4800 2024-04-24T23:35:19.253 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.246+0000 7fc4de8a7640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4d80c4c40 0x7fc4d80c3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.253 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.247+0000 7fc4de8a7640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc4d815a6f0 0x7fc4d815cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.253 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.247+0000 7fc4d7fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fc4d81572a0 0x7fc4d806ee10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:47654/0 (socket says 172.21.15.157:47654) 2024-04-24T23:35:19.253 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.247+0000 7fc4d7fff640 1 -- 172.21.15.157:0/2628093009 learned_addr learned my addr 172.21.15.157:0/2628093009 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-24T23:35:19.253 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.247+0000 7fc4de8a7640 1 -- 172.21.15.157:0/2628093009 --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4d8157680 con 0x7fc4d81572a0 2024-04-24T23:35:19.253 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.247+0000 7fc4de8a7640 1 -- 172.21.15.157:0/2628093009 --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fc4d81599e0 con 0x7fc4d815a6f0 2024-04-24T23:35:19.253 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.247+0000 7fc4de8a7640 1 -- 172.21.15.157:0/2628093009 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc4d80c50e0 con 0x7fc4d80c4c40 2024-04-24T23:35:19.253 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.247+0000 7fc4dce44640 1 --2- 172.21.15.157:0/2628093009 >> v2:172.21.15.169:3300/0 conn(0x7fc4d815a6f0 0x7fc4d815cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.254 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.247+0000 7f947d4c6640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f94700477a0 0x7f947004c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.254 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.247+0000 7f947d4c6640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f94700477a0 0x7f947004c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42742/0 (socket says 172.21.15.157:42742) 2024-04-24T23:35:19.254 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.247+0000 7f947d4c6640 1 -- 172.21.15.157:0/1516007815 learned_addr learned my addr 172.21.15.157:0/1516007815 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:19.254 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.247+0000 7fc4d77fe640 1 --2- 172.21.15.157:0/2628093009 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4d80c4c40 0x7fc4d80c3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.255 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.245+0000 7fd0675ce640 1 -- start start 2024-04-24T23:35:19.255 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.246+0000 7fd0675ce640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd060156e90 0x7fd0600c47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.255 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.246+0000 7fd0675ce640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd0600c1c50 0x7fd0600c4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.255 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.246+0000 7fd0675ce640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd060157270 con 0x7fd06006d960 2024-04-24T23:35:19.255 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.246+0000 7fd0675ce640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fd0600c2860 con 0x7fd060156e90 2024-04-24T23:35:19.255 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.246+0000 7fd0675ce640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fd0600c2c30 con 0x7fd0600c1c50 2024-04-24T23:35:19.255 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.246+0000 7fd064b69640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fd06006d960 0x7fd060159800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.157:42576/0 (socket says 172.21.15.157:42576) 2024-04-24T23:35:19.255 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.247+0000 7fd064b69640 1 -- 172.21.15.157:0/1621178143 learned_addr learned my addr 172.21.15.157:0/1621178143 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-24T23:35:19.256 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.247+0000 7fd06536a640 1 --2- 172.21.15.157:0/1621178143 >> v2:172.21.15.157:3300/0 conn(0x7fd060156e90 0x7fd0600c47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.256 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.247+0000 7fd065b6b640 1 --2- 172.21.15.157:0/1621178143 >> v2:172.21.15.169:3300/0 conn(0x7fd0600c1c50 0x7fd0600c4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.256 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.247+0000 7fc4dce44640 1 -- 172.21.15.157:0/2628093009 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4d80c4c40 msgr2=0x7fc4d80c3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:19.256 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.247+0000 7fd057fff640 1 -- 172.21.15.157:0/1621178143 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2300681991 0 0) 0x7fd060157270 con 0x7fd06006d960 2024-04-24T23:35:19.257 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.247+0000 7fcc2dc53640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcc28156c90 0x7fcc2806db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.257 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.247+0000 7fcc2dc53640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcc280c57c0 0x7fcc28159580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.257 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.247+0000 7fcc2dc53640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc2815cd10 con 0x7fcc2815d4d0 2024-04-24T23:35:19.257 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.247+0000 7fcc2dc53640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fcc2806e760 con 0x7fcc280c57c0 2024-04-24T23:35:19.257 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.247+0000 7fcc2dc53640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fcc2806e1a0 con 0x7fcc28156c90 2024-04-24T23:35:19.257 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.247+0000 7fd057fff640 1 -- 172.21.15.157:0/1621178143 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd04c003520 con 0x7fd06006d960 2024-04-24T23:35:19.258 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.247+0000 7f947ccc5640 1 --2- 172.21.15.157:0/1516007815 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94700d4800 0x7f9470047260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.258 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.247+0000 7f9477fff640 1 --2- 172.21.15.157:0/1516007815 >> v2:172.21.15.169:3300/0 conn(0x7f947004a7e0 0x7f947004bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.258 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.247+0000 7f947ccc5640 1 -- 172.21.15.157:0/1516007815 >> v2:172.21.15.169:3300/0 conn(0x7f947004a7e0 msgr2=0x7f947004bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:19.258 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.247+0000 7f947ccc5640 1 --2- 172.21.15.157:0/1516007815 >> v2:172.21.15.169:3300/0 conn(0x7f947004a7e0 0x7f947004bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.258 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.247+0000 7f947ccc5640 1 -- 172.21.15.157:0/1516007815 >> v2:172.21.15.157:3300/0 conn(0x7f94700477a0 msgr2=0x7f947004c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:19.258 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.247+0000 7f947ccc5640 1 --2- 172.21.15.157:0/1516007815 >> v2:172.21.15.157:3300/0 conn(0x7f94700477a0 0x7f947004c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.258 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.248+0000 7f947ccc5640 1 -- 172.21.15.157:0/1516007815 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94700d7ba0 con 0x7f94700d4800 2024-04-24T23:35:19.259 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.247+0000 7fd057fff640 1 -- 172.21.15.157:0/1621178143 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2154639049 0 0) 0x7fd04c003520 con 0x7fd06006d960 2024-04-24T23:35:19.259 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.248+0000 7f1c04e12640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1c00072630 0x7f1c000c2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.259 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.248+0000 7f1c04e12640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1c000c2d50 0x7f1c00152870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.259 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.247+0000 7fd057fff640 1 -- 172.21.15.157:0/1621178143 >> v2:172.21.15.169:3300/0 conn(0x7fd0600c1c50 msgr2=0x7fd0600c4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:19.260 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.247+0000 7fc4dce44640 1 --2- 172.21.15.157:0/2628093009 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4d80c4c40 0x7fc4d80c3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.260 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.247+0000 7fc4dce44640 1 -- 172.21.15.157:0/2628093009 >> v1:172.21.15.157:6789/0 conn(0x7fc4d81572a0 legacy=0x7fc4d806ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.260 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.247+0000 7fc4dce44640 1 -- 172.21.15.157:0/2628093009 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4d8159c10 con 0x7fc4d815a6f0 2024-04-24T23:35:19.261 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.248+0000 7f1c04e12640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c00151490 con 0x7f1c0014e990 2024-04-24T23:35:19.261 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.248+0000 7f1c04e12640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f1c00152060 con 0x7f1c000c2d50 2024-04-24T23:35:19.261 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.248+0000 7f1c04e12640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f1c00151730 con 0x7f1c00072630 2024-04-24T23:35:19.261 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.247+0000 7fd057fff640 1 --2- 172.21.15.157:0/1621178143 >> v2:172.21.15.169:3300/0 conn(0x7fd0600c1c50 0x7fd0600c4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.262 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.247+0000 7fd057fff640 1 -- 172.21.15.157:0/1621178143 >> v2:172.21.15.157:3300/0 conn(0x7fd060156e90 msgr2=0x7fd0600c47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:19.262 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.247+0000 7fd057fff640 1 --2- 172.21.15.157:0/1621178143 >> v2:172.21.15.157:3300/0 conn(0x7fd060156e90 0x7fd0600c47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.262 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.247+0000 7fd057fff640 1 -- 172.21.15.157:0/1621178143 --> v1:172.21.15.169:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd06006cec0 con 0x7fd06006d960 2024-04-24T23:35:19.262 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.247+0000 7fd057fff640 1 -- 172.21.15.157:0/1621178143 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1480525340 0 0) 0x7fd05c001670 con 0x7fd06006d960 2024-04-24T23:35:19.262 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.247+0000 7fd057fff640 1 -- 172.21.15.157:0/1621178143 >> v1:172.21.15.169:6789/0 conn(0x7fd06006d960 legacy=0x7fd060159800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.262 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.248+0000 7f433759e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f433806f860 0x7f43380c42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.263 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.248+0000 7fc4dce44640 1 --2- 172.21.15.157:0/2628093009 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4d815a6f0 0x7fc4d815cf50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0009d90 tx=0x7fc4d002a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7707dd5f6eb9acd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:19.263 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.248+0000 7f1bfed9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1c000c2d50 0x7f1c00152870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.264 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.247+0000 7f1931ab7640 1 -- 172.21.15.157:0/258337576 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f192c0c58a0 msgr2=0x7f192c0c9af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T23:35:19.264 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.247+0000 7f19322b8640 1 --2- 172.21.15.157:0/258337576 >> v2:172.21.15.157:3300/0 conn(0x7f192c0c6800 0x7f192c0c6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.264 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.247+0000 7f1931ab7640 1 --2- 172.21.15.157:0/258337576 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f192c0c58a0 0x7f192c0c9af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.264 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.247+0000 7f1931ab7640 1 -- 172.21.15.157:0/258337576 >> v2:172.21.15.157:3300/0 conn(0x7f192c0c6800 msgr2=0x7f192c0c6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:19.264 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.247+0000 7f1931ab7640 1 --2- 172.21.15.157:0/258337576 >> v2:172.21.15.157:3300/0 conn(0x7f192c0c6800 0x7f192c0c6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.264 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.247+0000 7f1931ab7640 1 -- 172.21.15.157:0/258337576 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f192c0c64b0 con 0x7f192c15a5d0 2024-04-24T23:35:19.264 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.248+0000 7f19322b8640 1 --2- 172.21.15.157:0/258337576 >> v2:172.21.15.157:3300/0 conn(0x7f192c0c6800 0x7f192c0c6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:35:19.264 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.249+0000 7f433759e640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43380c6e70 0x7f43380c7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.265 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.249+0000 7f433659c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f4338157370 0x7f43380c8d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:47674/0 (socket says 172.21.15.157:47674) 2024-04-24T23:35:19.265 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.248+0000 7f1bfed9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1c000c2d50 0x7f1c00152870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42770/0 (socket says 172.21.15.157:42770) 2024-04-24T23:35:19.265 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.249+0000 7f1931ab7640 1 --2- 172.21.15.157:0/258337576 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f192c15a5d0 0x7f192c0c95b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f192c0ad8a0 tx=0x7f192802a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=132f919d6344055c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:19.265 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.249+0000 7f1bfed9d640 1 -- 172.21.15.157:0/204256346 learned_addr learned my addr 172.21.15.157:0/204256346 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:19.266 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.249+0000 7f433759e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4338071170 con 0x7f4338157370 2024-04-24T23:35:19.266 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.249+0000 7f4336d9d640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43380c6e70 0x7f43380c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.266 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.249+0000 7f433759e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f43380705f0 con 0x7f433806f860 2024-04-24T23:35:19.266 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.249+0000 7f433759e640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f43380708d0 con 0x7f43380c6e70 2024-04-24T23:35:19.267 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.247+0000 7fd057fff640 1 --2- 172.21.15.157:0/1621178143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd04c003a00 0x7fd04c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.267 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.247+0000 7fd057fff640 1 --2- 172.21.15.157:0/1621178143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd04c004800 0x7fd04c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.267 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.248+0000 7fd057fff640 1 --2- 172.21.15.157:0/1621178143 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd04c0051c0 0x7fd04c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.267 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.248+0000 7fd057fff640 1 -- 172.21.15.157:0/1621178143 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd0600c2860 con 0x7fd04c004800 2024-04-24T23:35:19.267 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.248+0000 7fd057fff640 1 -- 172.21.15.157:0/1621178143 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd0600c2c30 con 0x7fd04c003a00 2024-04-24T23:35:19.267 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.248+0000 7fd057fff640 1 -- 172.21.15.157:0/1621178143 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd04c00c060 con 0x7fd04c0051c0 2024-04-24T23:35:19.267 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.248+0000 7fd064b69640 1 --2- 172.21.15.157:0/1621178143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd04c004800 0x7fd04c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.268 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.248+0000 7fd065b6b640 1 --2- 172.21.15.157:0/1621178143 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd04c0051c0 0x7fd04c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.268 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.248+0000 7fd065b6b640 1 -- 172.21.15.157:0/1621178143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd04c003a00 msgr2=0x7fd04c003e20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T23:35:19.268 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.248+0000 7fd065b6b640 1 --2- 172.21.15.157:0/1621178143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd04c003a00 0x7fd04c003e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.268 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.248+0000 7fd065b6b640 1 -- 172.21.15.157:0/1621178143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd04c004800 msgr2=0x7fd04c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:19.268 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.248+0000 7fd065b6b640 1 --2- 172.21.15.157:0/1621178143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd04c004800 0x7fd04c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.268 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.249+0000 7fc4d6ffd640 1 -- 172.21.15.157:0/2628093009 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d0007740 con 0x7fc4d815a6f0 2024-04-24T23:35:19.268 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.249+0000 7fc4d6ffd640 1 -- 172.21.15.157:0/2628093009 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4d00078e0 con 0x7fc4d815a6f0 2024-04-24T23:35:19.268 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.249+0000 7fc4d6ffd640 1 -- 172.21.15.157:0/2628093009 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d0007cf0 con 0x7fc4d815a6f0 2024-04-24T23:35:19.269 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.248+0000 7fd065b6b640 1 -- 172.21.15.157:0/1621178143 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd04c003520 con 0x7fd04c0051c0 2024-04-24T23:35:19.269 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.249+0000 7f4336d9d640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43380c6e70 0x7f43380c7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.157:52786/0 (socket says 172.21.15.157:52786) 2024-04-24T23:35:19.269 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.249+0000 7f4336d9d640 1 -- 172.21.15.157:0/245912577 learned_addr learned my addr 172.21.15.157:0/245912577 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:19.269 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.248+0000 7fcc277fe640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcc28156c90 0x7fcc2806db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.270 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.248+0000 7fcc277fe640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcc28156c90 0x7fcc2806db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.157:45014/0 (socket says 172.21.15.157:45014) 2024-04-24T23:35:19.270 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.248+0000 7fcc277fe640 1 -- 172.21.15.157:0/3272192236 learned_addr learned my addr 172.21.15.157:0/3272192236 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:19.270 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.248+0000 7fcc26ffd640 1 --2- 172.21.15.157:0/3272192236 >> v2:172.21.15.157:3300/0 conn(0x7fcc280c57c0 0x7fcc28159580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.270 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.248+0000 7fcc277fe640 1 -- 172.21.15.157:0/3272192236 >> v2:172.21.15.157:3300/0 conn(0x7fcc280c57c0 msgr2=0x7fcc28159580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:19.270 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.248+0000 7fcc277fe640 1 --2- 172.21.15.157:0/3272192236 >> v2:172.21.15.157:3300/0 conn(0x7fcc280c57c0 0x7fcc28159580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.270 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.248+0000 7fcc277fe640 1 -- 172.21.15.157:0/3272192236 >> v1:172.21.15.157:6789/0 conn(0x7fcc2815d4d0 legacy=0x7fcc280c2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:19.270 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.248+0000 7fcc277fe640 1 -- 172.21.15.157:0/3272192236 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc2806e480 con 0x7fcc28156c90 2024-04-24T23:35:19.271 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.248+0000 7fa46a226640 1 Processor -- start 2024-04-24T23:35:19.271 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.249+0000 7fa46a226640 1 -- start start 2024-04-24T23:35:19.271 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.249+0000 7f947ccc5640 1 --2- 172.21.15.157:0/1516007815 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94700d4800 0x7f9470047260 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f947000b2e0 tx=0x7f9464029ff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbff0c417b02712c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:19.271 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.249+0000 7f94777fe640 1 -- 172.21.15.157:0/1516007815 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9464037070 con 0x7f94700d4800 2024-04-24T23:35:19.271 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.249+0000 7f94777fe640 1 -- 172.21.15.157:0/1516007815 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f946402ad40 con 0x7f94700d4800 2024-04-24T23:35:19.272 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.249+0000 7fd065b6b640 1 -- 172.21.15.157:0/1621178143 reap_dead start 2024-04-24T23:35:19.272 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.249+0000 7f94777fe640 1 -- 172.21.15.157:0/1516007815 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9464032660 con 0x7f94700d4800 2024-04-24T23:35:19.272 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.249+0000 7f432ffff640 1 --2- 172.21.15.157:0/245912577 >> v2:172.21.15.157:3300/0 conn(0x7f433806f860 0x7f43380c42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.273 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.249+0000 7fd065b6b640 1 --2- 172.21.15.157:0/1621178143 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd04c0051c0 0x7fd04c00b9c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd058009fc0 tx=0x7fd04c005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e6a6e42213202b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:19.273 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.249+0000 7f1bfdd9b640 1 --2- 172.21.15.157:0/204256346 >> v2:172.21.15.169:3300/0 conn(0x7f1c00072630 0x7f1c000c2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.274 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.250+0000 7fd057fff640 1 -- 172.21.15.157:0/1621178143 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd058035020 con 0x7fd04c0051c0 2024-04-24T23:35:19.274 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.250+0000 7fd057fff640 1 -- 172.21.15.157:0/1621178143 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd058003bd0 con 0x7fd04c0051c0 2024-04-24T23:35:19.274 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.250+0000 7fd057fff640 1 -- 172.21.15.157:0/1621178143 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd058009c10 con 0x7fd04c0051c0 2024-04-24T23:35:19.275 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.250+0000 7fcc277fe640 1 --2- 172.21.15.157:0/3272192236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc28156c90 0x7fcc2806db50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fcc280abf10 tx=0x7fcc1c02a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbf04fee4c60f469 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:19.275 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.250+0000 7fa46a226640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa45c004b00 0x7fa45c0d7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.275 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.250+0000 7fa46a226640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa45c0d6d10 0x7fa45c0d3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.275 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.250+0000 7fa46a226640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa45c007330 con 0x7fa45c0d3d80 2024-04-24T23:35:19.275 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.250+0000 7fa46a226640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fa45c006e00 con 0x7fa45c0d6d10 2024-04-24T23:35:19.275 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.250+0000 7fa46a226640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa45c0d6510 con 0x7fa45c004b00 2024-04-24T23:35:19.276 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.250+0000 7f1bfdd9b640 1 -- 172.21.15.157:0/204256346 >> v2:172.21.15.157:3300/0 conn(0x7f1c000c2d50 msgr2=0x7f1c00152870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:19.276 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.250+0000 7f1bfdd9b640 1 --2- 172.21.15.157:0/204256346 >> v2:172.21.15.157:3300/0 conn(0x7f1c000c2d50 0x7f1c00152870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.276 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.250+0000 7f1bfdd9b640 1 -- 172.21.15.157:0/204256346 >> v1:172.21.15.157:6789/0 conn(0x7f1c0014e990 legacy=0x7f1c00155380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:19.276 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.250+0000 7f1bfdd9b640 1 -- 172.21.15.157:0/204256346 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c00151a10 con 0x7f1c00072630 2024-04-24T23:35:19.277 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.250+0000 7f947dcc7640 1 -- 172.21.15.157:0/1516007815 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94700d4800 msgr2=0x7f9470047260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.277 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.250+0000 7f947dcc7640 1 --2- 172.21.15.157:0/1516007815 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94700d4800 0x7f9470047260 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f947000b2e0 tx=0x7f9464029ff0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.277 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.250+0000 7fa469a25640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fa45c0d3d80 0x7fa45c0465a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.157:42584/0 (socket says 172.21.15.157:42584) 2024-04-24T23:35:19.277 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.250+0000 7fa469a25640 1 -- 172.21.15.157:0/3318805823 learned_addr learned my addr 172.21.15.157:0/3318805823 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-24T23:35:19.277 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.250+0000 7fa469224640 1 --2- 172.21.15.157:0/3318805823 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa45c004b00 0x7fa45c0d7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.277 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.250+0000 7fa468a23640 1 --2- 172.21.15.157:0/3318805823 >> v2:172.21.15.169:3300/0 conn(0x7fa45c0d6d10 0x7fa45c0d3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.277 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.250+0000 7fa45bfff640 1 -- 172.21.15.157:0/3318805823 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1474488670 0 0) 0x7fa45c007330 con 0x7fa45c0d3d80 2024-04-24T23:35:19.278 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.250+0000 7fa45bfff640 1 -- 172.21.15.157:0/3318805823 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa450003520 con 0x7fa45c0d3d80 2024-04-24T23:35:19.278 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.249+0000 7f4336d9d640 1 -- 172.21.15.157:0/245912577 >> v2:172.21.15.157:3300/0 conn(0x7f433806f860 msgr2=0x7f43380c42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:19.278 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.250+0000 7f4336d9d640 1 --2- 172.21.15.157:0/245912577 >> v2:172.21.15.157:3300/0 conn(0x7f433806f860 0x7f43380c42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.278 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.250+0000 7f4336d9d640 1 -- 172.21.15.157:0/245912577 >> v1:172.21.15.157:6789/0 conn(0x7f4338157370 legacy=0x7f43380c8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:19.278 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.250+0000 7f4336d9d640 1 -- 172.21.15.157:0/245912577 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4338070bb0 con 0x7f43380c6e70 2024-04-24T23:35:19.278 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.250+0000 7f432ffff640 1 --2- 172.21.15.157:0/245912577 >> v2:172.21.15.157:3300/0 conn(0x7f433806f860 0x7f43380c42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:35:19.279 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.250+0000 7fc4de8a7640 1 -- 172.21.15.157:0/2628093009 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4d815a6f0 msgr2=0x7fc4d815cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.279 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.250+0000 7fc4de8a7640 1 --2- 172.21.15.157:0/2628093009 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4d815a6f0 0x7fc4d815cf50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0009d90 tx=0x7fc4d002a5f0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.279 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.250+0000 7fc4de8a7640 1 -- 172.21.15.157:0/2628093009 shutdown_connections 2024-04-24T23:35:19.279 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.250+0000 7fc4de8a7640 1 --2- 172.21.15.157:0/2628093009 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4d815a6f0 0x7fc4d815cf50 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.279 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.250+0000 7fc4de8a7640 1 --2- 172.21.15.157:0/2628093009 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4d80c4c40 0x7fc4d80c3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.279 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.250+0000 7fc4de8a7640 1 -- 172.21.15.157:0/2628093009 >> 172.21.15.157:0/2628093009 conn(0x7fc4d806de00 msgr2=0x7fc4d80a96a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:19.279 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.250+0000 7fc4de8a7640 1 -- 172.21.15.157:0/2628093009 shutdown_connections 2024-04-24T23:35:19.279 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.250+0000 7fc4de8a7640 1 -- 172.21.15.157:0/2628093009 wait complete. 2024-04-24T23:35:19.279 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.250+0000 7f947dcc7640 1 -- 172.21.15.157:0/1516007815 shutdown_connections 2024-04-24T23:35:19.279 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.250+0000 7f947dcc7640 1 --2- 172.21.15.157:0/1516007815 >> v2:172.21.15.157:3300/0 conn(0x7f94700477a0 0x7f947004c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.280 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.250+0000 7f947dcc7640 1 --2- 172.21.15.157:0/1516007815 >> v2:172.21.15.169:3300/0 conn(0x7f947004a7e0 0x7f947004bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.280 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.250+0000 7f947dcc7640 1 --2- 172.21.15.157:0/1516007815 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94700d4800 0x7f9470047260 secure :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f947000b2e0 tx=0x7f9464029ff0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.280 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.250+0000 7f947dcc7640 1 -- 172.21.15.157:0/1516007815 >> 172.21.15.157:0/1516007815 conn(0x7f9470049360 msgr2=0x7f9470049770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:19.280 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.250+0000 7fd0675ce640 1 -- 172.21.15.157:0/1621178143 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd04c0051c0 msgr2=0x7fd04c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.280 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.251+0000 7fd0675ce640 1 --2- 172.21.15.157:0/1621178143 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd04c0051c0 0x7fd04c00b9c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd058009fc0 tx=0x7fd04c005830 comp rx=0 tx=0).stop 2024-04-24T23:35:19.281 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.251+0000 7fd0675ce640 1 -- 172.21.15.157:0/1621178143 shutdown_connections 2024-04-24T23:35:19.281 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.251+0000 7fa45bfff640 1 -- 172.21.15.157:0/3318805823 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3067341228 0 0) 0x7fa450003520 con 0x7fa45c0d3d80 2024-04-24T23:35:19.281 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.251+0000 7fa45bfff640 1 -- 172.21.15.157:0/3318805823 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa45c004b00 msgr2=0x7fa45c0d7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:19.281 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.250+0000 7f1930ab5640 1 -- 172.21.15.157:0/258337576 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1928037070 con 0x7f192c15a5d0 2024-04-24T23:35:19.282 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.251+0000 7fa45bfff640 1 --2- 172.21.15.157:0/3318805823 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa45c004b00 0x7fa45c0d7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.282 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.251+0000 7fa45bfff640 1 -- 172.21.15.157:0/3318805823 >> v2:172.21.15.169:3300/0 conn(0x7fa45c0d6d10 msgr2=0x7fa45c0d3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:19.282 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.251+0000 7f947dcc7640 1 -- 172.21.15.157:0/1516007815 shutdown_connections 2024-04-24T23:35:19.283 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.251+0000 7fa45bfff640 1 --2- 172.21.15.157:0/3318805823 >> v2:172.21.15.169:3300/0 conn(0x7fa45c0d6d10 0x7fa45c0d3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.283 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.250+0000 7f1930ab5640 1 -- 172.21.15.157:0/258337576 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1928032440 con 0x7f192c15a5d0 2024-04-24T23:35:19.283 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.250+0000 7f1930ab5640 1 -- 172.21.15.157:0/258337576 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1928032810 con 0x7f192c15a5d0 2024-04-24T23:35:19.283 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.251+0000 7f1933d1b640 1 -- 172.21.15.157:0/258337576 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f192c15a5d0 msgr2=0x7f192c0c95b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.284 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.251+0000 7f1933d1b640 1 --2- 172.21.15.157:0/258337576 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f192c15a5d0 0x7f192c0c95b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f192c0ad8a0 tx=0x7f192802a210 comp rx=0 tx=0).stop 2024-04-24T23:35:19.284 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.251+0000 7fa45bfff640 1 -- 172.21.15.157:0/3318805823 --> v1:172.21.15.169:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa45c0d67f0 con 0x7fa45c0d3d80 2024-04-24T23:35:19.284 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.251+0000 7fd0675ce640 1 --2- 172.21.15.157:0/1621178143 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd04c0051c0 0x7fd04c00b9c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.284 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.251+0000 7fd0675ce640 1 -- 172.21.15.157:0/1621178143 >> 172.21.15.157:0/1621178143 conn(0x7fd0600a8690 msgr2=0x7fd0600a8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:19.285 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.251+0000 7fa45bfff640 1 -- 172.21.15.157:0/3318805823 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1480525340 0 0) 0x7fa46404fd50 con 0x7fa45c0d3d80 2024-04-24T23:35:19.285 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.251+0000 7fcc267fc640 1 -- 172.21.15.157:0/3272192236 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c037070 con 0x7fcc28156c90 2024-04-24T23:35:19.286 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.251+0000 7f1933d1b640 1 -- 172.21.15.157:0/258337576 shutdown_connections 2024-04-24T23:35:19.286 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.251+0000 7f1933d1b640 1 --2- 172.21.15.157:0/258337576 >> v2:172.21.15.157:3300/0 conn(0x7f192c0c6800 0x7f192c0c6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.286 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.251+0000 7f1933d1b640 1 --2- 172.21.15.157:0/258337576 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f192c0c58a0 0x7f192c0c9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.286 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.251+0000 7f1933d1b640 1 --2- 172.21.15.157:0/258337576 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f192c15a5d0 0x7f192c0c95b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.286 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.251+0000 7f1933d1b640 1 -- 172.21.15.157:0/258337576 >> 172.21.15.157:0/258337576 conn(0x7f192c0caca0 msgr2=0x7f192c0cb0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:19.286 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.251+0000 7fcc267fc640 1 -- 172.21.15.157:0/3272192236 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc1c032440 con 0x7fcc28156c90 2024-04-24T23:35:19.286 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.251+0000 7fcc267fc640 1 -- 172.21.15.157:0/3272192236 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c032850 con 0x7fcc28156c90 2024-04-24T23:35:19.287 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.251+0000 7fa45bfff640 1 -- 172.21.15.157:0/3318805823 >> v1:172.21.15.169:6789/0 conn(0x7fa45c0d3d80 legacy=0x7fa45c0465a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.287 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.251+0000 7fd0675ce640 1 -- 172.21.15.157:0/1621178143 shutdown_connections 2024-04-24T23:35:19.287 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.251+0000 7fd0675ce640 1 -- 172.21.15.157:0/1621178143 wait complete. 2024-04-24T23:35:19.287 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.251+0000 7f4336d9d640 1 --2- 172.21.15.157:0/245912577 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43380c6e70 0x7f43380c7300 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4330009e50 tx=0x7f433002a5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88a0cb6ac694a3f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:19.288 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.251+0000 7fa45bfff640 1 --2- 172.21.15.157:0/3318805823 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4500039b0 0x7fa450003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.288 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.251+0000 7fa45bfff640 1 --2- 172.21.15.157:0/3318805823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa450004310 0x7fa450004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.288 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.251+0000 7fa45bfff640 1 --2- 172.21.15.157:0/3318805823 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa450004db0 0x7fa450005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.288 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.251+0000 7f4335d9b640 1 -- 172.21.15.157:0/245912577 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4330036070 con 0x7f43380c6e70 2024-04-24T23:35:19.289 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.251+0000 7fa45bfff640 1 -- 172.21.15.157:0/3318805823 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa45c006e00 con 0x7fa450004310 2024-04-24T23:35:19.289 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.251+0000 7fa45bfff640 1 -- 172.21.15.157:0/3318805823 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa45c0d6510 con 0x7fa450004db0 2024-04-24T23:35:19.289 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.251+0000 7f4335d9b640 1 -- 172.21.15.157:0/245912577 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4330031600 con 0x7f43380c6e70 2024-04-24T23:35:19.289 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.251+0000 7fa45bfff640 1 -- 172.21.15.157:0/3318805823 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa45000bd30 con 0x7fa4500039b0 2024-04-24T23:35:19.290 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.251+0000 7fa469a25640 1 --2- 172.21.15.157:0/3318805823 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa450004db0 0x7fa450005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.290 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.251+0000 7f947dcc7640 1 -- 172.21.15.157:0/1516007815 wait complete. 2024-04-24T23:35:19.290 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.252+0000 7f1bfdd9b640 1 --2- 172.21.15.157:0/204256346 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c00072630 0x7f1c000c2810 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1c00073730 tx=0x7f1bf402a0e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7aad0f4684d943f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:19.291 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.252+0000 7f1933d1b640 1 -- 172.21.15.157:0/258337576 shutdown_connections 2024-04-24T23:35:19.291 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.252+0000 7f1933d1b640 1 -- 172.21.15.157:0/258337576 wait complete. 2024-04-24T23:35:19.291 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.252+0000 7fcc2dc53640 1 -- 172.21.15.157:0/3272192236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc28156c90 msgr2=0x7fcc2806db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.291 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.252+0000 7fcc2dc53640 1 --2- 172.21.15.157:0/3272192236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc28156c90 0x7fcc2806db50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fcc280abf10 tx=0x7fcc1c02a250 comp rx=0 tx=0).stop 2024-04-24T23:35:19.291 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.252+0000 7fc4de8a7640 1 Processor -- start 2024-04-24T23:35:19.292 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.252+0000 7fcc2dc53640 1 -- 172.21.15.157:0/3272192236 shutdown_connections 2024-04-24T23:35:19.292 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.252+0000 7fa469224640 1 --2- 172.21.15.157:0/3318805823 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4500039b0 0x7fa450003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.292 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.252+0000 7fa469224640 1 -- 172.21.15.157:0/3318805823 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa450004db0 msgr2=0x7fa450005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:19.292 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.252+0000 7fa469224640 1 --2- 172.21.15.157:0/3318805823 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa450004db0 0x7fa450005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.292 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.252+0000 7fa469224640 1 -- 172.21.15.157:0/3318805823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa450004310 msgr2=0x7fa450004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:19.292 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.252+0000 7fa469224640 1 --2- 172.21.15.157:0/3318805823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa450004310 0x7fa450004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.293 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.252+0000 7fa469224640 1 -- 172.21.15.157:0/3318805823 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa450003520 con 0x7fa4500039b0 2024-04-24T23:35:19.293 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.252+0000 7fcc2dc53640 1 --2- 172.21.15.157:0/3272192236 >> v2:172.21.15.157:3300/0 conn(0x7fcc280c57c0 0x7fcc28159580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.293 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.252+0000 7f433759e640 1 -- 172.21.15.157:0/245912577 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43380c6e70 msgr2=0x7f43380c7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.293 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.252+0000 7f4335d9b640 1 -- 172.21.15.157:0/245912577 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43300319d0 con 0x7f43380c6e70 2024-04-24T23:35:19.293 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.252+0000 7f433759e640 1 --2- 172.21.15.157:0/245912577 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43380c6e70 0x7f43380c7300 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4330009e50 tx=0x7f433002a5b0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.294 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.252+0000 7fc4de8a7640 1 -- start start 2024-04-24T23:35:19.294 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.252+0000 7fcc2dc53640 1 --2- 172.21.15.157:0/3272192236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc28156c90 0x7fcc2806db50 secure :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fcc280abf10 tx=0x7fcc1c02a250 comp rx=0 tx=0).stop 2024-04-24T23:35:19.294 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.252+0000 7fcc2dc53640 1 -- 172.21.15.157:0/3272192236 >> 172.21.15.157:0/3272192236 conn(0x7fcc2806bae0 msgr2=0x7fcc280c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:19.295 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.252+0000 7f433759e640 1 -- 172.21.15.157:0/245912577 shutdown_connections 2024-04-24T23:35:19.295 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.252+0000 7f433759e640 1 --2- 172.21.15.157:0/245912577 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43380c6e70 0x7f43380c7300 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.295 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.253+0000 7f433759e640 1 --2- 172.21.15.157:0/245912577 >> v2:172.21.15.157:3300/0 conn(0x7f433806f860 0x7f43380c42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.295 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.253+0000 7f433759e640 1 -- 172.21.15.157:0/245912577 >> 172.21.15.157:0/245912577 conn(0x7f4338067080 msgr2=0x7f4338067460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:19.295 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.253+0000 7f947dcc7640 1 Processor -- start 2024-04-24T23:35:19.296 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.253+0000 7f433759e640 1 -- 172.21.15.157:0/245912577 shutdown_connections 2024-04-24T23:35:19.296 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.253+0000 7fcc2dc53640 1 -- 172.21.15.157:0/3272192236 shutdown_connections 2024-04-24T23:35:19.296 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.253+0000 7f433759e640 1 -- 172.21.15.157:0/245912577 wait complete. 2024-04-24T23:35:19.297 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.253+0000 7fa469224640 1 --2- 172.21.15.157:0/3318805823 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4500039b0 0x7fa450003d90 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa45c009db0 tx=0x7fa450005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab237fa3a2964c7f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:19.297 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.253+0000 7fcc2dc53640 1 -- 172.21.15.157:0/3272192236 wait complete. 2024-04-24T23:35:19.298 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.253+0000 7fa45bfff640 1 -- 172.21.15.157:0/3318805823 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa454036070 con 0x7fa4500039b0 2024-04-24T23:35:19.298 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.253+0000 7fa45bfff640 1 -- 172.21.15.157:0/3318805823 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa454031400 con 0x7fa4500039b0 2024-04-24T23:35:19.298 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.253+0000 7fa45bfff640 1 -- 172.21.15.157:0/3318805823 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa454009650 con 0x7fa4500039b0 2024-04-24T23:35:19.298 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.253+0000 7fc4de8a7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4d80c4c40 0x7fc4d80a8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.298 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.254+0000 7fc4de8a7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4d81572a0 0x7fc4d80a8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.299 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.254+0000 7fc4de8a7640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4d815a6f0 0x7fc4d81f2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.299 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.254+0000 7fc4de8a7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d806f520 con 0x7fc4d81572a0 2024-04-24T23:35:19.299 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.254+0000 7fc4de8a7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d815a1d0 con 0x7fc4d80c4c40 2024-04-24T23:35:19.299 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.254+0000 7fc4de8a7640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc4d806f6a0 con 0x7fc4d815a6f0 2024-04-24T23:35:19.299 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.254+0000 7f1933d1b640 1 Processor -- start 2024-04-24T23:35:19.300 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.254+0000 7f1933d1b640 1 -- start start 2024-04-24T23:35:19.300 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.254+0000 7fc4d7fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4d80c4c40 0x7fc4d80a8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.300 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.254+0000 7fc4d7fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4d80c4c40 0x7fc4d80a8910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.157:45044/0 (socket says 172.21.15.157:45044) 2024-04-24T23:35:19.300 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.254+0000 7fc4d7fff640 1 -- 172.21.15.157:0/3247641281 learned_addr learned my addr 172.21.15.157:0/3247641281 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:19.300 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.254+0000 7fc4dce44640 1 --2- 172.21.15.157:0/3247641281 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4d815a6f0 0x7fc4d81f2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.301 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.253+0000 7fd0675ce640 1 Processor -- start 2024-04-24T23:35:19.301 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.253+0000 7fd0675ce640 1 -- start start 2024-04-24T23:35:19.301 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.254+0000 7fd0675ce640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd04c0051c0 0x7fd060141ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.301 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.253+0000 7f947dcc7640 1 -- start start 2024-04-24T23:35:19.301 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.254+0000 7f947dcc7640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94700477a0 0x7f94700d06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.301 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.254+0000 7f947dcc7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f947004a7e0 0x7f9470046220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.301 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.254+0000 7f947dcc7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94700d1850 0x7f947016fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.301 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.254+0000 7f947dcc7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f947004af40 con 0x7f94700d1850 2024-04-24T23:35:19.302 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.254+0000 7f947dcc7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f94700489f0 con 0x7f947004a7e0 2024-04-24T23:35:19.302 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.254+0000 7f947dcc7640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f947000afe0 con 0x7f94700477a0 2024-04-24T23:35:19.302 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.254+0000 7fc4d77fe640 1 --2- 172.21.15.157:0/3247641281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4d81572a0 0x7fc4d80a8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.303 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.254+0000 7fd0675ce640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd06015a170 0x7fd060146c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.303 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.254+0000 7fd0675ce640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0601471c0 0x7fd060157f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.303 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.254+0000 7fcc2dc53640 1 Processor -- start 2024-04-24T23:35:19.303 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.254+0000 7fd064b69640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd06015a170 0x7fd060146c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.303 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.254+0000 7fd064b69640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd06015a170 0x7fd060146c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.157:52832/0 (socket says 172.21.15.157:52832) 2024-04-24T23:35:19.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.254+0000 7fd065b6b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0601471c0 0x7fd060157f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.254+0000 7fa46a226640 1 -- 172.21.15.157:0/3318805823 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4500039b0 msgr2=0x7fa450003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.254+0000 7fa46a226640 1 --2- 172.21.15.157:0/3318805823 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4500039b0 0x7fa450003d90 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa45c009db0 tx=0x7fa450005830 comp rx=0 tx=0).stop 2024-04-24T23:35:19.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.254+0000 7fd065b6b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0601471c0 0x7fd060157f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42814/0 (socket says 172.21.15.157:42814) 2024-04-24T23:35:19.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.254+0000 7f9477fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f947004a7e0 0x7f9470046220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.305 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.254+0000 7f9477fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f947004a7e0 0x7f9470046220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.157:45054/0 (socket says 172.21.15.157:45054) 2024-04-24T23:35:19.305 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.254+0000 7f9477fff640 1 -- 172.21.15.157:0/4181601864 learned_addr learned my addr 172.21.15.157:0/4181601864 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:19.305 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.254+0000 7fd064b69640 1 -- 172.21.15.157:0/3839321514 learned_addr learned my addr 172.21.15.157:0/3839321514 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:19.305 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.254+0000 7fd0675ce640 1 -- 172.21.15.157:0/3839321514 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd06006d440 con 0x7fd0601471c0 2024-04-24T23:35:19.305 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.254+0000 7fd0675ce640 1 -- 172.21.15.157:0/3839321514 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd06006d160 con 0x7fd04c0051c0 2024-04-24T23:35:19.305 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.254+0000 7fd0675ce640 1 -- 172.21.15.157:0/3839321514 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd0600c3160 con 0x7fd06015a170 2024-04-24T23:35:19.306 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.255+0000 7f947d4c6640 1 --2- 172.21.15.157:0/4181601864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94700d1850 0x7f947016fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.306 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.255+0000 7fcc2dc53640 1 -- start start 2024-04-24T23:35:19.306 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.254+0000 7fc4d7fff640 1 -- 172.21.15.157:0/3247641281 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4d815a6f0 msgr2=0x7fc4d81f2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.307 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.254+0000 7fc4d7fff640 1 --2- 172.21.15.157:0/3247641281 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4d815a6f0 0x7fc4d81f2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.307 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.254+0000 7fc4d7fff640 1 -- 172.21.15.157:0/3247641281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4d81572a0 msgr2=0x7fc4d80a8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.307 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.254+0000 7fc4d7fff640 1 --2- 172.21.15.157:0/3247641281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4d81572a0 0x7fc4d80a8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.307 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.254+0000 7fc4d7fff640 1 -- 172.21.15.157:0/3247641281 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4d80ab8f0 con 0x7fc4d80c4c40 2024-04-24T23:35:19.307 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.254+0000 7fc4dce44640 1 --2- 172.21.15.157:0/3247641281 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4d815a6f0 0x7fc4d81f2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:35:19.307 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.255+0000 7fc4d7fff640 1 --2- 172.21.15.157:0/3247641281 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4d80c4c40 0x7fc4d80a8910 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc4c800c430 tx=0x7fc4c800c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:19.307 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.255+0000 7fd065b6b640 1 -- 172.21.15.157:0/3839321514 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd06015a170 msgr2=0x7fd060146c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.308 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.255+0000 7f947ccc5640 1 --2- 172.21.15.157:0/4181601864 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94700477a0 0x7f94700d06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.308 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.255+0000 7fd065b6b640 1 --2- 172.21.15.157:0/3839321514 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd06015a170 0x7fd060146c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.308 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.255+0000 7fd065b6b640 1 -- 172.21.15.157:0/3839321514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd04c0051c0 msgr2=0x7fd060141ab0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-24T23:35:19.308 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.255+0000 7fd065b6b640 1 --2- 172.21.15.157:0/3839321514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd04c0051c0 0x7fd060141ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.308 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.255+0000 7fd065b6b640 1 -- 172.21.15.157:0/3839321514 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd060141ff0 con 0x7fd0601471c0 2024-04-24T23:35:19.308 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.254+0000 7f1bfd59a640 1 -- 172.21.15.157:0/204256346 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf4037070 con 0x7f1c00072630 2024-04-24T23:35:19.309 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.254+0000 7f1bfd59a640 1 -- 172.21.15.157:0/204256346 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bf402adf0 con 0x7f1c00072630 2024-04-24T23:35:19.309 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.254+0000 7f1bfd59a640 1 -- 172.21.15.157:0/204256346 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf4032660 con 0x7f1c00072630 2024-04-24T23:35:19.309 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.255+0000 7f1933d1b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f192c0c58a0 0x7f192c0aeb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.310 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.255+0000 7f1933d1b640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f192c0c6800 0x7f192c155f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.310 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.255+0000 7f1933d1b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f192c15a5d0 0x7f192c1ef6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.310 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.255+0000 7fc4d4ff9640 1 -- 172.21.15.157:0/3247641281 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c8017070 con 0x7fc4d80c4c40 2024-04-24T23:35:19.310 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.255+0000 7fc4d4ff9640 1 -- 172.21.15.157:0/3247641281 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4c8005ce0 con 0x7fc4d80c4c40 2024-04-24T23:35:19.310 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.255+0000 7fc4d4ff9640 1 -- 172.21.15.157:0/3247641281 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c80072f0 con 0x7fc4d80c4c40 2024-04-24T23:35:19.311 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.255+0000 7f1c04e12640 1 -- 172.21.15.157:0/204256346 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c00072630 msgr2=0x7f1c000c2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.311 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.255+0000 7f1c04e12640 1 --2- 172.21.15.157:0/204256346 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c00072630 0x7f1c000c2810 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1c00073730 tx=0x7f1bf402a0e0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.311 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.255+0000 7f1933d1b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f192c0c7350 con 0x7f192c15a5d0 2024-04-24T23:35:19.311 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.255+0000 7f1931ab7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f192c0c58a0 0x7f192c0aeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.311 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.255+0000 7f1933d1b640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f192c0c71d0 con 0x7f192c0c58a0 2024-04-24T23:35:19.311 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.255+0000 7f1933d1b640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f192c0c74d0 con 0x7f192c0c6800 2024-04-24T23:35:19.311 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.255+0000 7fc4de8a7640 1 -- 172.21.15.157:0/3247641281 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4d80abbd0 con 0x7fc4d80c4c40 2024-04-24T23:35:19.312 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.256+0000 7fc4de8a7640 1 -- 172.21.15.157:0/3247641281 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4d80a83c0 con 0x7fc4d80c4c40 2024-04-24T23:35:19.312 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.255+0000 7fd065b6b640 1 --2- 172.21.15.157:0/3839321514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0601471c0 0x7fd060157f60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd0600c43f0 tx=0x7fd058009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:19.312 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.255+0000 7f9477fff640 1 -- 172.21.15.157:0/4181601864 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94700477a0 msgr2=0x7f94700d06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.313 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.255+0000 7f9477fff640 1 --2- 172.21.15.157:0/4181601864 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94700477a0 0x7f94700d06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.313 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.255+0000 7f9477fff640 1 -- 172.21.15.157:0/4181601864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94700d1850 msgr2=0x7f947016fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.313 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.256+0000 7f9477fff640 1 --2- 172.21.15.157:0/4181601864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94700d1850 0x7f947016fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.313 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.256+0000 7f9477fff640 1 -- 172.21.15.157:0/4181601864 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94700446f0 con 0x7f947004a7e0 2024-04-24T23:35:19.313 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.256+0000 7f1c04e12640 1 -- 172.21.15.157:0/204256346 shutdown_connections 2024-04-24T23:35:19.313 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.256+0000 7f947d4c6640 1 --2- 172.21.15.157:0/4181601864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94700d1850 0x7f947016fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:35:19.314 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.256+0000 7f1c04e12640 1 --2- 172.21.15.157:0/204256346 >> v2:172.21.15.157:3300/0 conn(0x7f1c000c2d50 0x7f1c00152870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.314 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.256+0000 7f1c04e12640 1 --2- 172.21.15.157:0/204256346 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c00072630 0x7f1c000c2810 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.314 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.256+0000 7f1c04e12640 1 -- 172.21.15.157:0/204256346 >> 172.21.15.157:0/204256346 conn(0x7f1c000c3900 msgr2=0x7f1c000c3ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:19.314 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.256+0000 7fcc2dc53640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc280c57c0 0x7fcc281f2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.315 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.256+0000 7f1931ab7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f192c0c58a0 0x7f192c0aeb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.157:45066/0 (socket says 172.21.15.157:45066) 2024-04-24T23:35:19.315 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.256+0000 7f1931ab7640 1 -- 172.21.15.157:0/2323035963 learned_addr learned my addr 172.21.15.157:0/2323035963 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:19.315 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.256+0000 7f19322b8640 1 --2- 172.21.15.157:0/2323035963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f192c15a5d0 0x7f192c1ef6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.315 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.256+0000 7fcc2dc53640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc2815d4d0 0x7fcc2814ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.316 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.256+0000 7f1931ab7640 1 -- 172.21.15.157:0/2323035963 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f192c0c6800 msgr2=0x7f192c155f10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-24T23:35:19.316 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.256+0000 7f1931ab7640 1 --2- 172.21.15.157:0/2323035963 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f192c0c6800 0x7f192c155f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.316 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.256+0000 7f1931ab7640 1 -- 172.21.15.157:0/2323035963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f192c15a5d0 msgr2=0x7f192c1ef6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.316 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.256+0000 7f1931ab7640 1 --2- 172.21.15.157:0/2323035963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f192c15a5d0 0x7f192c1ef6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.316 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.256+0000 7f1931ab7640 1 -- 172.21.15.157:0/2323035963 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f192c1569b0 con 0x7f192c0c58a0 2024-04-24T23:35:19.317 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.256+0000 7f9477fff640 1 --2- 172.21.15.157:0/4181601864 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f947004a7e0 0x7f9470046220 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f946c00d830 tx=0x7f946c00dcf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:19.317 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.256+0000 7fcc2dc53640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc280ab3d0 0x7fcc281eef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.317 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.256+0000 7f19322b8640 1 --2- 172.21.15.157:0/2323035963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f192c15a5d0 0x7f192c1ef6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:35:19.317 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.256+0000 7fd055ffb640 1 -- 172.21.15.157:0/3839321514 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd058040070 con 0x7fd0601471c0 2024-04-24T23:35:19.318 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.256+0000 7f1c04e12640 1 -- 172.21.15.157:0/204256346 shutdown_connections 2024-04-24T23:35:19.318 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.256+0000 7f1c04e12640 1 -- 172.21.15.157:0/204256346 wait complete. 2024-04-24T23:35:19.318 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.256+0000 7fa469a25640 1 -- 172.21.15.157:0/3318805823 reap_dead start 2024-04-24T23:35:19.318 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.256+0000 7f94757fa640 1 -- 172.21.15.157:0/4181601864 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946c00b6c0 con 0x7f947004a7e0 2024-04-24T23:35:19.319 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.256+0000 7fa46a226640 1 -- 172.21.15.157:0/3318805823 shutdown_connections 2024-04-24T23:35:19.319 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.256+0000 7fa46a226640 1 -- 172.21.15.157:0/3318805823 >> 172.21.15.157:0/3318805823 conn(0x7fa45c0062e0 msgr2=0x7fa45c0066f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:19.319 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.256+0000 7fcc27fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc280ab3d0 0x7fcc281eef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.320 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.256+0000 7fcc27fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc280ab3d0 0x7fcc281eef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42846/0 (socket says 172.21.15.157:42846) 2024-04-24T23:35:19.320 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.256+0000 7f94757fa640 1 -- 172.21.15.157:0/4181601864 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f946c019070 con 0x7f947004a7e0 2024-04-24T23:35:19.320 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.256+0000 7f94757fa640 1 -- 172.21.15.157:0/4181601864 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946c00ba60 con 0x7f947004a7e0 2024-04-24T23:35:19.321 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.256+0000 7fcc26ffd640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc2815d4d0 0x7fcc2814ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.321 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.256+0000 7fcc26ffd640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc2815d4d0 0x7fcc2814ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.157:52860/0 (socket says 172.21.15.157:52860) 2024-04-24T23:35:19.321 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.256+0000 7f947dcc7640 1 -- 172.21.15.157:0/4181601864 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94700d0d70 con 0x7f947004a7e0 2024-04-24T23:35:19.321 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.256+0000 7fa46a226640 1 -- 172.21.15.157:0/3318805823 shutdown_connections 2024-04-24T23:35:19.322 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.257+0000 7f947dcc7640 1 -- 172.21.15.157:0/4181601864 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94700449d0 con 0x7f947004a7e0 2024-04-24T23:35:19.322 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.257+0000 7fcc27fff640 1 -- 172.21.15.157:0/3139754253 learned_addr learned my addr 172.21.15.157:0/3139754253 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:19.323 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.257+0000 7fcc2dc53640 1 -- 172.21.15.157:0/3139754253 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcc280c64c0 con 0x7fcc280ab3d0 2024-04-24T23:35:19.323 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.257+0000 7fcc2dc53640 1 -- 172.21.15.157:0/3139754253 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcc28159ac0 con 0x7fcc280c57c0 2024-04-24T23:35:19.323 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.257+0000 7f1931ab7640 1 --2- 172.21.15.157:0/2323035963 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f192c0c58a0 0x7f192c0aeb30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1928000c00 tx=0x7f19280046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:19.323 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.257+0000 7fcc2dc53640 1 -- 172.21.15.157:0/3139754253 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcc280c2d50 con 0x7fcc2815d4d0 2024-04-24T23:35:19.323 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.257+0000 7fc4de8a7640 1 -- 172.21.15.157:0/3247641281 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4d80688d0 con 0x7fc4d80c4c40 2024-04-24T23:35:19.324 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.257+0000 7fa46a226640 1 -- 172.21.15.157:0/3318805823 wait complete. 2024-04-24T23:35:19.324 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.256+0000 7fd055ffb640 1 -- 172.21.15.157:0/3839321514 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd05803c470 con 0x7fd0601471c0 2024-04-24T23:35:19.324 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.256+0000 7fd055ffb640 1 -- 172.21.15.157:0/3839321514 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd058035030 con 0x7fd0601471c0 2024-04-24T23:35:19.324 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.256+0000 7fd0675ce640 1 -- 172.21.15.157:0/3839321514 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd060145320 con 0x7fd0601471c0 2024-04-24T23:35:19.324 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.257+0000 7fd0675ce640 1 -- 172.21.15.157:0/3839321514 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd060145040 con 0x7fd0601471c0 2024-04-24T23:35:19.325 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.257+0000 7fcc27fff640 1 -- 172.21.15.157:0/3139754253 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc2815d4d0 msgr2=0x7fcc2814ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.325 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.257+0000 7fcc27fff640 1 --2- 172.21.15.157:0/3139754253 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc2815d4d0 0x7fcc2814ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.325 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.257+0000 7fcc27fff640 1 -- 172.21.15.157:0/3139754253 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc280c57c0 msgr2=0x7fcc281f2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.325 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.257+0000 7fcc27fff640 1 --2- 172.21.15.157:0/3139754253 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc280c57c0 0x7fcc281f2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.325 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.257+0000 7fcc27fff640 1 -- 172.21.15.157:0/3139754253 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc28157a90 con 0x7fcc280ab3d0 2024-04-24T23:35:19.326 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.256+0000 7f433759e640 1 Processor -- start 2024-04-24T23:35:19.326 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.257+0000 7f19227fc640 1 -- 172.21.15.157:0/2323035963 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1928041020 con 0x7f192c0c58a0 2024-04-24T23:35:19.326 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.258+0000 7fa46a226640 1 Processor -- start 2024-04-24T23:35:19.327 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.257+0000 7f19227fc640 1 -- 172.21.15.157:0/2323035963 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19280309b0 con 0x7f192c0c58a0 2024-04-24T23:35:19.327 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.258+0000 7f947dcc7640 1 -- 172.21.15.157:0/4181601864 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94700057e0 con 0x7f947004a7e0 2024-04-24T23:35:19.327 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.258+0000 7f19227fc640 1 -- 172.21.15.157:0/2323035963 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1928037040 con 0x7f192c0c58a0 2024-04-24T23:35:19.328 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.258+0000 7f1c04e12640 1 Processor -- start 2024-04-24T23:35:19.328 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.259+0000 7f1c04e12640 1 -- start start 2024-04-24T23:35:19.328 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.259+0000 7f433759e640 1 -- start start 2024-04-24T23:35:19.329 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.259+0000 7f1933d1b640 1 -- 172.21.15.157:0/2323035963 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f192c0c47d0 con 0x7f192c0c58a0 2024-04-24T23:35:19.329 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.259+0000 7f1933d1b640 1 -- 172.21.15.157:0/2323035963 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f192c0c3840 con 0x7f192c0c58a0 2024-04-24T23:35:19.329 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.258+0000 7fc4d4ff9640 1 -- 172.21.15.157:0/3247641281 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc4c801a070 con 0x7fc4d80c4c40 2024-04-24T23:35:19.329 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.259+0000 7fcc27fff640 1 --2- 172.21.15.157:0/3139754253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc280ab3d0 0x7fcc281eef60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcc2000b970 tx=0x7fcc2000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:19.330 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.259+0000 7fd0675ce640 1 -- 172.21.15.157:0/3839321514 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0600681c0 con 0x7fd0601471c0 2024-04-24T23:35:19.330 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.258+0000 7fa46a226640 1 -- start start 2024-04-24T23:35:19.330 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.259+0000 7fcc13fff640 1 -- 172.21.15.157:0/3139754253 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc20019070 con 0x7fcc280ab3d0 2024-04-24T23:35:19.331 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.259+0000 7fa46a226640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa45c02c610 0x7fa45c0d5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.331 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.259+0000 7fa46a226640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa45c042cf0 0x7fa45c02c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.331 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.260+0000 7fa46a226640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa45c179990 0x7fa45c179e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.331 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.259+0000 7fcc13fff640 1 -- 172.21.15.157:0/3139754253 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc2000cb40 con 0x7fcc280ab3d0 2024-04-24T23:35:19.332 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.259+0000 7fc4d4ff9640 1 --2- 172.21.15.157:0/3247641281 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fc4c406bf00 0x7fc4c406e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.332 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.259+0000 7fc4d4ff9640 1 -- 172.21.15.157:0/3247641281 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc4c808a4e0 con 0x7fc4d80c4c40 2024-04-24T23:35:19.332 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.260+0000 7fcc13fff640 1 -- 172.21.15.157:0/3139754253 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc20014440 con 0x7fcc280ab3d0 2024-04-24T23:35:19.333 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.260+0000 7f433759e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f433806f860 0x7f43381dd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.333 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.260+0000 7f1c04e12640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c00072630 0x7f1c001e7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.333 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.260+0000 7f1c04e12640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c000c2d50 0x7f1c001ea9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.333 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.260+0000 7fa46a226640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa45c009be0 con 0x7fa45c179990 2024-04-24T23:35:19.334 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.260+0000 7fa46a226640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa45c009a60 con 0x7fa45c02c610 2024-04-24T23:35:19.334 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.260+0000 7fa46a226640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa45c02e4d0 con 0x7fa45c042cf0 2024-04-24T23:35:19.334 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.260+0000 7f1c04e12640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c0014e990 0x7f1c0006d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.334 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.259+0000 7f94757fa640 1 -- 172.21.15.157:0/4181601864 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f946c01e460 con 0x7f947004a7e0 2024-04-24T23:35:19.334 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.260+0000 7f94757fa640 1 --2- 172.21.15.157:0/4181601864 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f945c06bf30 0x7f945c06e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.334 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.260+0000 7f94757fa640 1 -- 172.21.15.157:0/4181601864 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f946c08b6e0 con 0x7f947004a7e0 2024-04-24T23:35:19.335 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.260+0000 7fc4d77fe640 1 --2- 172.21.15.157:0/3247641281 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fc4c406bf00 0x7fc4c406e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.335 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.260+0000 7f1bfdd9b640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c000c2d50 0x7f1c001ea9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.335 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.260+0000 7fa469224640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa45c02c610 0x7fa45c0d5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.336 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.260+0000 7f1bfdd9b640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c000c2d50 0x7f1c001ea9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.157:52878/0 (socket says 172.21.15.157:52878) 2024-04-24T23:35:19.336 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.260+0000 7f947ccc5640 1 --2- 172.21.15.157:0/4181601864 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f945c06bf30 0x7f945c06e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.336 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.260+0000 7fa468a23640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa45c042cf0 0x7fa45c02c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.337 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.260+0000 7fa469a25640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa45c179990 0x7fa45c179e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.337 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.260+0000 7f1bfe59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c00072630 0x7f1c001e7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.337 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.260+0000 7f1bfe59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c00072630 0x7f1c001e7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42872/0 (socket says 172.21.15.157:42872) 2024-04-24T23:35:19.337 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.260+0000 7fa469a25640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa45c179990 0x7fa45c179e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42882/0 (socket says 172.21.15.157:42882) 2024-04-24T23:35:19.337 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.260+0000 7fa469a25640 1 -- 172.21.15.157:0/1839507841 learned_addr learned my addr 172.21.15.157:0/1839507841 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:19.337 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.260+0000 7fa469224640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa45c02c610 0x7fa45c0d5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.157:45072/0 (socket says 172.21.15.157:45072) 2024-04-24T23:35:19.339 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.260+0000 7f1bfed9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c0014e990 0x7f1c0006d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.339 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.261+0000 7f1bfed9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c0014e990 0x7f1c0006d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.157:45088/0 (socket says 172.21.15.157:45088) 2024-04-24T23:35:19.339 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.261+0000 7f1bfdd9b640 1 -- 172.21.15.157:0/1553882363 learned_addr learned my addr 172.21.15.157:0/1553882363 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:19.339 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.261+0000 7f1c04e12640 1 -- 172.21.15.157:0/1553882363 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1c000681c0 con 0x7f1c00072630 2024-04-24T23:35:19.339 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.261+0000 7f1c04e12640 1 -- 172.21.15.157:0/1553882363 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0014ed70 con 0x7f1c0014e990 2024-04-24T23:35:19.339 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.261+0000 7f1c04e12640 1 -- 172.21.15.157:0/1553882363 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1c00068340 con 0x7f1c000c2d50 2024-04-24T23:35:19.339 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.260+0000 7f1933d1b640 1 -- 172.21.15.157:0/2323035963 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f192c0681c0 con 0x7f192c0c58a0 2024-04-24T23:35:19.340 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.261+0000 7fd055ffb640 1 -- 172.21.15.157:0/3839321514 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd058043070 con 0x7fd0601471c0 2024-04-24T23:35:19.340 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.261+0000 7fd055ffb640 1 --2- 172.21.15.157:0/3839321514 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fd04406bf00 0x7fd04406e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.340 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.261+0000 7fd055ffb640 1 -- 172.21.15.157:0/3839321514 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd0580ac150 con 0x7fd0601471c0 2024-04-24T23:35:19.340 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.261+0000 7f433759e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43380c6e70 0x7f433814c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.340 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.261+0000 7f433759e640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4338157370 0x7f433814ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.340 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.261+0000 7f433759e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f43380c7840 con 0x7f433806f860 2024-04-24T23:35:19.340 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.260+0000 7fcc2dc53640 1 -- 172.21.15.157:0/3139754253 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc28157d70 con 0x7fcc280ab3d0 2024-04-24T23:35:19.341 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.260+0000 7fcc2dc53640 1 -- 172.21.15.157:0/3139754253 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc281f2a00 con 0x7fcc280ab3d0 2024-04-24T23:35:19.341 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.261+0000 7fc4d77fe640 1 --2- 172.21.15.157:0/3247641281 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fc4c406bf00 0x7fc4c406e3c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc4d80aa7c0 tx=0x7fc4cc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:19.341 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.261+0000 7fd06536a640 1 --2- 172.21.15.157:0/3839321514 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fd04406bf00 0x7fd04406e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.342 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.261+0000 7f1bfdd9b640 1 -- 172.21.15.157:0/1553882363 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c0014e990 msgr2=0x7f1c0006d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.342 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.261+0000 7f1bfdd9b640 1 --2- 172.21.15.157:0/1553882363 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c0014e990 0x7f1c0006d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.342 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.261+0000 7f1bfdd9b640 1 -- 172.21.15.157:0/1553882363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c00072630 msgr2=0x7f1c001e7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.342 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.261+0000 7f1bfdd9b640 1 --2- 172.21.15.157:0/1553882363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c00072630 0x7f1c001e7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.342 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.261+0000 7f1bfdd9b640 1 -- 172.21.15.157:0/1553882363 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c001e8450 con 0x7f1c000c2d50 2024-04-24T23:35:19.342 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.261+0000 7fa469224640 1 -- 172.21.15.157:0/1839507841 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa45c042cf0 msgr2=0x7fa45c02c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.342 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.261+0000 7fa469224640 1 --2- 172.21.15.157:0/1839507841 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa45c042cf0 0x7fa45c02c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.343 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.261+0000 7fa469224640 1 -- 172.21.15.157:0/1839507841 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa45c179990 msgr2=0x7fa45c179e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.343 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.261+0000 7fa469224640 1 --2- 172.21.15.157:0/1839507841 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa45c179990 0x7fa45c179e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.343 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.261+0000 7fa469224640 1 -- 172.21.15.157:0/1839507841 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa45c17d3c0 con 0x7fa45c02c610 2024-04-24T23:35:19.343 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.261+0000 7fa469224640 1 --2- 172.21.15.157:0/1839507841 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa45c02c610 0x7fa45c0d5f40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa454031000 tx=0x7fa454004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:19.343 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.262+0000 7f1bfed9d640 1 --2- 172.21.15.157:0/1553882363 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c0014e990 0x7f1c0006d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:35:19.343 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.262+0000 7fcc2dc53640 1 -- 172.21.15.157:0/3139754253 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc280a9380 con 0x7fcc280ab3d0 2024-04-24T23:35:19.344 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.262+0000 7f1bfdd9b640 1 --2- 172.21.15.157:0/1553882363 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c000c2d50 0x7f1c001ea9c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1bf4035040 tx=0x7f1bf40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:19.344 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.262+0000 7f1bfe59c640 1 --2- 172.21.15.157:0/1553882363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c00072630 0x7f1c001e7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:35:19.345 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.261+0000 7f432ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43380c6e70 0x7f433814c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.345 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.261+0000 7f432ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43380c6e70 0x7f433814c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.157:45094/0 (socket says 172.21.15.157:45094) 2024-04-24T23:35:19.345 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.261+0000 7f432ffff640 1 -- 172.21.15.157:0/2936089590 learned_addr learned my addr 172.21.15.157:0/2936089590 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:19.345 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.262+0000 7f433659c640 1 --2- 172.21.15.157:0/2936089590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f433806f860 0x7f43381dd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.345 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.262+0000 7f433759e640 1 -- 172.21.15.157:0/2936089590 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f43380c6950 con 0x7f43380c6e70 2024-04-24T23:35:19.345 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.262+0000 7f433759e640 1 -- 172.21.15.157:0/2936089590 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f43380c9460 con 0x7f4338157370 2024-04-24T23:35:19.345 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.262+0000 7f4336d9d640 1 --2- 172.21.15.157:0/2936089590 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4338157370 0x7f433814ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.345 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.262+0000 7f947ccc5640 1 --2- 172.21.15.157:0/4181601864 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f945c06bf30 0x7f945c06e3f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9464034000 tx=0x7f946402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:19.346 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.262+0000 7f1beeffd640 1 -- 172.21.15.157:0/1553882363 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf4041070 con 0x7f1c000c2d50 2024-04-24T23:35:19.346 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.262+0000 7f1beeffd640 1 -- 172.21.15.157:0/1553882363 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bf403c670 con 0x7f1c000c2d50 2024-04-24T23:35:19.346 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.262+0000 7f1beeffd640 1 -- 172.21.15.157:0/1553882363 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf4037040 con 0x7f1c000c2d50 2024-04-24T23:35:19.347 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.262+0000 7f432ffff640 1 -- 172.21.15.157:0/2936089590 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4338157370 msgr2=0x7f433814ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.347 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.262+0000 7f432ffff640 1 --2- 172.21.15.157:0/2936089590 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4338157370 0x7f433814ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.347 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.262+0000 7f1c04e12640 1 -- 172.21.15.157:0/1553882363 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c001eaf00 con 0x7f1c000c2d50 2024-04-24T23:35:19.347 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.262+0000 7f432ffff640 1 -- 172.21.15.157:0/2936089590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f433806f860 msgr2=0x7f43381dd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.348 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.262+0000 7f432ffff640 1 --2- 172.21.15.157:0/2936089590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f433806f860 0x7f43381dd9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.348 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.262+0000 7fd06536a640 1 --2- 172.21.15.157:0/3839321514 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fd04406bf00 0x7fd04406e3c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd05000b790 tx=0x7fd050008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:19.348 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.262+0000 7f432ffff640 1 -- 172.21.15.157:0/2936089590 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43381de260 con 0x7f43380c6e70 2024-04-24T23:35:19.348 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.262+0000 7f432ffff640 1 --2- 172.21.15.157:0/2936089590 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43380c6e70 0x7f433814c8f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4320010b20 tx=0x7f432000e050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:19.349 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.263+0000 7fcc13fff640 1 -- 172.21.15.157:0/3139754253 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcc2001c070 con 0x7fcc280ab3d0 2024-04-24T23:35:19.349 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.263+0000 7f1c04e12640 1 -- 172.21.15.157:0/1553882363 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c000c67d0 con 0x7f1c000c2d50 2024-04-24T23:35:19.350 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.263+0000 7fcc13fff640 1 --2- 172.21.15.157:0/3139754253 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fcc0c06be80 0x7fcc0c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.350 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.263+0000 7fcc13fff640 1 -- 172.21.15.157:0/3139754253 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fcc2008ad90 con 0x7fcc280ab3d0 2024-04-24T23:35:19.350 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.263+0000 7fcc277fe640 1 --2- 172.21.15.157:0/3139754253 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fcc0c06be80 0x7fcc0c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.351 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.264+0000 7fcc277fe640 1 --2- 172.21.15.157:0/3139754253 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fcc0c06be80 0x7fcc0c06e340 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c034000 tx=0x7fcc1c02c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:19.351 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.264+0000 7f1c04e12640 1 -- 172.21.15.157:0/1553882363 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c000688d0 con 0x7f1c000c2d50 2024-04-24T23:35:19.352 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.264+0000 7fc4d4ff9640 1 -- 172.21.15.157:0/3247641281 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc4c8058570 con 0x7fc4d80c4c40 2024-04-24T23:35:19.352 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.264+0000 7fa459ffb640 1 -- 172.21.15.157:0/1839507841 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa454041020 con 0x7fa45c02c610 2024-04-24T23:35:19.353 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.264+0000 7f432effd640 1 -- 172.21.15.157:0/2936089590 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432000ec90 con 0x7f43380c6e70 2024-04-24T23:35:19.353 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.264+0000 7fa46a226640 1 -- 172.21.15.157:0/1839507841 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa45c044e60 con 0x7fa45c02c610 2024-04-24T23:35:19.353 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.264+0000 7f433759e640 1 -- 172.21.15.157:0/2936089590 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43380a87c0 con 0x7f43380c6e70 2024-04-24T23:35:19.354 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.264+0000 7f433759e640 1 -- 172.21.15.157:0/2936089590 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4338152460 con 0x7f43380c6e70 2024-04-24T23:35:19.354 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.265+0000 7fa46a226640 1 -- 172.21.15.157:0/1839507841 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa45c0454b0 con 0x7fa45c02c610 2024-04-24T23:35:19.355 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.265+0000 7f1beeffd640 1 -- 172.21.15.157:0/1553882363 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1bf403a070 con 0x7f1c000c2d50 2024-04-24T23:35:19.355 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.265+0000 7f19227fc640 1 -- 172.21.15.157:0/2323035963 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1928044020 con 0x7f192c0c58a0 2024-04-24T23:35:19.356 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.265+0000 7f1beeffd640 1 --2- 172.21.15.157:0/1553882363 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f1be006be30 0x7f1be006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.356 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.266+0000 7f1beeffd640 1 -- 172.21.15.157:0/1553882363 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f1bf40ab690 con 0x7f1c000c2d50 2024-04-24T23:35:19.356 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.266+0000 7f19227fc640 1 --2- 172.21.15.157:0/2323035963 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f191006be80 0x7f191006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.356 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.266+0000 7f433759e640 1 -- 172.21.15.157:0/2936089590 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4338156f20 con 0x7f43380c6e70 2024-04-24T23:35:19.357 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.266+0000 7f1bfe59c640 1 --2- 172.21.15.157:0/1553882363 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f1be006be30 0x7f1be006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.357 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.266+0000 7fa46a226640 1 -- 172.21.15.157:0/1839507841 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa45c0d8590 con 0x7fa45c02c610 2024-04-24T23:35:19.357 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.266+0000 7f19312b6640 1 --2- 172.21.15.157:0/2323035963 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f191006be80 0x7f191006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.358 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.266+0000 7f1bfe59c640 1 --2- 172.21.15.157:0/1553882363 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f1be006be30 0x7f1be006e2f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8004600 tx=0x7f1bf8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:19.358 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.267+0000 7f19312b6640 1 --2- 172.21.15.157:0/2323035963 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f191006be80 0x7f191006e340 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f192c06b780 tx=0x7f191c006b30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:19.359 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.268+0000 7fd055ffb640 1 -- 172.21.15.157:0/3839321514 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd05807a1e0 con 0x7fd0601471c0 2024-04-24T23:35:19.359 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.269+0000 7f19227fc640 1 -- 172.21.15.157:0/2323035963 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f1928078080 con 0x7f192c0c58a0 2024-04-24T23:35:19.360 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.269+0000 7fa459ffb640 1 -- 172.21.15.157:0/1839507841 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa454009710 con 0x7fa45c02c610 2024-04-24T23:35:19.360 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.269+0000 7fa459ffb640 1 -- 172.21.15.157:0/1839507841 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa454036040 con 0x7fa45c02c610 2024-04-24T23:35:19.361 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.269+0000 7f432effd640 1 -- 172.21.15.157:0/2936089590 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f432001d070 con 0x7f43380c6e70 2024-04-24T23:35:19.361 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.269+0000 7f432effd640 1 -- 172.21.15.157:0/2936089590 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4320005bb0 con 0x7f43380c6e70 2024-04-24T23:35:19.361 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.269+0000 7f94757fa640 1 -- 172.21.15.157:0/4181601864 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f946c059dc0 con 0x7f947004a7e0 2024-04-24T23:35:19.361 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.270+0000 7f1beeffd640 1 -- 172.21.15.157:0/1553882363 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1bf4079d90 con 0x7f1c000c2d50 2024-04-24T23:35:19.362 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.274+0000 7f19227fc640 1 -- 172.21.15.157:0/2323035963 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f192804b030 con 0x7f192c0c58a0 2024-04-24T23:35:19.362 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.273+0000 7fcc13fff640 1 -- 172.21.15.157:0/3139754253 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcc280a9380 con 0x7fcc280ab3d0 2024-04-24T23:35:19.363 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.276+0000 7f432effd640 1 -- 172.21.15.157:0/2936089590 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4320022460 con 0x7f43380c6e70 2024-04-24T23:35:19.363 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.276+0000 7f432effd640 1 --2- 172.21.15.157:0/2936089590 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f431806be30 0x7f431806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.363 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.277+0000 7f432effd640 1 -- 172.21.15.157:0/2936089590 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f4320090430 con 0x7f43380c6e70 2024-04-24T23:35:19.364 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.277+0000 7fa459ffb640 1 -- 172.21.15.157:0/1839507841 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa454044020 con 0x7fa45c02c610 2024-04-24T23:35:19.364 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.277+0000 7f433659c640 1 --2- 172.21.15.157:0/2936089590 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f431806be30 0x7f431806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.364 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.277+0000 7fa459ffb640 1 --2- 172.21.15.157:0/1839507841 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fa44806be30 0x7fa44806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:19.365 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.278+0000 7f433659c640 1 --2- 172.21.15.157:0/2936089590 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f431806be30 0x7f431806e2f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4328006fd0 tx=0x7f4328009060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:19.365 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.278+0000 7fa459ffb640 1 -- 172.21.15.157:0/1839507841 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa4540abcc0 con 0x7fa45c02c610 2024-04-24T23:35:19.366 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.278+0000 7fa468a23640 1 --2- 172.21.15.157:0/1839507841 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fa44806be30 0x7fa44806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:19.366 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.278+0000 7fa468a23640 1 --2- 172.21.15.157:0/1839507841 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fa44806be30 0x7fa44806e2f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa45c00c140 tx=0x7fa460009590 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:19.366 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.279+0000 7fa459ffb640 1 -- 172.21.15.157:0/1839507841 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa45404b070 con 0x7fa45c02c610 2024-04-24T23:35:19.366 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.283+0000 7f432effd640 1 -- 172.21.15.157:0/2936089590 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4338156f20 con 0x7f43380c6e70 2024-04-24T23:35:19.673 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.672+0000 7fc4de8a7640 1 -- 172.21.15.157:0/3247641281 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fc4d80c4a20 con 0x7fc4d80c4c40 2024-04-24T23:35:19.673 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.673+0000 7fc4d4ff9640 1 -- 172.21.15.157:0/3247641281 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc4c805c870 con 0x7fc4d80c4c40 2024-04-24T23:35:19.673 INFO:teuthology.orchestra.run.smithi157.stdout:60129542150 2024-04-24T23:35:19.677 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.676+0000 7fc4be7fc640 1 -- 172.21.15.157:0/3247641281 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fc4c406bf00 msgr2=0x7fc4c406e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.677 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.676+0000 7fc4be7fc640 1 --2- 172.21.15.157:0/3247641281 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fc4c406bf00 0x7fc4c406e3c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc4d80aa7c0 tx=0x7fc4cc008040 comp rx=0 tx=0).stop 2024-04-24T23:35:19.677 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.676+0000 7fc4be7fc640 1 -- 172.21.15.157:0/3247641281 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4d80c4c40 msgr2=0x7fc4d80a8910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.678 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.676+0000 7fc4be7fc640 1 --2- 172.21.15.157:0/3247641281 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4d80c4c40 0x7fc4d80a8910 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc4c800c430 tx=0x7fc4c800c8f0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.678 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.677+0000 7fc4be7fc640 1 -- 172.21.15.157:0/3247641281 shutdown_connections 2024-04-24T23:35:19.678 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.677+0000 7fc4be7fc640 1 --2- 172.21.15.157:0/3247641281 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fc4c406bf00 0x7fc4c406e3c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.678 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.677+0000 7fc4be7fc640 1 --2- 172.21.15.157:0/3247641281 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4d815a6f0 0x7fc4d81f2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.678 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.677+0000 7fc4be7fc640 1 --2- 172.21.15.157:0/3247641281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4d81572a0 0x7fc4d80a8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.678 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.677+0000 7fc4be7fc640 1 --2- 172.21.15.157:0/3247641281 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4d80c4c40 0x7fc4d80a8910 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.678 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.677+0000 7fc4be7fc640 1 -- 172.21.15.157:0/3247641281 >> 172.21.15.157:0/3247641281 conn(0x7fc4d806de00 msgr2=0x7fc4d80c3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:19.678 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.677+0000 7fc4be7fc640 1 -- 172.21.15.157:0/3247641281 shutdown_connections 2024-04-24T23:35:19.679 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.677+0000 7fc4be7fc640 1 -- 172.21.15.157:0/3247641281 wait complete. 2024-04-24T23:35:19.685 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.684+0000 7f1c04e12640 1 -- 172.21.15.157:0/1553882363 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f1c0006eb70 con 0x7f1c000c2d50 2024-04-24T23:35:19.685 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.685+0000 7f1beeffd640 1 -- 172.21.15.157:0/1553882363 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1bf4079690 con 0x7f1c000c2d50 2024-04-24T23:35:19.686 INFO:teuthology.orchestra.run.smithi157.stdout:60129542150 2024-04-24T23:35:19.688 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.688+0000 7fcc2dc53640 1 -- 172.21.15.157:0/3139754253 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fcc2804f500 con 0x7fcc280ab3d0 2024-04-24T23:35:19.689 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.688+0000 7f1becff9640 1 -- 172.21.15.157:0/1553882363 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f1be006be30 msgr2=0x7f1be006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.689 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.688+0000 7f1becff9640 1 --2- 172.21.15.157:0/1553882363 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f1be006be30 0x7f1be006e2f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8004600 tx=0x7f1bf8009040 comp rx=0 tx=0).stop 2024-04-24T23:35:19.689 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.688+0000 7fcc13fff640 1 -- 172.21.15.157:0/3139754253 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fcc20059570 con 0x7fcc280ab3d0 2024-04-24T23:35:19.690 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.688+0000 7f1becff9640 1 -- 172.21.15.157:0/1553882363 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c000c2d50 msgr2=0x7f1c001ea9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.690 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.689+0000 7f1becff9640 1 --2- 172.21.15.157:0/1553882363 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c000c2d50 0x7f1c001ea9c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1bf4035040 tx=0x7f1bf40046c0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.690 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.689+0000 7f1becff9640 1 -- 172.21.15.157:0/1553882363 shutdown_connections 2024-04-24T23:35:19.690 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.689+0000 7f1becff9640 1 --2- 172.21.15.157:0/1553882363 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f1be006be30 0x7f1be006e2f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.690 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.689+0000 7f1becff9640 1 --2- 172.21.15.157:0/1553882363 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c0014e990 0x7f1c0006d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.690 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.689+0000 7f1becff9640 1 --2- 172.21.15.157:0/1553882363 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c000c2d50 0x7f1c001ea9c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.690 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.689+0000 7f1becff9640 1 --2- 172.21.15.157:0/1553882363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c00072630 0x7f1c001e7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.690 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.689+0000 7f1becff9640 1 -- 172.21.15.157:0/1553882363 >> 172.21.15.157:0/1553882363 conn(0x7f1c000c3900 msgr2=0x7f1c00148410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:19.691 INFO:teuthology.orchestra.run.smithi157.stdout:55834574856 2024-04-24T23:35:19.691 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.689+0000 7f1becff9640 1 -- 172.21.15.157:0/1553882363 shutdown_connections 2024-04-24T23:35:19.691 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.689+0000 7f1becff9640 1 -- 172.21.15.157:0/1553882363 wait complete. 2024-04-24T23:35:19.691 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.690+0000 7f947dcc7640 1 -- 172.21.15.157:0/4181601864 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f9470049f50 con 0x7f947004a7e0 2024-04-24T23:35:19.691 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.691+0000 7f94757fa640 1 -- 172.21.15.157:0/4181601864 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f946c0596c0 con 0x7f947004a7e0 2024-04-24T23:35:19.692 INFO:teuthology.orchestra.run.smithi157.stdout:60129542150 2024-04-24T23:35:19.693 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.692+0000 7fcc11ffb640 1 -- 172.21.15.157:0/3139754253 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fcc0c06be80 msgr2=0x7fcc0c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.693 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.692+0000 7fcc11ffb640 1 --2- 172.21.15.157:0/3139754253 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fcc0c06be80 0x7fcc0c06e340 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c034000 tx=0x7fcc1c02c000 comp rx=0 tx=0).stop 2024-04-24T23:35:19.693 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.692+0000 7fcc11ffb640 1 -- 172.21.15.157:0/3139754253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc280ab3d0 msgr2=0x7fcc281eef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.693 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.692+0000 7fcc11ffb640 1 --2- 172.21.15.157:0/3139754253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc280ab3d0 0x7fcc281eef60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcc2000b970 tx=0x7fcc2000be30 comp rx=0 tx=0).stop 2024-04-24T23:35:19.693 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.693+0000 7fd0675ce640 1 -- 172.21.15.157:0/3839321514 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fd06006caf0 con 0x7fd0601471c0 2024-04-24T23:35:19.694 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.693+0000 7fcc11ffb640 1 -- 172.21.15.157:0/3139754253 shutdown_connections 2024-04-24T23:35:19.694 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.693+0000 7fcc11ffb640 1 --2- 172.21.15.157:0/3139754253 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fcc0c06be80 0x7fcc0c06e340 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.694 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.693+0000 7fcc11ffb640 1 --2- 172.21.15.157:0/3139754253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc280ab3d0 0x7fcc281eef60 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.694 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.693+0000 7fcc11ffb640 1 --2- 172.21.15.157:0/3139754253 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc2815d4d0 0x7fcc2814ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.694 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.693+0000 7fcc11ffb640 1 --2- 172.21.15.157:0/3139754253 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc280c57c0 0x7fcc281f2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.694 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.693+0000 7fcc11ffb640 1 -- 172.21.15.157:0/3139754253 >> 172.21.15.157:0/3139754253 conn(0x7fcc2806bae0 msgr2=0x7fcc280c2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:19.695 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.693+0000 7fd055ffb640 1 -- 172.21.15.157:0/3839321514 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd05807e4e0 con 0x7fd0601471c0 2024-04-24T23:35:19.695 INFO:teuthology.orchestra.run.smithi157.stdout:60129542150 2024-04-24T23:35:19.696 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542150 for osd.7 2024-04-24T23:35:19.696 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.694+0000 7fcc11ffb640 1 -- 172.21.15.157:0/3139754253 shutdown_connections 2024-04-24T23:35:19.696 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.694+0000 7fcc11ffb640 1 -- 172.21.15.157:0/3139754253 wait complete. 2024-04-24T23:35:19.698 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.694+0000 7f947dcc7640 1 -- 172.21.15.157:0/4181601864 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f945c06bf30 msgr2=0x7f945c06e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.698 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.694+0000 7f947dcc7640 1 --2- 172.21.15.157:0/4181601864 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f945c06bf30 0x7f945c06e3f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9464034000 tx=0x7f946402e000 comp rx=0 tx=0).stop 2024-04-24T23:35:19.698 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.694+0000 7f947dcc7640 1 -- 172.21.15.157:0/4181601864 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f947004a7e0 msgr2=0x7f9470046220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.698 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.694+0000 7f947dcc7640 1 --2- 172.21.15.157:0/4181601864 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f947004a7e0 0x7f9470046220 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f946c00d830 tx=0x7f946c00dcf0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.698 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.694+0000 7f947dcc7640 1 -- 172.21.15.157:0/4181601864 shutdown_connections 2024-04-24T23:35:19.698 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.694+0000 7f947dcc7640 1 --2- 172.21.15.157:0/4181601864 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f945c06bf30 0x7f945c06e3f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.698 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.694+0000 7f947dcc7640 1 --2- 172.21.15.157:0/4181601864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94700d1850 0x7f947016fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.698 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.694+0000 7f947dcc7640 1 --2- 172.21.15.157:0/4181601864 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f947004a7e0 0x7f9470046220 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.698 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.695+0000 7f947dcc7640 1 --2- 172.21.15.157:0/4181601864 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94700477a0 0x7f94700d06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.698 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.695+0000 7f947dcc7640 1 -- 172.21.15.157:0/4181601864 >> 172.21.15.157:0/4181601864 conn(0x7f9470049360 msgr2=0x7f947004bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:19.699 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.695+0000 7f947dcc7640 1 -- 172.21.15.157:0/4181601864 shutdown_connections 2024-04-24T23:35:19.699 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.695+0000 7f947dcc7640 1 -- 172.21.15.157:0/4181601864 wait complete. 2024-04-24T23:35:19.699 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.696+0000 7f1933d1b640 1 -- 172.21.15.157:0/2323035963 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f192c0c5680 con 0x7f192c0c58a0 2024-04-24T23:35:19.699 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.697+0000 7f19227fc640 1 -- 172.21.15.157:0/2323035963 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f192807a6e0 con 0x7f192c0c58a0 2024-04-24T23:35:19.699 INFO:teuthology.orchestra.run.smithi157.stdout:60129542150 2024-04-24T23:35:19.701 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.697+0000 7fd0675ce640 1 -- 172.21.15.157:0/3839321514 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fd04406bf00 msgr2=0x7fd04406e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.701 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.697+0000 7fd0675ce640 1 --2- 172.21.15.157:0/3839321514 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fd04406bf00 0x7fd04406e3c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd05000b790 tx=0x7fd050008040 comp rx=0 tx=0).stop 2024-04-24T23:35:19.701 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.697+0000 7fd0675ce640 1 -- 172.21.15.157:0/3839321514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0601471c0 msgr2=0x7fd060157f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.701 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.697+0000 7fd0675ce640 1 --2- 172.21.15.157:0/3839321514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0601471c0 0x7fd060157f60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd0600c43f0 tx=0x7fd058009750 comp rx=0 tx=0).stop 2024-04-24T23:35:19.701 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.697+0000 7fd0675ce640 1 -- 172.21.15.157:0/3839321514 shutdown_connections 2024-04-24T23:35:19.701 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.697+0000 7fd0675ce640 1 --2- 172.21.15.157:0/3839321514 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fd04406bf00 0x7fd04406e3c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.701 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.697+0000 7fd0675ce640 1 --2- 172.21.15.157:0/3839321514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0601471c0 0x7fd060157f60 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.701 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.697+0000 7fd0675ce640 1 --2- 172.21.15.157:0/3839321514 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd06015a170 0x7fd060146c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.701 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.697+0000 7fd0675ce640 1 --2- 172.21.15.157:0/3839321514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd04c0051c0 0x7fd060141ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.701 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.697+0000 7fd0675ce640 1 -- 172.21.15.157:0/3839321514 >> 172.21.15.157:0/3839321514 conn(0x7fd0600a8690 msgr2=0x7fd0600c5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:19.701 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.698+0000 7fd0675ce640 1 -- 172.21.15.157:0/3839321514 shutdown_connections 2024-04-24T23:35:19.702 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.698+0000 7fd0675ce640 1 -- 172.21.15.157:0/3839321514 wait complete. 2024-04-24T23:35:19.702 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.701+0000 7fa46a226640 1 -- 172.21.15.157:0/1839507841 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fa45c009330 con 0x7fa45c02c610 2024-04-24T23:35:19.702 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.700+0000 7f1933d1b640 1 -- 172.21.15.157:0/2323035963 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f191006be80 msgr2=0x7f191006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.702 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.700+0000 7f1933d1b640 1 --2- 172.21.15.157:0/2323035963 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f191006be80 0x7f191006e340 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f192c06b780 tx=0x7f191c006b30 comp rx=0 tx=0).stop 2024-04-24T23:35:19.702 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.700+0000 7f1933d1b640 1 -- 172.21.15.157:0/2323035963 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f192c0c58a0 msgr2=0x7f192c0aeb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.702 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.700+0000 7f1933d1b640 1 --2- 172.21.15.157:0/2323035963 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f192c0c58a0 0x7f192c0aeb30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1928000c00 tx=0x7f19280046c0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.702 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.701+0000 7f1933d1b640 1 -- 172.21.15.157:0/2323035963 shutdown_connections 2024-04-24T23:35:19.703 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.701+0000 7f1933d1b640 1 --2- 172.21.15.157:0/2323035963 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f191006be80 0x7f191006e340 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.703 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.701+0000 7f1933d1b640 1 --2- 172.21.15.157:0/2323035963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f192c15a5d0 0x7f192c1ef6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.703 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.701+0000 7f1933d1b640 1 --2- 172.21.15.157:0/2323035963 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f192c0c6800 0x7f192c155f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.703 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.701+0000 7f1933d1b640 1 --2- 172.21.15.157:0/2323035963 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f192c0c58a0 0x7f192c0aeb30 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.703 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.701+0000 7f1933d1b640 1 -- 172.21.15.157:0/2323035963 >> 172.21.15.157:0/2323035963 conn(0x7f192c0caca0 msgr2=0x7f192c15cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:19.703 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.701+0000 7fa459ffb640 1 -- 172.21.15.157:0/1839507841 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa4540044e0 con 0x7fa45c02c610 2024-04-24T23:35:19.703 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.701+0000 7f1933d1b640 1 -- 172.21.15.157:0/2323035963 shutdown_connections 2024-04-24T23:35:19.704 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.701+0000 7f1933d1b640 1 -- 172.21.15.157:0/2323035963 wait complete. 2024-04-24T23:35:19.704 INFO:teuthology.orchestra.run.smithi157.stdout:60129542150 2024-04-24T23:35:19.705 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.705+0000 7f433759e640 1 -- 172.21.15.157:0/2936089590 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f433804f500 con 0x7f43380c6e70 2024-04-24T23:35:19.707 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.705+0000 7fa46a226640 1 -- 172.21.15.157:0/1839507841 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fa44806be30 msgr2=0x7fa44806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.707 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.705+0000 7fa46a226640 1 --2- 172.21.15.157:0/1839507841 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fa44806be30 0x7fa44806e2f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa45c00c140 tx=0x7fa460009590 comp rx=0 tx=0).stop 2024-04-24T23:35:19.707 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.705+0000 7fa46a226640 1 -- 172.21.15.157:0/1839507841 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa45c02c610 msgr2=0x7fa45c0d5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.707 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.705+0000 7fa46a226640 1 --2- 172.21.15.157:0/1839507841 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa45c02c610 0x7fa45c0d5f40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa454031000 tx=0x7fa454004730 comp rx=0 tx=0).stop 2024-04-24T23:35:19.707 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.705+0000 7fa46a226640 1 -- 172.21.15.157:0/1839507841 shutdown_connections 2024-04-24T23:35:19.707 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.705+0000 7fa46a226640 1 --2- 172.21.15.157:0/1839507841 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fa44806be30 0x7fa44806e2f0 secure :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa45c00c140 tx=0x7fa460009590 comp rx=0 tx=0).stop 2024-04-24T23:35:19.707 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.705+0000 7fa46a226640 1 --2- 172.21.15.157:0/1839507841 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa45c179990 0x7fa45c179e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.707 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.705+0000 7fa46a226640 1 --2- 172.21.15.157:0/1839507841 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa45c042cf0 0x7fa45c02c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.707 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.705+0000 7fa46a226640 1 --2- 172.21.15.157:0/1839507841 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa45c02c610 0x7fa45c0d5f40 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.707 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.705+0000 7fa46a226640 1 -- 172.21.15.157:0/1839507841 >> 172.21.15.157:0/1839507841 conn(0x7fa45c0062e0 msgr2=0x7fa45c046530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:19.708 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.706+0000 7f432effd640 1 -- 172.21.15.157:0/2936089590 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4320013070 con 0x7f43380c6e70 2024-04-24T23:35:19.708 INFO:teuthology.orchestra.run.smithi157.stdout:60129542151 2024-04-24T23:35:19.708 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.706+0000 7fa46a226640 1 -- 172.21.15.157:0/1839507841 shutdown_connections 2024-04-24T23:35:19.708 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.706+0000 7fa46a226640 1 -- 172.21.15.157:0/1839507841 wait complete. 2024-04-24T23:35:19.710 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.709+0000 7f433759e640 1 -- 172.21.15.157:0/2936089590 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f431806be30 msgr2=0x7f431806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.710 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.709+0000 7f433759e640 1 --2- 172.21.15.157:0/2936089590 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f431806be30 0x7f431806e2f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4328006fd0 tx=0x7f4328009060 comp rx=0 tx=0).stop 2024-04-24T23:35:19.710 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.709+0000 7f433759e640 1 -- 172.21.15.157:0/2936089590 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43380c6e70 msgr2=0x7f433814c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:19.710 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.709+0000 7f433759e640 1 --2- 172.21.15.157:0/2936089590 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43380c6e70 0x7f433814c8f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4320010b20 tx=0x7f432000e050 comp rx=0 tx=0).stop 2024-04-24T23:35:19.710 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.710+0000 7f433759e640 1 -- 172.21.15.157:0/2936089590 shutdown_connections 2024-04-24T23:35:19.711 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.710+0000 7f433759e640 1 --2- 172.21.15.157:0/2936089590 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f431806be30 0x7f431806e2f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.711 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.710+0000 7f433759e640 1 --2- 172.21.15.157:0/2936089590 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4338157370 0x7f433814ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.711 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.710+0000 7f433759e640 1 --2- 172.21.15.157:0/2936089590 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43380c6e70 0x7f433814c8f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.711 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.710+0000 7f433759e640 1 --2- 172.21.15.157:0/2936089590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f433806f860 0x7f43381dd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:19.711 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.710+0000 7f433759e640 1 -- 172.21.15.157:0/2936089590 >> 172.21.15.157:0/2936089590 conn(0x7f4338067080 msgr2=0x7f433806c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:19.711 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.710+0000 7f433759e640 1 -- 172.21.15.157:0/2936089590 shutdown_connections 2024-04-24T23:35:19.711 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:19.710+0000 7f433759e640 1 -- 172.21.15.157:0/2936089590 wait complete. 2024-04-24T23:35:19.712 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542150 for osd.2 2024-04-24T23:35:19.715 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.3 2024-04-24T23:35:19.718 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542150 for osd.5 2024-04-24T23:35:19.721 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542150 for osd.0 2024-04-24T23:35:19.723 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542150 for osd.1 2024-04-24T23:35:19.726 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542151 for osd.6 2024-04-24T23:35:19.727 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542150 for osd.4 2024-04-24T23:35:20.697 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-24T23:35:20.713 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-24T23:35:20.716 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-24T23:35:20.719 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-24T23:35:20.722 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-24T23:35:20.723 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-24T23:35:20.727 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-24T23:35:20.727 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-24T23:35:20.863 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.861+0000 7f957d98c640 1 Processor -- start 2024-04-24T23:35:20.863 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.861+0000 7f957d98c640 1 -- start start 2024-04-24T23:35:20.863 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.862+0000 7f957d98c640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9570047240 0x7f9570007bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:20.863 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.862+0000 7f957d98c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f957004a250 con 0x7f957004adc0 2024-04-24T23:35:20.863 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.862+0000 7f957d98c640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95700476d0 con 0x7f957004a9e0 2024-04-24T23:35:20.864 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.862+0000 7f957d98c640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f957004b250 con 0x7f9570047240 2024-04-24T23:35:20.864 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.863+0000 7f9577fff640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9570047240 0x7f9570007bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:20.864 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.863+0000 7f9577fff640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9570047240 0x7f9570007bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.157:52894/0 (socket says 172.21.15.157:52894) 2024-04-24T23:35:20.864 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.863+0000 7f9577fff640 1 -- 172.21.15.157:0/2698677902 learned_addr learned my addr 172.21.15.157:0/2698677902 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:20.865 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.863+0000 7f9577fff640 1 -- 172.21.15.157:0/2698677902 >> v1:172.21.15.169:6789/0 conn(0x7f957004a9e0 legacy=0x7f95700db000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:20.865 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.864+0000 7f9577fff640 1 -- 172.21.15.157:0/2698677902 >> v1:172.21.15.157:6789/0 conn(0x7f957004adc0 legacy=0x7f957004bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:20.865 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.864+0000 7f9577fff640 1 -- 172.21.15.157:0/2698677902 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95700479b0 con 0x7f9570047240 2024-04-24T23:35:20.866 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.865+0000 7f9577fff640 1 --2- 172.21.15.157:0/2698677902 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9570047240 0x7f9570007bc0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f95780698d0 tx=0x7f95780a7910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20f9d2925ef83953 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:20.866 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.865+0000 7f95777fe640 1 -- 172.21.15.157:0/2698677902 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957806d040 con 0x7f9570047240 2024-04-24T23:35:20.866 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.865+0000 7f95777fe640 1 -- 172.21.15.157:0/2698677902 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95780ae660 con 0x7f9570047240 2024-04-24T23:35:20.868 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.866+0000 7f957d98c640 1 -- 172.21.15.157:0/2698677902 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9570047240 msgr2=0x7f9570007bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:20.868 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.866+0000 7f957d98c640 1 --2- 172.21.15.157:0/2698677902 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9570047240 0x7f9570007bc0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f95780698d0 tx=0x7f95780a7910 comp rx=0 tx=0).stop 2024-04-24T23:35:20.868 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.867+0000 7f957d98c640 1 -- 172.21.15.157:0/2698677902 shutdown_connections 2024-04-24T23:35:20.868 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.867+0000 7f957d98c640 1 --2- 172.21.15.157:0/2698677902 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9570047240 0x7f9570007bc0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:20.868 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.867+0000 7f957d98c640 1 -- 172.21.15.157:0/2698677902 >> 172.21.15.157:0/2698677902 conn(0x7f9570049340 msgr2=0x7f9570049750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:20.868 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.867+0000 7f957d98c640 1 -- 172.21.15.157:0/2698677902 shutdown_connections 2024-04-24T23:35:20.869 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.867+0000 7f957d98c640 1 -- 172.21.15.157:0/2698677902 wait complete. 2024-04-24T23:35:20.870 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.869+0000 7f957d98c640 1 Processor -- start 2024-04-24T23:35:20.872 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.869+0000 7f957d98c640 1 -- start start 2024-04-24T23:35:20.872 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.871+0000 7f957d98c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f957004a9e0 0x7f95700453c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:20.872 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.871+0000 7f957d98c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f957004adc0 0x7f9570174600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:20.872 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.871+0000 7f957d98c640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9570042260 0x7f95700445f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:20.872 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.871+0000 7f957d98c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f957004c5d0 con 0x7f957004adc0 2024-04-24T23:35:20.873 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.871+0000 7f957d98c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f957004ba30 con 0x7f957004a9e0 2024-04-24T23:35:20.873 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.871+0000 7f957d98c640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9570047c90 con 0x7f9570042260 2024-04-24T23:35:20.873 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.871+0000 7f9577fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f957004adc0 0x7f9570174600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:20.873 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.871+0000 7f9577fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f957004adc0 0x7f9570174600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42886/0 (socket says 172.21.15.157:42886) 2024-04-24T23:35:20.873 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.871+0000 7f9577fff640 1 -- 172.21.15.157:0/2378755558 learned_addr learned my addr 172.21.15.157:0/2378755558 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:20.873 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.872+0000 7f9577fff640 1 -- 172.21.15.157:0/2378755558 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9570042260 msgr2=0x7f95700445f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:20.874 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.872+0000 7f9577fff640 1 --2- 172.21.15.157:0/2378755558 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9570042260 0x7f95700445f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:20.874 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.872+0000 7f9577fff640 1 -- 172.21.15.157:0/2378755558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f957004a9e0 msgr2=0x7f95700453c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:20.874 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.872+0000 7f9577fff640 1 --2- 172.21.15.157:0/2378755558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f957004a9e0 0x7f95700453c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:20.874 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.872+0000 7f9577fff640 1 -- 172.21.15.157:0/2378755558 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9570175530 con 0x7f957004adc0 2024-04-24T23:35:20.874 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.872+0000 7f9577fff640 1 --2- 172.21.15.157:0/2378755558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f957004adc0 0x7f9570174600 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f957806a550 tx=0x7f957806b090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:20.874 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.873+0000 7f95757fa640 1 -- 172.21.15.157:0/2378755558 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95780bd070 con 0x7f957004adc0 2024-04-24T23:35:20.875 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.873+0000 7f957d98c640 1 -- 172.21.15.157:0/2378755558 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9570174f70 con 0x7f957004adc0 2024-04-24T23:35:20.875 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.873+0000 7f957d98c640 1 -- 172.21.15.157:0/2378755558 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9570043df0 con 0x7f957004adc0 2024-04-24T23:35:20.878 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.877+0000 7f95757fa640 1 -- 172.21.15.157:0/2378755558 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95780b8410 con 0x7f957004adc0 2024-04-24T23:35:20.878 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.877+0000 7f95757fa640 1 -- 172.21.15.157:0/2378755558 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95780b8860 con 0x7f957004adc0 2024-04-24T23:35:20.878 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.878+0000 7f95757fa640 1 -- 172.21.15.157:0/2378755558 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f95780c0070 con 0x7f957004adc0 2024-04-24T23:35:20.879 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.879+0000 7f95757fa640 1 --2- 172.21.15.157:0/2378755558 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f955c06bf30 0x7f955c06e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:20.880 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.879+0000 7f957c98a640 1 --2- 172.21.15.157:0/2378755558 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f955c06bf30 0x7f955c06e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:20.880 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.879+0000 7f95757fa640 1 -- 172.21.15.157:0/2378755558 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f95781265a0 con 0x7f957004adc0 2024-04-24T23:35:20.880 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.880+0000 7f957c98a640 1 --2- 172.21.15.157:0/2378755558 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f955c06bf30 0x7f955c06e3f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f956c003cc0 tx=0x7f956c0079b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:20.881 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.880+0000 7f957d98c640 1 -- 172.21.15.157:0/2378755558 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9558005290 con 0x7f957004adc0 2024-04-24T23:35:20.890 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.889+0000 7f95757fa640 1 -- 172.21.15.157:0/2378755558 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f95780f45a0 con 0x7f957004adc0 2024-04-24T23:35:20.913 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.912+0000 7f8de34a5640 1 Processor -- start 2024-04-24T23:35:20.913 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.912+0000 7f8de34a5640 1 -- start start 2024-04-24T23:35:20.913 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.913+0000 7fe24b59e640 1 Processor -- start 2024-04-24T23:35:20.914 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.913+0000 7fe24b59e640 1 -- start start 2024-04-24T23:35:20.914 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.913+0000 7f8de34a5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8ddc156e90 0x7f8ddc0c47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:20.914 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.913+0000 7f8de34a5640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8ddc06d960 0x7f8ddc0c4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:20.914 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.913+0000 7f8de34a5640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ddc157270 con 0x7f8ddc0c1c50 2024-04-24T23:35:20.914 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.913+0000 7f8de34a5640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f8ddc0c2860 con 0x7f8ddc156e90 2024-04-24T23:35:20.915 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.913+0000 7f8de34a5640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f8ddc0c2c30 con 0x7f8ddc06d960 2024-04-24T23:35:20.915 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.914+0000 7f8de1241640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8ddc156e90 0x7f8ddc0c47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:20.915 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.914+0000 7f78cf363640 1 Processor -- start 2024-04-24T23:35:20.915 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.914+0000 7f8de1a42640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f8ddc0c1c50 0x7f8ddc159800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:47698/0 (socket says 172.21.15.157:47698) 2024-04-24T23:35:20.915 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.914+0000 7f8de1a42640 1 -- 172.21.15.157:0/814090756 learned_addr learned my addr 172.21.15.157:0/814090756 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-24T23:35:20.916 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.914+0000 7f8de1241640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8ddc156e90 0x7f8ddc0c47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42902/0 (socket says 172.21.15.157:42902) 2024-04-24T23:35:20.916 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.914+0000 7ff382ad4640 1 Processor -- start 2024-04-24T23:35:20.916 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.914+0000 7f78cf363640 1 -- start start 2024-04-24T23:35:20.916 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.914+0000 7f8de0a40640 1 --2- 172.21.15.157:0/814090756 >> v2:172.21.15.169:3300/0 conn(0x7f8ddc06d960 0x7f8ddc0c4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:20.917 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.914+0000 7f92f6e88640 1 Processor -- start 2024-04-24T23:35:20.917 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.914+0000 7f92f6e88640 1 -- start start 2024-04-24T23:35:20.917 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.914+0000 7fe24b59e640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe244050720 0x7fe2440062c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:20.917 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.915+0000 7fe24b59e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe24400fcf0 0x7fe244006800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:20.918 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.914+0000 7f8dd3fff640 1 -- 172.21.15.157:0/814090756 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2795626643 0 0) 0x7f8ddc157270 con 0x7f8ddc0c1c50 2024-04-24T23:35:20.918 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.914+0000 7f8dd3fff640 1 -- 172.21.15.157:0/814090756 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8dc8003520 con 0x7f8ddc0c1c50 2024-04-24T23:35:20.918 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.914+0000 7f8de1241640 1 -- 172.21.15.157:0/814090756 >> v2:172.21.15.169:3300/0 conn(0x7f8ddc06d960 msgr2=0x7f8ddc0c4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:20.918 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.914+0000 7f8de1241640 1 --2- 172.21.15.157:0/814090756 >> v2:172.21.15.169:3300/0 conn(0x7f8ddc06d960 0x7f8ddc0c4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:20.918 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.914+0000 7f8de1241640 1 -- 172.21.15.157:0/814090756 >> v1:172.21.15.157:6789/0 conn(0x7f8ddc0c1c50 legacy=0x7f8ddc159800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:20.918 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.914+0000 7f8de1241640 1 -- 172.21.15.157:0/814090756 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ddc06cec0 con 0x7f8ddc156e90 2024-04-24T23:35:20.919 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.915+0000 7f78cf363640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78c00d3b70 0x7f78c0046b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:20.919 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.915+0000 7f78cf363640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f78c00d6740 0x7f78c00d6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:20.919 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.916+0000 7f92f6e88640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f92f00c6080 0x7f92f0155cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:20.919 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.916+0000 7f06e759e640 1 Processor -- start 2024-04-24T23:35:20.920 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.916+0000 7f8de1241640 1 --2- 172.21.15.157:0/814090756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ddc156e90 0x7f8ddc0c47e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc0a9380 tx=0x7f8dcc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a76ee3fda4bb58b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:20.920 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.915+0000 7ff382ad4640 1 -- start start 2024-04-24T23:35:20.920 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.916+0000 7ff382ad4640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff37c072e40 0x7ff37c0717b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:20.920 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.916+0000 7ff382ad4640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff37c155600 0x7ff37c155a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:20.920 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.916+0000 7f92f6e88640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92f00c24d0 con 0x7f92f006b3d0 2024-04-24T23:35:20.921 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.916+0000 7f92effff640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f92f00c6080 0x7f92f0155cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:20.921 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.916+0000 7f92effff640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f92f00c6080 0x7f92f0155cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.157:45118/0 (socket says 172.21.15.157:45118) 2024-04-24T23:35:20.921 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.916+0000 7ff382ad4640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff37c156580 con 0x7ff37c0c30d0 2024-04-24T23:35:20.921 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.916+0000 7ff382ad4640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7ff37c06d4c0 con 0x7ff37c155600 2024-04-24T23:35:20.921 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.916+0000 7ff382ad4640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7ff37c0c6350 con 0x7ff37c072e40 2024-04-24T23:35:20.921 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.916+0000 7f92effff640 1 -- 172.21.15.157:0/2309171270 learned_addr learned my addr 172.21.15.157:0/2309171270 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:20.922 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.915+0000 7fe249d9b640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe244050720 0x7fe2440062c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:20.922 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.915+0000 7fe249d9b640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe244050720 0x7fe2440062c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.157:52908/0 (socket says 172.21.15.157:52908) 2024-04-24T23:35:20.922 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.915+0000 7fe24b59e640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2440d7d00 con 0x7fe2440d7920 2024-04-24T23:35:20.922 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.915+0000 7fe24b59e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fe2440db250 con 0x7fe24400fcf0 2024-04-24T23:35:20.922 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.916+0000 7f92f6e88640 1 -- 172.21.15.157:0/2309171270 --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92f00c0ec0 con 0x7f92f006f630 2024-04-24T23:35:20.922 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.915+0000 7fe24b59e640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe24404ba00 con 0x7fe244050720 2024-04-24T23:35:20.923 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.915+0000 7fe249d9b640 1 -- 172.21.15.157:0/3812705051 learned_addr learned my addr 172.21.15.157:0/3812705051 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:20.923 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.916+0000 7fe24ad9d640 1 --2- 172.21.15.157:0/3812705051 >> v2:172.21.15.157:3300/0 conn(0x7fe24400fcf0 0x7fe244006800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:20.923 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.916+0000 7fe249d9b640 1 -- 172.21.15.157:0/3812705051 >> v2:172.21.15.157:3300/0 conn(0x7fe24400fcf0 msgr2=0x7fe244006800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:20.923 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.916+0000 7fe249d9b640 1 --2- 172.21.15.157:0/3812705051 >> v2:172.21.15.157:3300/0 conn(0x7fe24400fcf0 0x7fe244006800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:20.923 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.916+0000 7fe249d9b640 1 -- 172.21.15.157:0/3812705051 >> v1:172.21.15.169:6789/0 conn(0x7fe2440d7920 legacy=0x7fe24400edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:20.923 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.916+0000 7f06e759e640 1 -- start start 2024-04-24T23:35:20.923 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.916+0000 7f92f6e88640 1 -- 172.21.15.157:0/2309171270 --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f92f00c37c0 con 0x7f92f00c6080 2024-04-24T23:35:20.924 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.916+0000 7fe249d9b640 1 -- 172.21.15.157:0/3812705051 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe24404bce0 con 0x7fe244050720 2024-04-24T23:35:20.924 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.916+0000 7f8dd3fff640 1 -- 172.21.15.157:0/814090756 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc037070 con 0x7f8ddc156e90 2024-04-24T23:35:20.924 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.916+0000 7ff381071640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff37c155600 0x7ff37c155a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:20.925 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.916+0000 7ff381071640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff37c155600 0x7ff37c155a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42926/0 (socket says 172.21.15.157:42926) 2024-04-24T23:35:20.925 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.916+0000 7ff381071640 1 -- 172.21.15.157:0/843756642 learned_addr learned my addr 172.21.15.157:0/843756642 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:20.925 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.916+0000 7f78cd0ff640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78c00d3b70 0x7f78c0046b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:20.925 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.916+0000 7f78cc8fe640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f78c004b140 0x7f78c0006ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.157:42604/0 (socket says 172.21.15.157:42604) 2024-04-24T23:35:20.925 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.916+0000 7f78cd0ff640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78c00d3b70 0x7f78c0046b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.157:52916/0 (socket says 172.21.15.157:52916) 2024-04-24T23:35:20.925 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.916+0000 7f8dd3fff640 1 -- 172.21.15.157:0/814090756 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dcc032440 con 0x7f8ddc156e90 2024-04-24T23:35:20.926 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.916+0000 7f78cc8fe640 1 -- 172.21.15.157:0/257344436 learned_addr learned my addr 172.21.15.157:0/257344436 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-24T23:35:20.926 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.916+0000 7f78cf363640 1 -- 172.21.15.157:0/257344436 --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78c0047080 con 0x7f78c004b140 2024-04-24T23:35:20.926 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.916+0000 7f78cf363640 1 -- 172.21.15.157:0/257344436 --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f78c00d9c90 con 0x7f78c00d6740 2024-04-24T23:35:20.926 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.916+0000 7f78cf363640 1 -- 172.21.15.157:0/257344436 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f78c00d9f70 con 0x7f78c00d3b70 2024-04-24T23:35:20.926 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.916+0000 7f8dd3fff640 1 -- 172.21.15.157:0/814090756 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc0327f0 con 0x7f8ddc156e90 2024-04-24T23:35:20.927 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.917+0000 7f92effff640 1 -- 172.21.15.157:0/2309171270 >> v1:172.21.15.169:6789/0 conn(0x7f92f006f630 legacy=0x7f92f015fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:20.927 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.917+0000 7ff381071640 1 -- 172.21.15.157:0/843756642 >> v2:172.21.15.169:3300/0 conn(0x7ff37c072e40 msgr2=0x7ff37c0717b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T23:35:20.927 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.917+0000 7ff381071640 1 --2- 172.21.15.157:0/843756642 >> v2:172.21.15.169:3300/0 conn(0x7ff37c072e40 0x7ff37c0717b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:20.927 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.917+0000 7ff381071640 1 -- 172.21.15.157:0/843756642 >> v1:172.21.15.157:6789/0 conn(0x7ff37c0c30d0 legacy=0x7ff37c071cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:20.927 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.917+0000 7ff381071640 1 -- 172.21.15.157:0/843756642 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff37c0c6630 con 0x7ff37c155600 2024-04-24T23:35:20.928 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.917+0000 7f78cd900640 1 --2- 172.21.15.157:0/257344436 >> v2:172.21.15.169:3300/0 conn(0x7f78c00d6740 0x7f78c00d6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:20.928 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.917+0000 7f92effff640 1 -- 172.21.15.157:0/2309171270 >> v1:172.21.15.157:6789/0 conn(0x7f92f006b3d0 legacy=0x7f92f0158970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:20.928 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.917+0000 7f92effff640 1 -- 172.21.15.157:0/2309171270 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92f00c11a0 con 0x7f92f00c6080 2024-04-24T23:35:20.928 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.917+0000 7f8f21638640 1 Processor -- start 2024-04-24T23:35:20.929 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.917+0000 7f78cd0ff640 1 -- 172.21.15.157:0/257344436 >> v2:172.21.15.169:3300/0 conn(0x7f78c00d6740 msgr2=0x7f78c00d6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:20.929 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.917+0000 7f78cd0ff640 1 --2- 172.21.15.157:0/257344436 >> v2:172.21.15.169:3300/0 conn(0x7f78c00d6740 0x7f78c00d6bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:20.929 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.917+0000 7f78cd0ff640 1 -- 172.21.15.157:0/257344436 >> v1:172.21.15.169:6789/0 conn(0x7f78c004b140 legacy=0x7f78c0006ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:20.929 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.917+0000 7f78bffff640 1 -- 172.21.15.157:0/257344436 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3481585468 0 0) 0x7f78c0047080 con 0x7f78c004b140 2024-04-24T23:35:20.929 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.917+0000 7f78cd0ff640 1 -- 172.21.15.157:0/257344436 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78c00da250 con 0x7f78c00d3b70 2024-04-24T23:35:20.929 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.917+0000 7f06e759e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f06e00dbce0 0x7f06e00474a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:20.930 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.917+0000 7f06e759e640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06e00df180 0x7f06e0006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:20.930 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.917+0000 7f06e759e640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f06e00073e0 0x7f06e000a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:20.930 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.917+0000 7f06e759e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f06e000a420 con 0x7f06e00dbce0 2024-04-24T23:35:20.930 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.917+0000 7f06e759e640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f06e000bb50 con 0x7f06e00073e0 2024-04-24T23:35:20.930 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.917+0000 7f06e759e640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f06e0048060 con 0x7f06e00df180 2024-04-24T23:35:20.930 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.917+0000 7f06e5d9b640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06e00df180 0x7f06e0006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:20.930 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.917+0000 7f06e5d9b640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06e00df180 0x7f06e0006ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.157:52928/0 (socket says 172.21.15.157:52928) 2024-04-24T23:35:20.930 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.917+0000 7f06e5d9b640 1 -- 172.21.15.157:0/2805660033 learned_addr learned my addr 172.21.15.157:0/2805660033 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:20.930 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.917+0000 7f8de34a5640 1 -- 172.21.15.157:0/814090756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ddc156e90 msgr2=0x7f8ddc0c47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:20.930 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.917+0000 7f8de34a5640 1 --2- 172.21.15.157:0/814090756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ddc156e90 0x7f8ddc0c47e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc0a9380 tx=0x7f8dcc02a1a0 comp rx=0 tx=0).stop 2024-04-24T23:35:20.931 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.918+0000 7f06e6d9d640 1 --2- 172.21.15.157:0/2805660033 >> v2:172.21.15.169:3300/0 conn(0x7f06e00073e0 0x7f06e000a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:20.931 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.918+0000 7f06e659c640 1 --2- 172.21.15.157:0/2805660033 >> v2:172.21.15.157:3300/0 conn(0x7f06e00dbce0 0x7f06e00474a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:20.931 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.918+0000 7f8de34a5640 1 -- 172.21.15.157:0/814090756 shutdown_connections 2024-04-24T23:35:20.931 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.918+0000 7f8de34a5640 1 --2- 172.21.15.157:0/814090756 >> v2:172.21.15.169:3300/0 conn(0x7f8ddc06d960 0x7f8ddc0c4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:20.931 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.918+0000 7f8de34a5640 1 --2- 172.21.15.157:0/814090756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ddc156e90 0x7f8ddc0c47e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:20.932 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.918+0000 7f8de34a5640 1 -- 172.21.15.157:0/814090756 >> 172.21.15.157:0/814090756 conn(0x7f8ddc0a8690 msgr2=0x7f8ddc0a8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:20.932 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.918+0000 7fe249d9b640 1 --2- 172.21.15.157:0/3812705051 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe244050720 0x7fe2440062c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe240013db0 tx=0x7fe2400328d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52098be29958b1b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:20.932 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.918+0000 7f06e5d9b640 1 -- 172.21.15.157:0/2805660033 >> v2:172.21.15.169:3300/0 conn(0x7f06e00073e0 msgr2=0x7f06e000a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:20.932 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.918+0000 7f06e5d9b640 1 --2- 172.21.15.157:0/2805660033 >> v2:172.21.15.169:3300/0 conn(0x7f06e00073e0 0x7f06e000a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:20.932 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.918+0000 7f06e5d9b640 1 -- 172.21.15.157:0/2805660033 >> v2:172.21.15.157:3300/0 conn(0x7f06e00dbce0 msgr2=0x7f06e00474a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:20.933 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.918+0000 7f92effff640 1 --2- 172.21.15.157:0/2309171270 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92f00c6080 0x7f92f0155cb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f92e0009870 tx=0x7f92e002a280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b939f3b2b277d5ee server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:20.933 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.918+0000 7f06e5d9b640 1 --2- 172.21.15.157:0/2805660033 >> v2:172.21.15.157:3300/0 conn(0x7f06e00dbce0 0x7f06e00474a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:20.933 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.918+0000 7f06e5d9b640 1 -- 172.21.15.157:0/2805660033 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06e000a0f0 con 0x7f06e00df180 2024-04-24T23:35:20.933 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.918+0000 7f06e6d9d640 1 --2- 172.21.15.157:0/2805660033 >> v2:172.21.15.169:3300/0 conn(0x7f06e00073e0 0x7f06e000a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:35:20.933 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.918+0000 7f92ef7fe640 1 -- 172.21.15.157:0/2309171270 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e0037070 con 0x7f92f00c6080 2024-04-24T23:35:20.934 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.918+0000 7f8de34a5640 1 -- 172.21.15.157:0/814090756 shutdown_connections 2024-04-24T23:35:20.934 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.918+0000 7fe24959a640 1 -- 172.21.15.157:0/3812705051 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2400387d0 con 0x7fe244050720 2024-04-24T23:35:20.934 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.918+0000 7f92ef7fe640 1 -- 172.21.15.157:0/2309171270 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92e0032440 con 0x7f92f00c6080 2024-04-24T23:35:20.935 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.918+0000 7f78cd0ff640 1 --2- 172.21.15.157:0/257344436 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78c00d3b70 0x7f78c0046b40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f78b800a940 tx=0x7f78b802b4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=669c2b9293724158 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:20.935 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.918+0000 7f92ef7fe640 1 -- 172.21.15.157:0/2309171270 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e0032810 con 0x7f92f00c6080 2024-04-24T23:35:20.935 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.918+0000 7f78bffff640 1 -- 172.21.15.157:0/257344436 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b800b040 con 0x7f78c00d3b70 2024-04-24T23:35:20.935 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.919+0000 7f78bffff640 1 -- 172.21.15.157:0/257344436 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78b8030710 con 0x7f78c00d3b70 2024-04-24T23:35:20.935 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.919+0000 7f78bffff640 1 -- 172.21.15.157:0/257344436 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b8030ae0 con 0x7f78c00d3b70 2024-04-24T23:35:20.936 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.918+0000 7f8f21638640 1 -- start start 2024-04-24T23:35:20.936 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.919+0000 7ff381071640 1 --2- 172.21.15.157:0/843756642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff37c155600 0x7ff37c155a70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff37c0c2a10 tx=0x7ff37402ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f115b5d84c582383 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:20.936 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.918+0000 7f8de34a5640 1 -- 172.21.15.157:0/814090756 wait complete. 2024-04-24T23:35:20.937 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.918+0000 7fe24959a640 1 -- 172.21.15.157:0/3812705051 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe24003e070 con 0x7fe244050720 2024-04-24T23:35:20.937 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.919+0000 7fe24959a640 1 -- 172.21.15.157:0/3812705051 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2400387d0 con 0x7fe244050720 2024-04-24T23:35:20.937 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.919+0000 7f8f21638640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8f1c066ea0 0x7f8f1c073300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:20.938 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.919+0000 7f8f21638640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f1c15d190 0x7f8f1c15d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:20.938 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.919+0000 7f8f21638640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f1c15cc90 con 0x7f8f1c156ee0 2024-04-24T23:35:20.938 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.919+0000 7f8f1affd640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f8f1c156ee0 0x7f8f1c0c7790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:47724/0 (socket says 172.21.15.157:47724) 2024-04-24T23:35:20.938 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.919+0000 7f8f1affd640 1 -- 172.21.15.157:0/3955102598 learned_addr learned my addr 172.21.15.157:0/3955102598 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-24T23:35:20.938 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.919+0000 7f8f21638640 1 -- 172.21.15.157:0/3955102598 --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f8f1c15db60 con 0x7f8f1c066ea0 2024-04-24T23:35:20.938 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.919+0000 7f8f21638640 1 -- 172.21.15.157:0/3955102598 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8f1c067410 con 0x7f8f1c15d190 2024-04-24T23:35:20.938 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.919+0000 7f8f1a7fc640 1 --2- 172.21.15.157:0/3955102598 >> v2:172.21.15.169:3300/0 conn(0x7f8f1c066ea0 0x7f8f1c073300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:20.938 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.919+0000 7f92f6e88640 1 -- 172.21.15.157:0/2309171270 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92f00c6080 msgr2=0x7f92f0155cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:20.939 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.919+0000 7fe24b59e640 1 -- 172.21.15.157:0/3812705051 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe244050720 msgr2=0x7fe2440062c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:20.939 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.919+0000 7fe24b59e640 1 --2- 172.21.15.157:0/3812705051 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe244050720 0x7fe2440062c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe240013db0 tx=0x7fe2400328d0 comp rx=0 tx=0).stop 2024-04-24T23:35:20.939 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.919+0000 7f06e5d9b640 1 --2- 172.21.15.157:0/2805660033 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06e00df180 0x7f06e0006ea0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f06e000a2c0 tx=0x7f06d802a140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8b4c1e5e37c8fbb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:20.939 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.919+0000 7f8f1bfff640 1 --2- 172.21.15.157:0/3955102598 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f1c15d190 0x7f8f1c15d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:20.940 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.919+0000 7f92f6e88640 1 --2- 172.21.15.157:0/2309171270 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92f00c6080 0x7f92f0155cb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f92e0009870 tx=0x7f92e002a280 comp rx=0 tx=0).stop 2024-04-24T23:35:20.940 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.920+0000 7f92f6e88640 1 -- 172.21.15.157:0/2309171270 shutdown_connections 2024-04-24T23:35:20.940 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.920+0000 7f92f6e88640 1 --2- 172.21.15.157:0/2309171270 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92f00c6080 0x7f92f0155cb0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:20.940 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.920+0000 7f92f6e88640 1 -- 172.21.15.157:0/2309171270 >> 172.21.15.157:0/2309171270 conn(0x7f92f006c270 msgr2=0x7f92f006c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:20.940 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.919+0000 7ff37b7fe640 1 -- 172.21.15.157:0/843756642 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff374036070 con 0x7ff37c155600 2024-04-24T23:35:20.941 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.920+0000 7ff37b7fe640 1 -- 172.21.15.157:0/843756642 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff374032640 con 0x7ff37c155600 2024-04-24T23:35:20.941 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.920+0000 7ff37b7fe640 1 -- 172.21.15.157:0/843756642 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff374032a10 con 0x7ff37c155600 2024-04-24T23:35:20.941 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.920+0000 7f78cf363640 1 -- 172.21.15.157:0/257344436 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78c00d3b70 msgr2=0x7f78c0046b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:20.941 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.920+0000 7f8f1a7fc640 1 -- 172.21.15.157:0/3955102598 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f1c15d190 msgr2=0x7f8f1c15d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:20.941 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.920+0000 7f8f1a7fc640 1 --2- 172.21.15.157:0/3955102598 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f1c15d190 0x7f8f1c15d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:20.941 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.920+0000 7f8f19ffb640 1 -- 172.21.15.157:0/3955102598 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4078930471 0 0) 0x7f8f1c15cc90 con 0x7f8f1c156ee0 2024-04-24T23:35:20.942 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.920+0000 7f8f1a7fc640 1 -- 172.21.15.157:0/3955102598 >> v1:172.21.15.157:6789/0 conn(0x7f8f1c156ee0 legacy=0x7f8f1c0c7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:20.942 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.920+0000 7f8f1a7fc640 1 -- 172.21.15.157:0/3955102598 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f1c0c4970 con 0x7f8f1c066ea0 2024-04-24T23:35:20.942 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.920+0000 7f78cf363640 1 --2- 172.21.15.157:0/257344436 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78c00d3b70 0x7f78c0046b40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f78b800a940 tx=0x7f78b802b4d0 comp rx=0 tx=0).stop 2024-04-24T23:35:20.942 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.920+0000 7f92f6e88640 1 -- 172.21.15.157:0/2309171270 shutdown_connections 2024-04-24T23:35:20.942 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.920+0000 7f92f6e88640 1 -- 172.21.15.157:0/2309171270 wait complete. 2024-04-24T23:35:20.943 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.920+0000 7f78cf363640 1 -- 172.21.15.157:0/257344436 shutdown_connections 2024-04-24T23:35:20.943 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.920+0000 7f78cf363640 1 --2- 172.21.15.157:0/257344436 >> v2:172.21.15.169:3300/0 conn(0x7f78c00d6740 0x7f78c00d6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:20.943 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.920+0000 7f78cf363640 1 --2- 172.21.15.157:0/257344436 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78c00d3b70 0x7f78c0046b40 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:20.943 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.920+0000 7fe24b59e640 1 -- 172.21.15.157:0/3812705051 shutdown_connections 2024-04-24T23:35:20.943 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.920+0000 7fe24b59e640 1 --2- 172.21.15.157:0/3812705051 >> v2:172.21.15.157:3300/0 conn(0x7fe24400fcf0 0x7fe244006800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:20.944 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.920+0000 7f78cf363640 1 -- 172.21.15.157:0/257344436 >> 172.21.15.157:0/257344436 conn(0x7f78c0048280 msgr2=0x7f78c0048690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:20.944 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.920+0000 7fe24b59e640 1 --2- 172.21.15.157:0/3812705051 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe244050720 0x7fe2440062c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:20.944 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.920+0000 7fe24b59e640 1 -- 172.21.15.157:0/3812705051 >> 172.21.15.157:0/3812705051 conn(0x7fe24404fc00 msgr2=0x7fe244050010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:20.944 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.920+0000 7f8de34a5640 1 Processor -- start 2024-04-24T23:35:20.945 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.920+0000 7f78cf363640 1 -- 172.21.15.157:0/257344436 shutdown_connections 2024-04-24T23:35:20.945 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.920+0000 7f78cf363640 1 -- 172.21.15.157:0/257344436 wait complete. 2024-04-24T23:35:20.945 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.920+0000 7f06e559a640 1 -- 172.21.15.157:0/2805660033 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d8037070 con 0x7f06e00df180 2024-04-24T23:35:20.945 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.920+0000 7f06e559a640 1 -- 172.21.15.157:0/2805660033 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06d8032440 con 0x7f06e00df180 2024-04-24T23:35:20.945 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.920+0000 7f06e559a640 1 -- 172.21.15.157:0/2805660033 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d8032810 con 0x7f06e00df180 2024-04-24T23:35:20.946 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.920+0000 7fe24b59e640 1 -- 172.21.15.157:0/3812705051 shutdown_connections 2024-04-24T23:35:20.946 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.921+0000 7ff382ad4640 1 -- 172.21.15.157:0/843756642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff37c155600 msgr2=0x7ff37c155a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:20.946 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.921+0000 7ff382ad4640 1 --2- 172.21.15.157:0/843756642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff37c155600 0x7ff37c155a70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff37c0c2a10 tx=0x7ff37402ead0 comp rx=0 tx=0).stop 2024-04-24T23:35:20.947 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.921+0000 7ff382ad4640 1 -- 172.21.15.157:0/843756642 shutdown_connections 2024-04-24T23:35:20.947 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.921+0000 7ff382ad4640 1 --2- 172.21.15.157:0/843756642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff37c155600 0x7ff37c155a70 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:20.947 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.921+0000 7ff382ad4640 1 --2- 172.21.15.157:0/843756642 >> v2:172.21.15.169:3300/0 conn(0x7ff37c072e40 0x7ff37c0717b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:20.947 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.921+0000 7ff382ad4640 1 -- 172.21.15.157:0/843756642 >> 172.21.15.157:0/843756642 conn(0x7ff37c06c270 msgr2=0x7ff37c06c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:20.947 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.920+0000 7f8de34a5640 1 -- start start 2024-04-24T23:35:20.947 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.921+0000 7f8f1a7fc640 1 --2- 172.21.15.157:0/3955102598 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f1c066ea0 0x7f8f1c073300 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8f100097d0 tx=0x7f8f1002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32008e533cfe953a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:20.948 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.921+0000 7fe24b59e640 1 -- 172.21.15.157:0/3812705051 wait complete. 2024-04-24T23:35:20.948 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.921+0000 7f06e759e640 1 -- 172.21.15.157:0/2805660033 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06e00df180 msgr2=0x7f06e0006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:20.948 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.921+0000 7f06e759e640 1 --2- 172.21.15.157:0/2805660033 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06e00df180 0x7f06e0006ea0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f06e000a2c0 tx=0x7f06d802a140 comp rx=0 tx=0).stop 2024-04-24T23:35:20.948 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.921+0000 7f8de34a5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ddc06d960 0x7f8ddc15afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:20.948 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.921+0000 7f8de34a5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ddc0c1c50 0x7f8ddc15cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:20.948 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.921+0000 7f8de34a5640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ddc156e90 0x7f8ddc158c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:20.948 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.921+0000 7f8de34a5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8ddc06d440 con 0x7f8ddc06d960 2024-04-24T23:35:20.948 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.921+0000 7f8de34a5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8ddc06d160 con 0x7f8ddc0c1c50 2024-04-24T23:35:20.949 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.921+0000 7f8de34a5640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8ddc0c3160 con 0x7f8ddc156e90 2024-04-24T23:35:20.949 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.921+0000 7f8f19ffb640 1 -- 172.21.15.157:0/3955102598 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f10037070 con 0x7f8f1c066ea0 2024-04-24T23:35:20.949 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.921+0000 7f8f19ffb640 1 -- 172.21.15.157:0/3955102598 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f10032440 con 0x7f8f1c066ea0 2024-04-24T23:35:20.949 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.921+0000 7f8f19ffb640 1 -- 172.21.15.157:0/3955102598 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f10032810 con 0x7f8f1c066ea0 2024-04-24T23:35:20.949 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.921+0000 7ff382ad4640 1 -- 172.21.15.157:0/843756642 shutdown_connections 2024-04-24T23:35:20.950 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.921+0000 7f06e759e640 1 -- 172.21.15.157:0/2805660033 shutdown_connections 2024-04-24T23:35:20.950 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.921+0000 7f06e759e640 1 --2- 172.21.15.157:0/2805660033 >> v2:172.21.15.169:3300/0 conn(0x7f06e00073e0 0x7f06e000a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:20.950 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.921+0000 7f06e759e640 1 --2- 172.21.15.157:0/2805660033 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06e00df180 0x7f06e0006ea0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:20.950 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.921+0000 7f8de1241640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ddc06d960 0x7f8ddc15afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:20.951 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.921+0000 7f06e759e640 1 --2- 172.21.15.157:0/2805660033 >> v2:172.21.15.157:3300/0 conn(0x7f06e00dbce0 0x7f06e00474a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:20.951 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.922+0000 7f06e759e640 1 -- 172.21.15.157:0/2805660033 >> 172.21.15.157:0/2805660033 conn(0x7f06e0048250 msgr2=0x7f06e0048660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:20.951 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.922+0000 7ff382ad4640 1 -- 172.21.15.157:0/843756642 wait complete. 2024-04-24T23:35:20.951 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.922+0000 7f8de1241640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ddc06d960 0x7f8ddc15afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42940/0 (socket says 172.21.15.157:42940) 2024-04-24T23:35:20.951 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.922+0000 7f8de1241640 1 -- 172.21.15.157:0/440989145 learned_addr learned my addr 172.21.15.157:0/440989145 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:20.952 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.922+0000 7f8de1a42640 1 --2- 172.21.15.157:0/440989145 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ddc156e90 0x7f8ddc158c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:20.952 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.922+0000 7f8de0a40640 1 --2- 172.21.15.157:0/440989145 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ddc0c1c50 0x7f8ddc15cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:20.952 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.922+0000 7f06e759e640 1 -- 172.21.15.157:0/2805660033 shutdown_connections 2024-04-24T23:35:20.952 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.922+0000 7f06e759e640 1 -- 172.21.15.157:0/2805660033 wait complete. 2024-04-24T23:35:20.952 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.922+0000 7f78cf363640 1 Processor -- start 2024-04-24T23:35:20.952 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.922+0000 7f78cf363640 1 -- start start 2024-04-24T23:35:20.953 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.922+0000 7f8de1a42640 1 -- 172.21.15.157:0/440989145 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ddc0c1c50 msgr2=0x7f8ddc15cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:20.953 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.922+0000 7f8de1a42640 1 --2- 172.21.15.157:0/440989145 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ddc0c1c50 0x7f8ddc15cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:20.953 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.922+0000 7f8de1a42640 1 -- 172.21.15.157:0/440989145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ddc06d960 msgr2=0x7f8ddc15afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:20.953 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.922+0000 7f8de1a42640 1 --2- 172.21.15.157:0/440989145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ddc06d960 0x7f8ddc15afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:20.953 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.922+0000 7f8de1a42640 1 -- 172.21.15.157:0/440989145 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ddc15b4f0 con 0x7f8ddc156e90 2024-04-24T23:35:20.953 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.922+0000 7f8de1241640 1 --2- 172.21.15.157:0/440989145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ddc06d960 0x7f8ddc15afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:35:20.954 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.922+0000 7f92f6e88640 1 Processor -- start 2024-04-24T23:35:20.954 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.923+0000 7f92f6e88640 1 -- start start 2024-04-24T23:35:20.954 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.923+0000 7fe24b59e640 1 Processor -- start 2024-04-24T23:35:20.954 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.923+0000 7fe24b59e640 1 -- start start 2024-04-24T23:35:20.955 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.923+0000 7f78cf363640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78c004b140 0x7f78c003e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:20.955 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.923+0000 7f78cf363640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78c00d3b70 0x7f78c0044ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:20.955 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.924+0000 7f78cf363640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78c00d6740 0x7f78c003f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:20.955 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.924+0000 7f8f21638640 1 -- 172.21.15.157:0/3955102598 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f1c066ea0 msgr2=0x7f8f1c073300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:20.955 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.924+0000 7f8f21638640 1 --2- 172.21.15.157:0/3955102598 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f1c066ea0 0x7f8f1c073300 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8f100097d0 tx=0x7f8f1002a1c0 comp rx=0 tx=0).stop 2024-04-24T23:35:20.956 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.924+0000 7f78cd0ff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78c004b140 0x7f78c003e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:20.956 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.924+0000 7f78cd900640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78c00d6740 0x7f78c003f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:20.956 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.924+0000 7f78cd0ff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78c004b140 0x7f78c003e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.157:45174/0 (socket says 172.21.15.157:45174) 2024-04-24T23:35:20.956 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.924+0000 7f78cd900640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78c00d6740 0x7f78c003f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42948/0 (socket says 172.21.15.157:42948) 2024-04-24T23:35:20.956 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.924+0000 7f78cc8fe640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78c00d3b70 0x7f78c0044ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:20.956 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.924+0000 7f78cc8fe640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78c00d3b70 0x7f78c0044ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.157:52966/0 (socket says 172.21.15.157:52966) 2024-04-24T23:35:20.956 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.924+0000 7f92f6e88640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92f006b3d0 0x7f92f0151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:20.956 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.924+0000 7f92f6e88640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92f006f630 0x7f92f014a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:20.957 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.924+0000 7f8de1a42640 1 --2- 172.21.15.157:0/440989145 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ddc156e90 0x7f8ddc158c70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8dd400ca70 tx=0x7f8dd400cf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:20.957 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.924+0000 7ff382ad4640 1 Processor -- start 2024-04-24T23:35:20.957 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.924+0000 7ff382ad4640 1 -- start start 2024-04-24T23:35:20.957 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.924+0000 7f06e759e640 1 Processor -- start 2024-04-24T23:35:20.958 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.924+0000 7f78cd0ff640 1 -- 172.21.15.157:0/2078052207 learned_addr learned my addr 172.21.15.157:0/2078052207 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:20.958 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.924+0000 7f78cf363640 1 -- 172.21.15.157:0/2078052207 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f78c00d9960 con 0x7f78c00d6740 2024-04-24T23:35:20.958 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.924+0000 7f78cf363640 1 -- 172.21.15.157:0/2078052207 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f78c00da810 con 0x7f78c004b140 2024-04-24T23:35:20.958 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.924+0000 7f78cf363640 1 -- 172.21.15.157:0/2078052207 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f78c00075e0 con 0x7f78c00d3b70 2024-04-24T23:35:20.958 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.924+0000 7f92f6e88640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f92f00c6080 0x7f92f01ee700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:20.958 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.924+0000 7f78cd0ff640 1 -- 172.21.15.157:0/2078052207 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78c00d3b70 msgr2=0x7f78c0044ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:20.959 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.925+0000 7f78cd0ff640 1 --2- 172.21.15.157:0/2078052207 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78c00d3b70 0x7f78c0044ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:20.959 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.925+0000 7f78cd0ff640 1 -- 172.21.15.157:0/2078052207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78c00d6740 msgr2=0x7f78c003f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:20.959 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.925+0000 7f78cd0ff640 1 --2- 172.21.15.157:0/2078052207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78c00d6740 0x7f78c003f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:20.959 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.925+0000 7f92f4c24640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92f006b3d0 0x7f92f0151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:20.959 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.925+0000 7f92f6e88640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f92f0159080 con 0x7f92f006f630 2024-04-24T23:35:20.959 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.925+0000 7f92f6e88640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f92f00c7790 con 0x7f92f006b3d0 2024-04-24T23:35:20.959 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.925+0000 7f92f6e88640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f92f0159200 con 0x7f92f00c6080 2024-04-24T23:35:20.959 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.925+0000 7f92f4c24640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92f006b3d0 0x7f92f0151ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.157:45180/0 (socket says 172.21.15.157:45180) 2024-04-24T23:35:20.960 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.925+0000 7f92f4c24640 1 -- 172.21.15.157:0/2539671737 learned_addr learned my addr 172.21.15.157:0/2539671737 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:20.960 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.925+0000 7f78cd0ff640 1 -- 172.21.15.157:0/2078052207 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78c0044140 con 0x7f78c004b140 2024-04-24T23:35:20.960 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.925+0000 7f78cc8fe640 1 --2- 172.21.15.157:0/2078052207 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78c00d3b70 0x7f78c0044ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:35:20.960 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.925+0000 7f92effff640 1 --2- 172.21.15.157:0/2539671737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92f006f630 0x7f92f014a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:20.961 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.925+0000 7f78cd900640 1 --2- 172.21.15.157:0/2078052207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78c00d6740 0x7f78c003f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:35:20.961 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.925+0000 7f78cd0ff640 1 --2- 172.21.15.157:0/2078052207 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78c004b140 0x7f78c003e910 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f78b800aa30 tx=0x7f78b80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:20.961 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.924+0000 7f06e759e640 1 -- start start 2024-04-24T23:35:20.961 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.925+0000 7f92f4c24640 1 -- 172.21.15.157:0/2539671737 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f92f00c6080 msgr2=0x7f92f01ee700 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-24T23:35:20.961 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.925+0000 7f92f4c24640 1 --2- 172.21.15.157:0/2539671737 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f92f00c6080 0x7f92f01ee700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:20.961 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.925+0000 7f92f4c24640 1 -- 172.21.15.157:0/2539671737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92f006f630 msgr2=0x7f92f014a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:20.962 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.925+0000 7f92f4c24640 1 --2- 172.21.15.157:0/2539671737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92f006f630 0x7f92f014a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:20.962 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.925+0000 7f92f4c24640 1 -- 172.21.15.157:0/2539671737 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92f01ebd40 con 0x7f92f006b3d0 2024-04-24T23:35:20.962 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.925+0000 7f06e759e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06e00073e0 0x7f06e00dd210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:20.962 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.925+0000 7f92f4c24640 1 --2- 172.21.15.157:0/2539671737 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92f006b3d0 0x7f92f0151ac0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f92d800c400 tx=0x7f92d800c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:20.963 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.925+0000 7f06e759e640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06e00dbce0 0x7f06e016a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:20.963 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.925+0000 7f8f21638640 1 -- 172.21.15.157:0/3955102598 shutdown_connections 2024-04-24T23:35:20.963 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.925+0000 7f8f21638640 1 --2- 172.21.15.157:0/3955102598 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f1c15d190 0x7f8f1c15d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:20.963 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.925+0000 7f8f21638640 1 --2- 172.21.15.157:0/3955102598 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f1c066ea0 0x7f8f1c073300 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:20.963 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.925+0000 7f8f21638640 1 -- 172.21.15.157:0/3955102598 >> 172.21.15.157:0/3955102598 conn(0x7f8f1c0726c0 msgr2=0x7f8f1c072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:20.964 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.925+0000 7f78bdffb640 1 -- 172.21.15.157:0/2078052207 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b804a030 con 0x7f78c004b140 2024-04-24T23:35:20.964 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.925+0000 7f78bdffb640 1 -- 172.21.15.157:0/2078052207 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78b803de70 con 0x7f78c004b140 2024-04-24T23:35:20.964 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.925+0000 7f78bdffb640 1 -- 172.21.15.157:0/2078052207 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b803d2f0 con 0x7f78c004b140 2024-04-24T23:35:20.964 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.925+0000 7f06e759e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06e00df180 0x7f06e016a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:20.964 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.925+0000 7f78cf363640 1 -- 172.21.15.157:0/2078052207 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78c0043380 con 0x7f78c004b140 2024-04-24T23:35:20.965 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.925+0000 7f8dd1ffb640 1 -- 172.21.15.157:0/440989145 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd4017070 con 0x7f8ddc156e90 2024-04-24T23:35:20.965 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.925+0000 7fe24b59e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe24400fcf0 0x7fe24416bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:20.965 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.926+0000 7ff382ad4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff37c072e40 0x7ff37c0ad640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:20.966 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.926+0000 7f8dd1ffb640 1 -- 172.21.15.157:0/440989145 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dd400ab70 con 0x7f8ddc156e90 2024-04-24T23:35:20.966 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.926+0000 7f8dd1ffb640 1 -- 172.21.15.157:0/440989145 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd4012400 con 0x7f8ddc156e90 2024-04-24T23:35:20.967 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.926+0000 7f06e5d9b640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06e00dbce0 0x7f06e016a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:20.967 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.926+0000 7f06e5d9b640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06e00dbce0 0x7f06e016a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.157:53008/0 (socket says 172.21.15.157:53008) 2024-04-24T23:35:20.967 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.926+0000 7f06e5d9b640 1 -- 172.21.15.157:0/2014074655 learned_addr learned my addr 172.21.15.157:0/2014074655 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:20.967 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.926+0000 7f06e759e640 1 -- 172.21.15.157:0/2014074655 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f06e00dc0c0 con 0x7f06e00df180 2024-04-24T23:35:20.967 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.926+0000 7f8f21638640 1 -- 172.21.15.157:0/3955102598 shutdown_connections 2024-04-24T23:35:20.967 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.926+0000 7ff382ad4640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff37c0c30d0 0x7ff37c152750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:20.968 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.926+0000 7f06e759e640 1 -- 172.21.15.157:0/2014074655 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f06e0047d80 con 0x7f06e00073e0 2024-04-24T23:35:20.968 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.926+0000 7f8f21638640 1 -- 172.21.15.157:0/3955102598 wait complete. 2024-04-24T23:35:20.968 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.926+0000 7f06e759e640 1 -- 172.21.15.157:0/2014074655 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f06e00df5c0 con 0x7f06e00dbce0 2024-04-24T23:35:20.969 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.925+0000 7fe24b59e640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe244050720 0x7fe2440d9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:20.969 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.925+0000 7fe24b59e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2440d7920 0x7fe244032e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:20.969 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.925+0000 7fe24b59e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe24400f4c0 con 0x7fe2440d7920 2024-04-24T23:35:20.969 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.925+0000 7fe24b59e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe24404c2a0 con 0x7fe24400fcf0 2024-04-24T23:35:20.969 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.925+0000 7fe24b59e640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe24400f640 con 0x7fe244050720 2024-04-24T23:35:20.969 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.925+0000 7fe24a59c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe24400fcf0 0x7fe24416bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:20.969 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.925+0000 7fe249d9b640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe244050720 0x7fe2440d9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:20.969 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.925+0000 7fe249d9b640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe244050720 0x7fe2440d9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.157:52994/0 (socket says 172.21.15.157:52994) 2024-04-24T23:35:20.969 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.926+0000 7fe249d9b640 1 -- 172.21.15.157:0/2494099883 learned_addr learned my addr 172.21.15.157:0/2494099883 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:20.969 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.926+0000 7fe24ad9d640 1 --2- 172.21.15.157:0/2494099883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2440d7920 0x7fe244032e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:20.970 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.926+0000 7f78cf363640 1 -- 172.21.15.157:0/2078052207 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78c0043940 con 0x7f78c004b140 2024-04-24T23:35:20.970 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.926+0000 7ff382ad4640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff37c155600 0x7ff37c0aae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:20.970 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.926+0000 7ff380870640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff37c072e40 0x7ff37c0ad640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:20.970 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.926+0000 7f06e6d9d640 1 --2- 172.21.15.157:0/2014074655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06e00df180 0x7f06e016a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:20.971 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.926+0000 7ff380870640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff37c072e40 0x7ff37c0ad640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42980/0 (socket says 172.21.15.157:42980) 2024-04-24T23:35:20.971 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.926+0000 7ff380870640 1 -- 172.21.15.157:0/93140772 learned_addr learned my addr 172.21.15.157:0/93140772 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:20.971 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.926+0000 7ff381071640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff37c155600 0x7ff37c0aae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:20.971 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.926+0000 7ff37bfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff37c0c30d0 0x7ff37c152750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:20.971 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.926+0000 7ff382ad4640 1 -- 172.21.15.157:0/93140772 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff37c073220 con 0x7ff37c072e40 2024-04-24T23:35:20.972 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.926+0000 7f92ed7fa640 1 -- 172.21.15.157:0/2539671737 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d8017070 con 0x7f92f006b3d0 2024-04-24T23:35:20.972 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.926+0000 7f92ed7fa640 1 -- 172.21.15.157:0/2539671737 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92d80052d0 con 0x7f92f006b3d0 2024-04-24T23:35:20.972 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.926+0000 7f92ed7fa640 1 -- 172.21.15.157:0/2539671737 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d8003810 con 0x7f92f006b3d0 2024-04-24T23:35:20.972 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.926+0000 7f92f6e88640 1 -- 172.21.15.157:0/2539671737 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92f006e4c0 con 0x7f92f006b3d0 2024-04-24T23:35:20.972 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.926+0000 7f92f6e88640 1 -- 172.21.15.157:0/2539671737 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92f006efb0 con 0x7f92f006b3d0 2024-04-24T23:35:20.973 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.927+0000 7f06e659c640 1 --2- 172.21.15.157:0/2014074655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06e00073e0 0x7f06e00dd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:20.973 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.927+0000 7f06e5d9b640 1 -- 172.21.15.157:0/2014074655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06e00073e0 msgr2=0x7f06e00dd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:20.973 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.927+0000 7f06e5d9b640 1 --2- 172.21.15.157:0/2014074655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06e00073e0 0x7f06e00dd210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:20.973 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.927+0000 7f06e5d9b640 1 -- 172.21.15.157:0/2014074655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06e00df180 msgr2=0x7f06e016a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:20.973 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.927+0000 7f06e5d9b640 1 --2- 172.21.15.157:0/2014074655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06e00df180 0x7f06e016a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:20.973 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.927+0000 7f06e5d9b640 1 -- 172.21.15.157:0/2014074655 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06e0035b60 con 0x7f06e00dbce0 2024-04-24T23:35:20.973 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.928+0000 7f8f21638640 1 Processor -- start 2024-04-24T23:35:20.974 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.928+0000 7f78bdffb640 1 -- 172.21.15.157:0/2078052207 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f78b804d070 con 0x7f78c004b140 2024-04-24T23:35:20.974 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.928+0000 7f8f21638640 1 -- start start 2024-04-24T23:35:20.974 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.928+0000 7f06e5d9b640 1 --2- 172.21.15.157:0/2014074655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06e00dbce0 0x7f06e016a4a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f06d8009930 tx=0x7f06d80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:20.975 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.928+0000 7f92f6e88640 1 -- 172.21.15.157:0/2539671737 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92f00688d0 con 0x7f92f006b3d0 2024-04-24T23:35:20.975 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.928+0000 7ff382ad4640 1 -- 172.21.15.157:0/93140772 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff37c0c6bf0 con 0x7ff37c0c30d0 2024-04-24T23:35:20.975 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.928+0000 7ff382ad4640 1 -- 172.21.15.157:0/93140772 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff37c0c40b0 con 0x7ff37c155600 2024-04-24T23:35:20.975 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.926+0000 7fe249d9b640 1 -- 172.21.15.157:0/2494099883 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe24400fcf0 msgr2=0x7fe24416bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:20.975 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.926+0000 7fe249d9b640 1 --2- 172.21.15.157:0/2494099883 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe24400fcf0 0x7fe24416bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:20.976 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.929+0000 7f06d6ffd640 1 -- 172.21.15.157:0/2014074655 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d8043070 con 0x7f06e00dbce0 2024-04-24T23:35:20.976 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.926+0000 7fe249d9b640 1 -- 172.21.15.157:0/2494099883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2440d7920 msgr2=0x7fe244032e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:20.976 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.926+0000 7fe249d9b640 1 --2- 172.21.15.157:0/2494099883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2440d7920 0x7fe244032e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:20.976 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.926+0000 7fe249d9b640 1 -- 172.21.15.157:0/2494099883 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe24416c310 con 0x7fe244050720 2024-04-24T23:35:20.977 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.929+0000 7f06e759e640 1 -- 172.21.15.157:0/2014074655 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06e0030d30 con 0x7f06e00dbce0 2024-04-24T23:35:20.977 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.927+0000 7fe249d9b640 1 --2- 172.21.15.157:0/2494099883 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe244050720 0x7fe2440d9400 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe240032910 tx=0x7fe2400046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:20.977 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.929+0000 7ff381071640 1 -- 172.21.15.157:0/93140772 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff37c0c30d0 msgr2=0x7ff37c152750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:20.977 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.929+0000 7ff381071640 1 --2- 172.21.15.157:0/93140772 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff37c0c30d0 0x7ff37c152750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:20.977 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.929+0000 7ff381071640 1 -- 172.21.15.157:0/93140772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff37c072e40 msgr2=0x7ff37c0ad640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:20.977 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.929+0000 7ff381071640 1 --2- 172.21.15.157:0/93140772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff37c072e40 0x7ff37c0ad640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:20.978 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.928+0000 7f78bdffb640 1 --2- 172.21.15.157:0/2078052207 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f78ac06bf00 0x7f78ac06e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:20.978 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.929+0000 7f78cf363640 1 -- 172.21.15.157:0/2078052207 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78c00053c0 con 0x7f78c004b140 2024-04-24T23:35:20.978 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.929+0000 7f78bdffb640 1 -- 172.21.15.157:0/2078052207 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f78b80b4270 con 0x7f78c004b140 2024-04-24T23:35:20.978 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.929+0000 7f06e759e640 1 -- 172.21.15.157:0/2014074655 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f06e0030fd0 con 0x7f06e00dbce0 2024-04-24T23:35:20.979 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.927+0000 7fe23affd640 1 -- 172.21.15.157:0/2494099883 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe240048070 con 0x7fe244050720 2024-04-24T23:35:20.979 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.927+0000 7fe23affd640 1 -- 172.21.15.157:0/2494099883 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe240003e50 con 0x7fe244050720 2024-04-24T23:35:20.979 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.927+0000 7fe23affd640 1 -- 172.21.15.157:0/2494099883 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2400387f0 con 0x7fe244050720 2024-04-24T23:35:20.979 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.929+0000 7ff381071640 1 -- 172.21.15.157:0/93140772 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff37c0adb80 con 0x7ff37c155600 2024-04-24T23:35:20.979 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.928+0000 7fe24b59e640 1 -- 172.21.15.157:0/2494099883 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe24416c5f0 con 0x7fe244050720 2024-04-24T23:35:20.980 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.927+0000 7f8de34a5640 1 -- 172.21.15.157:0/440989145 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ddc15d6e0 con 0x7f8ddc156e90 2024-04-24T23:35:20.980 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.928+0000 7f8de34a5640 1 -- 172.21.15.157:0/440989145 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ddc0a9920 con 0x7f8ddc156e90 2024-04-24T23:35:20.980 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.929+0000 7f8f21638640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f1c066ea0 0x7f8f1c06d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:20.980 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.929+0000 7f8de34a5640 1 -- 172.21.15.157:0/440989145 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ddc0c47e0 con 0x7f8ddc156e90 2024-04-24T23:35:20.981 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.928+0000 7fe24b59e640 1 -- 172.21.15.157:0/2494099883 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe24416cbb0 con 0x7fe244050720 2024-04-24T23:35:20.981 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.929+0000 7f8f21638640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f1c156ee0 0x7f8f1c1faea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:20.981 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.929+0000 7ff381071640 1 --2- 172.21.15.157:0/93140772 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff37c155600 0x7ff37c0aae30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff37c0c7570 tx=0x7ff374004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:20.981 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.929+0000 7fe238ff9640 1 -- 172.21.15.157:0/2494099883 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe244005390 con 0x7fe244050720 2024-04-24T23:35:20.982 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.929+0000 7f8f21638640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f1c15d190 0x7f8f1c1fb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:20.982 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.930+0000 7f8f1affd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f1c066ea0 0x7f8f1c06d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:20.982 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.930+0000 7f8f1affd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f1c066ea0 0x7f8f1c06d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42994/0 (socket says 172.21.15.157:42994) 2024-04-24T23:35:20.982 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.930+0000 7f78cc8fe640 1 --2- 172.21.15.157:0/2078052207 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f78ac06bf00 0x7f78ac06e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:20.983 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.930+0000 7f8f21638640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1c0c7ea0 con 0x7f8f1c066ea0 2024-04-24T23:35:20.983 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.930+0000 7f8f21638640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1c0c4f30 con 0x7f8f1c15d190 2024-04-24T23:35:20.983 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.930+0000 7f8f21638640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8f1c0c8020 con 0x7f8f1c156ee0 2024-04-24T23:35:20.983 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.930+0000 7f8f1affd640 1 -- 172.21.15.157:0/3980965869 learned_addr learned my addr 172.21.15.157:0/3980965869 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:20.983 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.930+0000 7f8f1a7fc640 1 --2- 172.21.15.157:0/3980965869 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f1c156ee0 0x7f8f1c1faea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:20.983 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.929+0000 7f06d6ffd640 1 -- 172.21.15.157:0/2014074655 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06d8038040 con 0x7f06e00dbce0 2024-04-24T23:35:20.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.930+0000 7f8f1bfff640 1 --2- 172.21.15.157:0/3980965869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f1c15d190 0x7f8f1c1fb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:20.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.930+0000 7f06d6ffd640 1 -- 172.21.15.157:0/2014074655 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d8037050 con 0x7f06e00dbce0 2024-04-24T23:35:20.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.930+0000 7f8f1a7fc640 1 -- 172.21.15.157:0/3980965869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f1c15d190 msgr2=0x7f8f1c1fb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:20.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.930+0000 7f8f1a7fc640 1 --2- 172.21.15.157:0/3980965869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f1c15d190 0x7f8f1c1fb3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:20.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.930+0000 7f8f1a7fc640 1 -- 172.21.15.157:0/3980965869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f1c066ea0 msgr2=0x7f8f1c06d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:20.985 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.931+0000 7ff3797fa640 1 -- 172.21.15.157:0/93140772 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff374049070 con 0x7ff37c155600 2024-04-24T23:35:20.985 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.931+0000 7f8f1a7fc640 1 --2- 172.21.15.157:0/3980965869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f1c066ea0 0x7f8f1c06d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:20.985 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.931+0000 7f06e759e640 1 -- 172.21.15.157:0/2014074655 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06e0005b80 con 0x7f06e00dbce0 2024-04-24T23:35:20.985 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.931+0000 7f8f1a7fc640 1 -- 172.21.15.157:0/3980965869 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f1c0ae140 con 0x7f8f1c156ee0 2024-04-24T23:35:20.986 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.931+0000 7ff3797fa640 1 -- 172.21.15.157:0/93140772 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff374032440 con 0x7ff37c155600 2024-04-24T23:35:20.986 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.930+0000 7f8dd1ffb640 1 -- 172.21.15.157:0/440989145 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8dd401a070 con 0x7f8ddc156e90 2024-04-24T23:35:20.986 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.930+0000 7fe23affd640 1 -- 172.21.15.157:0/2494099883 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe240053060 con 0x7fe244050720 2024-04-24T23:35:20.987 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.931+0000 7f8f1a7fc640 1 --2- 172.21.15.157:0/3980965869 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f1c156ee0 0x7f8f1c1faea0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8f10031000 tx=0x7f8f100046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:20.987 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.931+0000 7f8dd1ffb640 1 --2- 172.21.15.157:0/440989145 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f8dc006bf00 0x7f8dc006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:20.987 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.931+0000 7fe23affd640 1 --2- 172.21.15.157:0/2494099883 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fe22c06bf00 0x7fe22c06e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:20.987 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.931+0000 7f8dd1ffb640 1 -- 172.21.15.157:0/440989145 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f8dd408adf0 con 0x7f8ddc156e90 2024-04-24T23:35:20.988 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.931+0000 7fe23affd640 1 -- 172.21.15.157:0/2494099883 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe2400b3a90 con 0x7fe244050720 2024-04-24T23:35:20.988 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.931+0000 7f92ed7fa640 1 -- 172.21.15.157:0/2539671737 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f92d801a070 con 0x7f92f006b3d0 2024-04-24T23:35:20.988 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.931+0000 7f92ed7fa640 1 --2- 172.21.15.157:0/2539671737 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f92d006bf30 0x7f92d006e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:20.988 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.931+0000 7f92ed7fa640 1 -- 172.21.15.157:0/2539671737 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f92d808a400 con 0x7f92f006b3d0 2024-04-24T23:35:20.988 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.931+0000 7f8de1241640 1 --2- 172.21.15.157:0/440989145 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f8dc006bf00 0x7f8dc006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:20.989 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.931+0000 7ff3797fa640 1 -- 172.21.15.157:0/93140772 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff374036050 con 0x7ff37c155600 2024-04-24T23:35:20.989 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.931+0000 7ff382ad4640 1 -- 172.21.15.157:0/93140772 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff37c14e900 con 0x7ff37c155600 2024-04-24T23:35:20.989 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.931+0000 7ff382ad4640 1 -- 172.21.15.157:0/93140772 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff37c150700 con 0x7ff37c155600 2024-04-24T23:35:20.989 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.932+0000 7f78cc8fe640 1 --2- 172.21.15.157:0/2078052207 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f78ac06bf00 0x7f78ac06e3c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f78c80502b0 tx=0x7f78c8067eb0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:20.989 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.932+0000 7f92effff640 1 --2- 172.21.15.157:0/2539671737 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f92d006bf30 0x7f92d006e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:20.990 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.932+0000 7fe24a59c640 1 --2- 172.21.15.157:0/2494099883 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fe22c06bf00 0x7fe22c06e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:20.990 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.932+0000 7f06d6ffd640 1 -- 172.21.15.157:0/2014074655 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f06d8046070 con 0x7f06e00dbce0 2024-04-24T23:35:20.990 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.932+0000 7f92effff640 1 --2- 172.21.15.157:0/2539671737 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f92d006bf30 0x7f92d006e3f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f92e0034000 tx=0x7f92e002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:20.991 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.932+0000 7f06d6ffd640 1 --2- 172.21.15.157:0/2014074655 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f06c806be80 0x7f06c806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:20.991 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.932+0000 7fe24a59c640 1 --2- 172.21.15.157:0/2494099883 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fe22c06bf00 0x7fe22c06e3c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe2440d9c10 tx=0x7fe23c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:20.991 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.932+0000 7f06d6ffd640 1 -- 172.21.15.157:0/2014074655 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f06d80ab0b0 con 0x7f06e00dbce0 2024-04-24T23:35:20.991 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.932+0000 7f06e659c640 1 --2- 172.21.15.157:0/2014074655 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f06c806be80 0x7f06c806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:20.992 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.932+0000 7f8de1241640 1 --2- 172.21.15.157:0/440989145 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f8dc006bf00 0x7f8dc006e3c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc02a1e0 tx=0x7f8dcc02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:20.992 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.933+0000 7f06e659c640 1 --2- 172.21.15.157:0/2014074655 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f06c806be80 0x7f06c806e340 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f06dc006fd0 tx=0x7f06dc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:20.992 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.933+0000 7ff382ad4640 1 -- 172.21.15.157:0/93140772 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff37c0688d0 con 0x7ff37c155600 2024-04-24T23:35:20.993 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.938+0000 7f78bdffb640 1 -- 172.21.15.157:0/2078052207 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f78b8054070 con 0x7f78c004b140 2024-04-24T23:35:20.993 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.941+0000 7f8dd1ffb640 1 -- 172.21.15.157:0/440989145 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8ddc0c47e0 con 0x7f8ddc156e90 2024-04-24T23:35:20.993 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.948+0000 7f92ed7fa640 1 -- 172.21.15.157:0/2539671737 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f92d8058490 con 0x7f92f006b3d0 2024-04-24T23:35:20.993 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.949+0000 7fe23affd640 1 -- 172.21.15.157:0/2494099883 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe240082290 con 0x7fe244050720 2024-04-24T23:35:20.994 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.953+0000 7f8f037fe640 1 -- 172.21.15.157:0/3980965869 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f10037050 con 0x7f8f1c156ee0 2024-04-24T23:35:20.994 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.953+0000 7f06d6ffd640 1 -- 172.21.15.157:0/2014074655 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f06e0005b80 con 0x7f06e00dbce0 2024-04-24T23:35:20.995 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.953+0000 7f8f037fe640 1 -- 172.21.15.157:0/3980965869 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f10043070 con 0x7f8f1c156ee0 2024-04-24T23:35:20.995 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.953+0000 7f8f037fe640 1 -- 172.21.15.157:0/3980965869 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1003cb30 con 0x7f8f1c156ee0 2024-04-24T23:35:20.995 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.954+0000 7f8f21638640 1 -- 172.21.15.157:0/3980965869 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f1c0ae420 con 0x7f8f1c156ee0 2024-04-24T23:35:20.995 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.954+0000 7f8f21638640 1 -- 172.21.15.157:0/3980965869 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f1c06bcc0 con 0x7f8f1c156ee0 2024-04-24T23:35:20.995 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.955+0000 7ff3797fa640 1 -- 172.21.15.157:0/93140772 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff37404c070 con 0x7ff37c155600 2024-04-24T23:35:20.996 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.955+0000 7f8f21638640 1 -- 172.21.15.157:0/3980965869 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f1c0688d0 con 0x7f8f1c156ee0 2024-04-24T23:35:20.996 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.955+0000 7ff3797fa640 1 --2- 172.21.15.157:0/93140772 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7ff36806bf00 0x7ff36806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:20.996 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.956+0000 7ff380870640 1 --2- 172.21.15.157:0/93140772 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7ff36806bf00 0x7ff36806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:20.996 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.956+0000 7ff3797fa640 1 -- 172.21.15.157:0/93140772 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ff3740afeb0 con 0x7ff37c155600 2024-04-24T23:35:20.996 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.956+0000 7ff380870640 1 --2- 172.21.15.157:0/93140772 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7ff36806bf00 0x7ff36806e3c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff36c006fd0 tx=0x7ff36c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:20.996 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.958+0000 7ff3797fa640 1 -- 172.21.15.157:0/93140772 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff374053080 con 0x7ff37c155600 2024-04-24T23:35:20.997 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.961+0000 7f8f037fe640 1 -- 172.21.15.157:0/3980965869 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8f1003ccd0 con 0x7f8f1c156ee0 2024-04-24T23:35:20.997 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.961+0000 7f8f037fe640 1 --2- 172.21.15.157:0/3980965869 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f8f0806bf30 0x7f8f0806e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:20.997 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.961+0000 7f8f037fe640 1 -- 172.21.15.157:0/3980965869 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f8f100ac720 con 0x7f8f1c156ee0 2024-04-24T23:35:20.997 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.962+0000 7f8f1affd640 1 --2- 172.21.15.157:0/3980965869 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f8f0806bf30 0x7f8f0806e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:20.997 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.963+0000 7f8f1affd640 1 --2- 172.21.15.157:0/3980965869 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f8f0806bf30 0x7f8f0806e3f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8f14007b10 tx=0x7f8f14007a60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:20.998 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:20.963+0000 7f8f037fe640 1 -- 172.21.15.157:0/3980965869 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8f1007a7b0 con 0x7f8f1c156ee0 2024-04-24T23:35:21.270 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.270+0000 7f957d98c640 1 -- 172.21.15.157:0/2378755558 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f9558005580 con 0x7f957004adc0 2024-04-24T23:35:21.274 INFO:teuthology.orchestra.run.smithi157.stdout:60129542152 2024-04-24T23:35:21.275 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.271+0000 7f95757fa640 1 -- 172.21.15.157:0/2378755558 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f95780f88a0 con 0x7f957004adc0 2024-04-24T23:35:21.276 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.274+0000 7f9562ffd640 1 -- 172.21.15.157:0/2378755558 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f955c06bf30 msgr2=0x7f955c06e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:21.276 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.274+0000 7f9562ffd640 1 --2- 172.21.15.157:0/2378755558 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f955c06bf30 0x7f955c06e3f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f956c003cc0 tx=0x7f956c0079b0 comp rx=0 tx=0).stop 2024-04-24T23:35:21.276 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.274+0000 7f9562ffd640 1 -- 172.21.15.157:0/2378755558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f957004adc0 msgr2=0x7f9570174600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:21.276 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.274+0000 7f9562ffd640 1 --2- 172.21.15.157:0/2378755558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f957004adc0 0x7f9570174600 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f957806a550 tx=0x7f957806b090 comp rx=0 tx=0).stop 2024-04-24T23:35:21.276 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.274+0000 7f9562ffd640 1 -- 172.21.15.157:0/2378755558 shutdown_connections 2024-04-24T23:35:21.276 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.275+0000 7f9562ffd640 1 --2- 172.21.15.157:0/2378755558 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f955c06bf30 0x7f955c06e3f0 secure :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f956c003cc0 tx=0x7f956c0079b0 comp rx=0 tx=0).stop 2024-04-24T23:35:21.276 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.275+0000 7f9562ffd640 1 --2- 172.21.15.157:0/2378755558 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9570042260 0x7f95700445f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:21.276 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.275+0000 7f9562ffd640 1 --2- 172.21.15.157:0/2378755558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f957004adc0 0x7f9570174600 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:21.276 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.275+0000 7f9562ffd640 1 --2- 172.21.15.157:0/2378755558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f957004a9e0 0x7f95700453c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:21.276 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.275+0000 7f9562ffd640 1 -- 172.21.15.157:0/2378755558 >> 172.21.15.157:0/2378755558 conn(0x7f9570049340 msgr2=0x7f95700081b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:21.277 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.276+0000 7f9562ffd640 1 -- 172.21.15.157:0/2378755558 shutdown_connections 2024-04-24T23:35:21.277 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.276+0000 7f9562ffd640 1 -- 172.21.15.157:0/2378755558 wait complete. 2024-04-24T23:35:21.293 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542152 for osd.7 2024-04-24T23:35:21.293 DEBUG:teuthology.parallel:result is None 2024-04-24T23:35:21.324 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.324+0000 7ff382ad4640 1 -- 172.21.15.157:0/93140772 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7ff37c150dc0 con 0x7ff37c155600 2024-04-24T23:35:21.325 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.324+0000 7ff3797fa640 1 -- 172.21.15.157:0/93140772 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff37c0adb80 con 0x7ff37c155600 2024-04-24T23:35:21.325 INFO:teuthology.orchestra.run.smithi157.stdout:60129542152 2024-04-24T23:35:21.328 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.327+0000 7ff382ad4640 1 -- 172.21.15.157:0/93140772 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7ff36806bf00 msgr2=0x7ff36806e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:21.328 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.327+0000 7ff382ad4640 1 --2- 172.21.15.157:0/93140772 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7ff36806bf00 0x7ff36806e3c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff36c006fd0 tx=0x7ff36c008040 comp rx=0 tx=0).stop 2024-04-24T23:35:21.328 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.327+0000 7ff382ad4640 1 -- 172.21.15.157:0/93140772 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff37c155600 msgr2=0x7ff37c0aae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:21.328 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.327+0000 7ff382ad4640 1 --2- 172.21.15.157:0/93140772 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff37c155600 0x7ff37c0aae30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff37c0c7570 tx=0x7ff374004a40 comp rx=0 tx=0).stop 2024-04-24T23:35:21.329 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.327+0000 7ff382ad4640 1 -- 172.21.15.157:0/93140772 shutdown_connections 2024-04-24T23:35:21.329 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.327+0000 7ff382ad4640 1 --2- 172.21.15.157:0/93140772 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7ff36806bf00 0x7ff36806e3c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:21.329 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.327+0000 7ff382ad4640 1 --2- 172.21.15.157:0/93140772 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff37c155600 0x7ff37c0aae30 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:21.329 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.327+0000 7ff382ad4640 1 --2- 172.21.15.157:0/93140772 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff37c0c30d0 0x7ff37c152750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:21.329 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.327+0000 7ff382ad4640 1 --2- 172.21.15.157:0/93140772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff37c072e40 0x7ff37c0ad640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:21.329 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.327+0000 7ff382ad4640 1 -- 172.21.15.157:0/93140772 >> 172.21.15.157:0/93140772 conn(0x7ff37c06c270 msgr2=0x7ff37c06c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:21.329 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.327+0000 7ff382ad4640 1 -- 172.21.15.157:0/93140772 shutdown_connections 2024-04-24T23:35:21.329 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.327+0000 7ff382ad4640 1 -- 172.21.15.157:0/93140772 wait complete. 2024-04-24T23:35:21.337 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.337+0000 7f8de34a5640 1 -- 172.21.15.157:0/440989145 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f8ddc0681c0 con 0x7f8ddc156e90 2024-04-24T23:35:21.339 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542152 for osd.0 2024-04-24T23:35:21.339 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.338+0000 7f8dd1ffb640 1 -- 172.21.15.157:0/440989145 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8dd4058e80 con 0x7f8ddc156e90 2024-04-24T23:35:21.339 INFO:teuthology.orchestra.run.smithi157.stdout:55834574857 2024-04-24T23:35:21.340 DEBUG:teuthology.parallel:result is None 2024-04-24T23:35:21.340 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.338+0000 7f78cf363640 1 -- 172.21.15.157:0/2078052207 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f78c0030ea0 con 0x7f78c004b140 2024-04-24T23:35:21.340 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.339+0000 7f78bdffb640 1 -- 172.21.15.157:0/2078052207 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f78b8082830 con 0x7f78c004b140 2024-04-24T23:35:21.340 INFO:teuthology.orchestra.run.smithi157.stdout:60129542152 2024-04-24T23:35:21.342 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.341+0000 7f8de34a5640 1 -- 172.21.15.157:0/440989145 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f8dc006bf00 msgr2=0x7f8dc006e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:21.342 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.341+0000 7f8de34a5640 1 --2- 172.21.15.157:0/440989145 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f8dc006bf00 0x7f8dc006e3c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc02a1e0 tx=0x7f8dcc02e000 comp rx=0 tx=0).stop 2024-04-24T23:35:21.342 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.341+0000 7f8de34a5640 1 -- 172.21.15.157:0/440989145 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ddc156e90 msgr2=0x7f8ddc158c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:21.342 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.341+0000 7f8de34a5640 1 --2- 172.21.15.157:0/440989145 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ddc156e90 0x7f8ddc158c70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8dd400ca70 tx=0x7f8dd400cf30 comp rx=0 tx=0).stop 2024-04-24T23:35:21.343 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.342+0000 7f8de34a5640 1 -- 172.21.15.157:0/440989145 shutdown_connections 2024-04-24T23:35:21.343 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.342+0000 7f8de34a5640 1 --2- 172.21.15.157:0/440989145 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f8dc006bf00 0x7f8dc006e3c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:21.343 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.342+0000 7f8de34a5640 1 --2- 172.21.15.157:0/440989145 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ddc156e90 0x7f8ddc158c70 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:21.343 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.342+0000 7f8de34a5640 1 --2- 172.21.15.157:0/440989145 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ddc0c1c50 0x7f8ddc15cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:21.343 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.342+0000 7f8de34a5640 1 --2- 172.21.15.157:0/440989145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ddc06d960 0x7f8ddc15afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:21.343 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.342+0000 7f8de34a5640 1 -- 172.21.15.157:0/440989145 >> 172.21.15.157:0/440989145 conn(0x7f8ddc0a8690 msgr2=0x7f8ddc06c410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:21.343 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.342+0000 7f8de34a5640 1 -- 172.21.15.157:0/440989145 shutdown_connections 2024-04-24T23:35:21.343 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.342+0000 7f8de34a5640 1 -- 172.21.15.157:0/440989145 wait complete. 2024-04-24T23:35:21.344 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.342+0000 7f78cf363640 1 -- 172.21.15.157:0/2078052207 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f78ac06bf00 msgr2=0x7f78ac06e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:21.344 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.342+0000 7f78cf363640 1 --2- 172.21.15.157:0/2078052207 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f78ac06bf00 0x7f78ac06e3c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f78c80502b0 tx=0x7f78c8067eb0 comp rx=0 tx=0).stop 2024-04-24T23:35:21.344 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.342+0000 7f78cf363640 1 -- 172.21.15.157:0/2078052207 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78c004b140 msgr2=0x7f78c003e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:21.344 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.342+0000 7f78cf363640 1 --2- 172.21.15.157:0/2078052207 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78c004b140 0x7f78c003e910 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f78b800aa30 tx=0x7f78b80046c0 comp rx=0 tx=0).stop 2024-04-24T23:35:21.344 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.342+0000 7f78cf363640 1 -- 172.21.15.157:0/2078052207 shutdown_connections 2024-04-24T23:35:21.345 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.342+0000 7f78cf363640 1 --2- 172.21.15.157:0/2078052207 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f78ac06bf00 0x7f78ac06e3c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:21.345 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.342+0000 7f78cf363640 1 --2- 172.21.15.157:0/2078052207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78c00d6740 0x7f78c003f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:21.345 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.342+0000 7f78cf363640 1 --2- 172.21.15.157:0/2078052207 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78c00d3b70 0x7f78c0044ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:21.345 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.342+0000 7f78cf363640 1 --2- 172.21.15.157:0/2078052207 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78c004b140 0x7f78c003e910 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:21.345 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.342+0000 7f78cf363640 1 -- 172.21.15.157:0/2078052207 >> 172.21.15.157:0/2078052207 conn(0x7f78c0048280 msgr2=0x7f78c00d6bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:21.345 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.342+0000 7f78cf363640 1 -- 172.21.15.157:0/2078052207 shutdown_connections 2024-04-24T23:35:21.345 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.342+0000 7f78cf363640 1 -- 172.21.15.157:0/2078052207 wait complete. 2024-04-24T23:35:21.357 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.357+0000 7fe238ff9640 1 -- 172.21.15.157:0/2494099883 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fe244049ec0 con 0x7fe244050720 2024-04-24T23:35:21.358 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.357+0000 7fe23affd640 1 -- 172.21.15.157:0/2494099883 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe240081b90 con 0x7fe244050720 2024-04-24T23:35:21.358 INFO:teuthology.orchestra.run.smithi157.stdout:60129542152 2024-04-24T23:35:21.359 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542152 for osd.4 2024-04-24T23:35:21.360 DEBUG:teuthology.parallel:result is None 2024-04-24T23:35:21.360 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.3 2024-04-24T23:35:21.361 DEBUG:teuthology.parallel:result is None 2024-04-24T23:35:21.361 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.360+0000 7fe238ff9640 1 -- 172.21.15.157:0/2494099883 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fe22c06bf00 msgr2=0x7fe22c06e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:21.361 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.360+0000 7fe238ff9640 1 --2- 172.21.15.157:0/2494099883 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fe22c06bf00 0x7fe22c06e3c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe2440d9c10 tx=0x7fe23c009040 comp rx=0 tx=0).stop 2024-04-24T23:35:21.361 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.360+0000 7fe238ff9640 1 -- 172.21.15.157:0/2494099883 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe244050720 msgr2=0x7fe2440d9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:21.362 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.360+0000 7fe238ff9640 1 --2- 172.21.15.157:0/2494099883 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe244050720 0x7fe2440d9400 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe240032910 tx=0x7fe2400046c0 comp rx=0 tx=0).stop 2024-04-24T23:35:21.362 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.360+0000 7fe238ff9640 1 -- 172.21.15.157:0/2494099883 shutdown_connections 2024-04-24T23:35:21.362 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.360+0000 7fe238ff9640 1 --2- 172.21.15.157:0/2494099883 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fe22c06bf00 0x7fe22c06e3c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:21.362 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.360+0000 7fe238ff9640 1 --2- 172.21.15.157:0/2494099883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2440d7920 0x7fe244032e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:21.362 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.360+0000 7fe238ff9640 1 --2- 172.21.15.157:0/2494099883 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe244050720 0x7fe2440d9400 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:21.362 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.360+0000 7fe238ff9640 1 --2- 172.21.15.157:0/2494099883 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe24400fcf0 0x7fe24416bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:21.362 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.360+0000 7fe238ff9640 1 -- 172.21.15.157:0/2494099883 >> 172.21.15.157:0/2494099883 conn(0x7fe24404fc00 msgr2=0x7fe24402fcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:21.362 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.360+0000 7fe238ff9640 1 -- 172.21.15.157:0/2494099883 shutdown_connections 2024-04-24T23:35:21.362 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.360+0000 7fe238ff9640 1 -- 172.21.15.157:0/2494099883 wait complete. 2024-04-24T23:35:21.366 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.365+0000 7f8f21638640 1 -- 172.21.15.157:0/3980965869 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f8f1c0c8590 con 0x7f8f1c156ee0 2024-04-24T23:35:21.366 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.366+0000 7f8f037fe640 1 -- 172.21.15.157:0/3980965869 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8f1007eab0 con 0x7f8f1c156ee0 2024-04-24T23:35:21.366 INFO:teuthology.orchestra.run.smithi157.stdout:60129542153 2024-04-24T23:35:21.369 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.369+0000 7f92f6e88640 1 -- 172.21.15.157:0/2539671737 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f92f00adc80 con 0x7f92f006b3d0 2024-04-24T23:35:21.370 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.369+0000 7f8f21638640 1 -- 172.21.15.157:0/3980965869 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f8f0806bf30 msgr2=0x7f8f0806e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:21.370 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.369+0000 7f8f21638640 1 --2- 172.21.15.157:0/3980965869 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f8f0806bf30 0x7f8f0806e3f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8f14007b10 tx=0x7f8f14007a60 comp rx=0 tx=0).stop 2024-04-24T23:35:21.370 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.369+0000 7f8f21638640 1 -- 172.21.15.157:0/3980965869 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f1c156ee0 msgr2=0x7f8f1c1faea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:21.370 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.369+0000 7f8f21638640 1 --2- 172.21.15.157:0/3980965869 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f1c156ee0 0x7f8f1c1faea0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8f10031000 tx=0x7f8f100046c0 comp rx=0 tx=0).stop 2024-04-24T23:35:21.370 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.369+0000 7f8f21638640 1 -- 172.21.15.157:0/3980965869 shutdown_connections 2024-04-24T23:35:21.370 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.369+0000 7f8f21638640 1 --2- 172.21.15.157:0/3980965869 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f8f0806bf30 0x7f8f0806e3f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:21.371 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.369+0000 7f8f21638640 1 --2- 172.21.15.157:0/3980965869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f1c15d190 0x7f8f1c1fb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:21.371 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.369+0000 7f8f21638640 1 --2- 172.21.15.157:0/3980965869 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f1c156ee0 0x7f8f1c1faea0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:21.371 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.369+0000 7f8f21638640 1 --2- 172.21.15.157:0/3980965869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f1c066ea0 0x7f8f1c06d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:21.371 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.369+0000 7f8f21638640 1 -- 172.21.15.157:0/3980965869 >> 172.21.15.157:0/3980965869 conn(0x7f8f1c0726c0 msgr2=0x7f8f1c154ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:21.371 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.369+0000 7f8f21638640 1 -- 172.21.15.157:0/3980965869 shutdown_connections 2024-04-24T23:35:21.371 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.369+0000 7f8f21638640 1 -- 172.21.15.157:0/3980965869 wait complete. 2024-04-24T23:35:21.371 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.369+0000 7f92ed7fa640 1 -- 172.21.15.157:0/2539671737 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f92d805c790 con 0x7f92f006b3d0 2024-04-24T23:35:21.371 INFO:teuthology.orchestra.run.smithi157.stdout:60129542152 2024-04-24T23:35:21.373 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542152 for osd.5 2024-04-24T23:35:21.373 DEBUG:teuthology.parallel:result is None 2024-04-24T23:35:21.374 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.373+0000 7f92f6e88640 1 -- 172.21.15.157:0/2539671737 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f92d006bf30 msgr2=0x7f92d006e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:21.374 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.373+0000 7f92f6e88640 1 --2- 172.21.15.157:0/2539671737 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f92d006bf30 0x7f92d006e3f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f92e0034000 tx=0x7f92e002e000 comp rx=0 tx=0).stop 2024-04-24T23:35:21.374 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.373+0000 7f92f6e88640 1 -- 172.21.15.157:0/2539671737 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92f006b3d0 msgr2=0x7f92f0151ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:21.374 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.373+0000 7f92f6e88640 1 --2- 172.21.15.157:0/2539671737 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92f006b3d0 0x7f92f0151ac0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f92d800c400 tx=0x7f92d800c8c0 comp rx=0 tx=0).stop 2024-04-24T23:35:21.374 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.373+0000 7f92f6e88640 1 -- 172.21.15.157:0/2539671737 shutdown_connections 2024-04-24T23:35:21.374 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.373+0000 7f92f6e88640 1 --2- 172.21.15.157:0/2539671737 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f92d006bf30 0x7f92d006e3f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:21.374 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.373+0000 7f92f6e88640 1 --2- 172.21.15.157:0/2539671737 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f92f00c6080 0x7f92f01ee700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:21.374 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.373+0000 7f92f6e88640 1 --2- 172.21.15.157:0/2539671737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92f006f630 0x7f92f014a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:21.375 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.373+0000 7f92f6e88640 1 --2- 172.21.15.157:0/2539671737 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92f006b3d0 0x7f92f0151ac0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:21.375 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.373+0000 7f92f6e88640 1 -- 172.21.15.157:0/2539671737 >> 172.21.15.157:0/2539671737 conn(0x7f92f006c270 msgr2=0x7f92f006c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:21.375 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.373+0000 7f92f6e88640 1 -- 172.21.15.157:0/2539671737 shutdown_connections 2024-04-24T23:35:21.375 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.373+0000 7f92f6e88640 1 -- 172.21.15.157:0/2539671737 wait complete. 2024-04-24T23:35:21.378 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.377+0000 7f06e759e640 1 -- 172.21.15.157:0/2014074655 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f06e0005390 con 0x7f06e00dbce0 2024-04-24T23:35:21.378 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.378+0000 7f06d6ffd640 1 -- 172.21.15.157:0/2014074655 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f06d80791b0 con 0x7f06e00dbce0 2024-04-24T23:35:21.379 INFO:teuthology.orchestra.run.smithi157.stdout:60129542152 2024-04-24T23:35:21.383 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542153 for osd.6 2024-04-24T23:35:21.383 DEBUG:teuthology.parallel:result is None 2024-04-24T23:35:21.383 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.382+0000 7f06e759e640 1 -- 172.21.15.157:0/2014074655 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f06c806be80 msgr2=0x7f06c806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:21.384 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.382+0000 7f06e759e640 1 --2- 172.21.15.157:0/2014074655 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f06c806be80 0x7f06c806e340 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f06dc006fd0 tx=0x7f06dc008040 comp rx=0 tx=0).stop 2024-04-24T23:35:21.384 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.382+0000 7f06e759e640 1 -- 172.21.15.157:0/2014074655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06e00dbce0 msgr2=0x7f06e016a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:21.384 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.382+0000 7f06e759e640 1 --2- 172.21.15.157:0/2014074655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06e00dbce0 0x7f06e016a4a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f06d8009930 tx=0x7f06d80046c0 comp rx=0 tx=0).stop 2024-04-24T23:35:21.384 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.382+0000 7f06e759e640 1 -- 172.21.15.157:0/2014074655 shutdown_connections 2024-04-24T23:35:21.384 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.382+0000 7f06e759e640 1 --2- 172.21.15.157:0/2014074655 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f06c806be80 0x7f06c806e340 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:21.384 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.382+0000 7f06e759e640 1 --2- 172.21.15.157:0/2014074655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06e00df180 0x7f06e016a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:21.384 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.382+0000 7f06e759e640 1 --2- 172.21.15.157:0/2014074655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06e00dbce0 0x7f06e016a4a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:21.384 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.382+0000 7f06e759e640 1 --2- 172.21.15.157:0/2014074655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06e00073e0 0x7f06e00dd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:21.384 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.382+0000 7f06e759e640 1 -- 172.21.15.157:0/2014074655 >> 172.21.15.157:0/2014074655 conn(0x7f06e0048250 msgr2=0x7f06e0048630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:21.384 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.382+0000 7f06e759e640 1 -- 172.21.15.157:0/2014074655 shutdown_connections 2024-04-24T23:35:21.384 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.382+0000 7f06e759e640 1 -- 172.21.15.157:0/2014074655 wait complete. 2024-04-24T23:35:21.389 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542152 for osd.2 2024-04-24T23:35:21.389 DEBUG:teuthology.parallel:result is None 2024-04-24T23:35:21.394 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542152 for osd.1 2024-04-24T23:35:21.394 DEBUG:teuthology.parallel:result is None 2024-04-24T23:35:21.395 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-24T23:35:21.395 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:35:21.532 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.531+0000 7f4e11812640 1 Processor -- start 2024-04-24T23:35:21.532 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.531+0000 7f4e11812640 1 -- start start 2024-04-24T23:35:21.534 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.532+0000 7f4e11812640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4e0c06b3d0 0x7f4e0c06b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:21.534 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.532+0000 7f4e11812640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e0c155d60 con 0x7f4e0c0c2110 2024-04-24T23:35:21.534 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.532+0000 7f4e11812640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e0c156040 con 0x7f4e0c0c3820 2024-04-24T23:35:21.534 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.532+0000 7f4e11812640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f4e0c06e030 con 0x7f4e0c06b3d0 2024-04-24T23:35:21.534 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.532+0000 7f4e03fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f4e0c0c2110 0x7f4e0c0ab5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:47736/0 (socket says 172.21.15.157:47736) 2024-04-24T23:35:21.534 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.532+0000 7f4e03fff640 1 -- 172.21.15.157:0/2914026029 learned_addr learned my addr 172.21.15.157:0/2914026029 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-24T23:35:21.535 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.532+0000 7f4e0affd640 1 --2- 172.21.15.157:0/2914026029 >> v2:172.21.15.169:3300/0 conn(0x7f4e0c06b3d0 0x7f4e0c06b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:21.535 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.532+0000 7f4e0a7fc640 1 -- 172.21.15.157:0/2914026029 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2836640548 0 0) 0x7f4e0c155d60 con 0x7f4e0c0c2110 2024-04-24T23:35:21.535 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.532+0000 7f4e0a7fc640 1 -- 172.21.15.157:0/2914026029 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4df8003520 con 0x7f4e0c0c2110 2024-04-24T23:35:21.535 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.533+0000 7f4e0affd640 1 -- 172.21.15.157:0/2914026029 >> v1:172.21.15.169:6789/0 conn(0x7f4e0c0c3820 legacy=0x7f4e0c0c0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:21.535 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.533+0000 7f4e0a7fc640 1 -- 172.21.15.157:0/2914026029 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3574281403 0 0) 0x7f4df8003520 con 0x7f4e0c0c2110 2024-04-24T23:35:21.535 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.533+0000 7f4e0affd640 1 -- 172.21.15.157:0/2914026029 >> v1:172.21.15.157:6789/0 conn(0x7f4e0c0c2110 legacy=0x7f4e0c0ab5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:21.535 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.533+0000 7f4e0affd640 1 -- 172.21.15.157:0/2914026029 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e0c0ac3d0 con 0x7f4e0c06b3d0 2024-04-24T23:35:21.535 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.533+0000 7f4e0affd640 1 --2- 172.21.15.157:0/2914026029 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e0c06b3d0 0x7f4e0c06b7b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc0097a0 tx=0x7f4dfc02a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e905413e8beb98b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:21.535 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.533+0000 7f4e0a7fc640 1 -- 172.21.15.157:0/2914026029 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc037070 con 0x7f4e0c06b3d0 2024-04-24T23:35:21.535 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.533+0000 7f4e0a7fc640 1 -- 172.21.15.157:0/2914026029 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dfc02ac50 con 0x7f4e0c06b3d0 2024-04-24T23:35:21.535 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.534+0000 7f4e0a7fc640 1 -- 172.21.15.157:0/2914026029 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df4001670 con 0x7f4e0c06b3d0 2024-04-24T23:35:21.538 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.534+0000 7f4e11812640 1 -- 172.21.15.157:0/2914026029 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e0c06b3d0 msgr2=0x7f4e0c06b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:21.538 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.534+0000 7f4e11812640 1 --2- 172.21.15.157:0/2914026029 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e0c06b3d0 0x7f4e0c06b7b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc0097a0 tx=0x7f4dfc02a1c0 comp rx=0 tx=0).stop 2024-04-24T23:35:21.538 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.534+0000 7f4e11812640 1 -- 172.21.15.157:0/2914026029 shutdown_connections 2024-04-24T23:35:21.538 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.534+0000 7f4e11812640 1 --2- 172.21.15.157:0/2914026029 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e0c06b3d0 0x7f4e0c06b7b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:21.538 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.534+0000 7f4e11812640 1 -- 172.21.15.157:0/2914026029 >> 172.21.15.157:0/2914026029 conn(0x7f4e0c0c71f0 msgr2=0x7f4e0c0c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:21.538 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.534+0000 7f4e11812640 1 -- 172.21.15.157:0/2914026029 shutdown_connections 2024-04-24T23:35:21.538 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.534+0000 7f4e11812640 1 -- 172.21.15.157:0/2914026029 wait complete. 2024-04-24T23:35:21.538 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.535+0000 7f4e11812640 1 Processor -- start 2024-04-24T23:35:21.539 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.535+0000 7f4e11812640 1 -- start start 2024-04-24T23:35:21.539 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.536+0000 7f4e11812640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e0c06b3d0 0x7f4e0c1529e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:21.539 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.536+0000 7f4e11812640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e0c0c2110 0x7f4e0c1e7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:21.539 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.536+0000 7f4e11812640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4e0c0c3820 0x7f4e0c0aa310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:21.539 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.536+0000 7f4e11812640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4e0c0c1bf0 con 0x7f4e0c06b3d0 2024-04-24T23:35:21.539 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.536+0000 7f4e11812640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4e0c14f7f0 con 0x7f4e0c0c2110 2024-04-24T23:35:21.539 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.536+0000 7f4e11812640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4e0c0abcf0 con 0x7f4e0c0c3820 2024-04-24T23:35:21.539 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.536+0000 7f4e0affd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e0c06b3d0 0x7f4e0c1529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:21.539 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.536+0000 7f4e0affd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e0c06b3d0 0x7f4e0c1529e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43010/0 (socket says 172.21.15.157:43010) 2024-04-24T23:35:21.539 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.536+0000 7f4e0affd640 1 -- 172.21.15.157:0/1629386869 learned_addr learned my addr 172.21.15.157:0/1629386869 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:21.539 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.536+0000 7f4e03fff640 1 --2- 172.21.15.157:0/1629386869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e0c0c2110 0x7f4e0c1e7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:21.539 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.536+0000 7f4e0b7fe640 1 --2- 172.21.15.157:0/1629386869 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4e0c0c3820 0x7f4e0c0aa310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:21.539 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.536+0000 7f4e0affd640 1 -- 172.21.15.157:0/1629386869 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4e0c0c3820 msgr2=0x7f4e0c0aa310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:21.540 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.536+0000 7f4e0affd640 1 --2- 172.21.15.157:0/1629386869 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4e0c0c3820 0x7f4e0c0aa310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:21.540 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.536+0000 7f4e0affd640 1 -- 172.21.15.157:0/1629386869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e0c0c2110 msgr2=0x7f4e0c1e7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:21.540 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.536+0000 7f4e0affd640 1 --2- 172.21.15.157:0/1629386869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e0c0c2110 0x7f4e0c1e7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:21.540 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.536+0000 7f4e0affd640 1 -- 172.21.15.157:0/1629386869 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e0c13fb90 con 0x7f4e0c06b3d0 2024-04-24T23:35:21.540 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.536+0000 7f4e0affd640 1 --2- 172.21.15.157:0/1629386869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e0c06b3d0 0x7f4e0c1529e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc035040 tx=0x7f4dfc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:21.540 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.537+0000 7f4e08ff9640 1 -- 172.21.15.157:0/1629386869 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc042020 con 0x7f4e0c06b3d0 2024-04-24T23:35:21.540 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.537+0000 7f4e08ff9640 1 -- 172.21.15.157:0/1629386869 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dfc03d470 con 0x7f4e0c06b3d0 2024-04-24T23:35:21.540 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.537+0000 7f4e11812640 1 -- 172.21.15.157:0/1629386869 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e0c1ea9c0 con 0x7f4e0c06b3d0 2024-04-24T23:35:21.540 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.537+0000 7f4e08ff9640 1 -- 172.21.15.157:0/1629386869 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc037040 con 0x7f4e0c06b3d0 2024-04-24T23:35:21.540 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.537+0000 7f4e11812640 1 -- 172.21.15.157:0/1629386869 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e0c1eaf80 con 0x7f4e0c06b3d0 2024-04-24T23:35:21.541 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.538+0000 7f4e11812640 1 -- 172.21.15.157:0/1629386869 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e0c0681c0 con 0x7f4e0c06b3d0 2024-04-24T23:35:21.541 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.538+0000 7f4e08ff9640 1 -- 172.21.15.157:0/1629386869 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4dfc045020 con 0x7f4e0c06b3d0 2024-04-24T23:35:21.541 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.538+0000 7f4e08ff9640 1 --2- 172.21.15.157:0/1629386869 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f4dec06be30 0x7f4dec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:21.541 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.538+0000 7f4e08ff9640 1 -- 172.21.15.157:0/1629386869 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f4dfc0ab4a0 con 0x7f4e0c06b3d0 2024-04-24T23:35:21.541 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.539+0000 7f4e03fff640 1 --2- 172.21.15.157:0/1629386869 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f4dec06be30 0x7f4dec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:21.542 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.539+0000 7f4e03fff640 1 --2- 172.21.15.157:0/1629386869 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f4dec06be30 0x7f4dec06e2f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c1596a0 tx=0x7f4df4000bc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:21.543 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.543+0000 7f4e08ff9640 1 -- 172.21.15.157:0/1629386869 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4dfc04c070 con 0x7f4e0c06b3d0 2024-04-24T23:35:21.712 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.712+0000 7f4e11812640 1 -- 172.21.15.157:0/1629386869 --> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4e0c06c1b0 con 0x7f4dec06be30 2024-04-24T23:35:21.720 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.720+0000 7f4e08ff9640 1 -- 172.21.15.157:0/1629386869 <== mgr.4105 v2:172.21.15.169:6832/662378289 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332611 (secure 0 0 0) 0x7f4e0c06c1b0 con 0x7f4dec06be30 2024-04-24T23:35:21.724 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:35:21.726 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2024-04-24T23:35:21.728 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.723+0000 7f4e11812640 1 -- 172.21.15.157:0/1629386869 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f4dec06be30 msgr2=0x7f4dec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:21.728 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.723+0000 7f4e11812640 1 --2- 172.21.15.157:0/1629386869 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f4dec06be30 0x7f4dec06e2f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c1596a0 tx=0x7f4df4000bc0 comp rx=0 tx=0).stop 2024-04-24T23:35:21.728 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.723+0000 7f4e11812640 1 -- 172.21.15.157:0/1629386869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e0c06b3d0 msgr2=0x7f4e0c1529e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:21.728 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.723+0000 7f4e11812640 1 --2- 172.21.15.157:0/1629386869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e0c06b3d0 0x7f4e0c1529e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc035040 tx=0x7f4dfc0046c0 comp rx=0 tx=0).stop 2024-04-24T23:35:21.728 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.723+0000 7f4e11812640 1 -- 172.21.15.157:0/1629386869 shutdown_connections 2024-04-24T23:35:21.728 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.723+0000 7f4e11812640 1 --2- 172.21.15.157:0/1629386869 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f4dec06be30 0x7f4dec06e2f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:21.728 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.723+0000 7f4e11812640 1 --2- 172.21.15.157:0/1629386869 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4e0c0c3820 0x7f4e0c0aa310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:21.728 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.723+0000 7f4e11812640 1 --2- 172.21.15.157:0/1629386869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e0c0c2110 0x7f4e0c1e7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:21.729 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.723+0000 7f4e11812640 1 --2- 172.21.15.157:0/1629386869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e0c06b3d0 0x7f4e0c1529e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:21.729 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.723+0000 7f4e11812640 1 -- 172.21.15.157:0/1629386869 >> 172.21.15.157:0/1629386869 conn(0x7f4e0c0c71f0 msgr2=0x7f4e0c1547b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:21.729 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.723+0000 7f4e11812640 1 -- 172.21.15.157:0/1629386869 shutdown_connections 2024-04-24T23:35:21.729 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.723+0000 7f4e11812640 1 -- 172.21.15.157:0/1629386869 wait complete. 2024-04-24T23:35:21.735 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2024-04-24T23:35:19.772624+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235052,"kb_used_data":2240,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739668,"statfs":{"total":773094113280,"available":772853420032,"internally_reserved":0,"allocated":2293760,"data_stored":2015416,"data_compressed":65248,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003082"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.428046+0000","last_change":"2024-04-24T23:34:53.362409+0000","last_active":"2024-04-24T23:34:59.428046+0000","last_peered":"2024-04-24T23:34:59.428046+0000","last_clean":"2024-04-24T23:34:59.428046+0000","last_became_active":"2024-04-24T23:34:53.362104+0000","last_became_peered":"2024-04-24T23:34:53.362104+0000","last_unstale":"2024-04-24T23:34:59.428046+0000","last_undegraded":"2024-04-24T23:34:59.428046+0000","last_fullsized":"2024-04-24T23:34:59.428046+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:53.159219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396596+0000","last_change":"2024-04-24T23:34:54.373560+0000","last_active":"2024-04-24T23:34:59.396596+0000","last_peered":"2024-04-24T23:34:59.396596+0000","last_clean":"2024-04-24T23:34:59.396596+0000","last_became_active":"2024-04-24T23:34:54.373289+0000","last_became_peered":"2024-04-24T23:34:54.373289+0000","last_unstale":"2024-04-24T23:34:59.396596+0000","last_undegraded":"2024-04-24T23:34:59.396596+0000","last_fullsized":"2024-04-24T23:34:59.396596+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:31.447899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.395661+0000","last_change":"2024-04-24T23:34:58.730165+0000","last_active":"2024-04-24T23:34:59.395661+0000","last_peered":"2024-04-24T23:34:59.395661+0000","last_clean":"2024-04-24T23:34:59.395661+0000","last_became_active":"2024-04-24T23:34:58.729816+0000","last_became_peered":"2024-04-24T23:34:58.729816+0000","last_unstale":"2024-04-24T23:34:59.395661+0000","last_undegraded":"2024-04-24T23:34:59.395661+0000","last_fullsized":"2024-04-24T23:34:59.395661+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:02:02.335135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396776+0000","last_change":"2024-04-24T23:34:53.361650+0000","last_active":"2024-04-24T23:34:59.396776+0000","last_peered":"2024-04-24T23:34:59.396776+0000","last_clean":"2024-04-24T23:34:59.396776+0000","last_became_active":"2024-04-24T23:34:53.361366+0000","last_became_peered":"2024-04-24T23:34:53.361366+0000","last_unstale":"2024-04-24T23:34:59.396776+0000","last_undegraded":"2024-04-24T23:34:59.396776+0000","last_fullsized":"2024-04-24T23:34:59.396776+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:56:15.784223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.427859+0000","last_change":"2024-04-24T23:34:54.383602+0000","last_active":"2024-04-24T23:34:59.427859+0000","last_peered":"2024-04-24T23:34:59.427859+0000","last_clean":"2024-04-24T23:34:59.427859+0000","last_became_active":"2024-04-24T23:34:54.383160+0000","last_became_peered":"2024-04-24T23:34:54.383160+0000","last_unstale":"2024-04-24T23:34:59.427859+0000","last_undegraded":"2024-04-24T23:34:59.427859+0000","last_fullsized":"2024-04-24T23:34:59.427859+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:15.477061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396642+0000","last_change":"2024-04-24T23:34:58.404416+0000","last_active":"2024-04-24T23:34:59.396642+0000","last_peered":"2024-04-24T23:34:59.396642+0000","last_clean":"2024-04-24T23:34:59.396642+0000","last_became_active":"2024-04-24T23:34:58.403984+0000","last_became_peered":"2024-04-24T23:34:58.403984+0000","last_unstale":"2024-04-24T23:34:59.396642+0000","last_undegraded":"2024-04-24T23:34:59.396642+0000","last_fullsized":"2024-04-24T23:34:59.396642+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:46.237082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501886+0000","last_change":"2024-04-24T23:34:53.366730+0000","last_active":"2024-04-24T23:34:59.501886+0000","last_peered":"2024-04-24T23:34:59.501886+0000","last_clean":"2024-04-24T23:34:59.501886+0000","last_became_active":"2024-04-24T23:34:53.366430+0000","last_became_peered":"2024-04-24T23:34:53.366430+0000","last_unstale":"2024-04-24T23:34:59.501886+0000","last_undegraded":"2024-04-24T23:34:59.501886+0000","last_fullsized":"2024-04-24T23:34:59.501886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:41.422591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540654+0000","last_change":"2024-04-24T23:34:54.373592+0000","last_active":"2024-04-24T23:35:00.540654+0000","last_peered":"2024-04-24T23:35:00.540654+0000","last_clean":"2024-04-24T23:35:00.540654+0000","last_became_active":"2024-04-24T23:34:54.372844+0000","last_became_peered":"2024-04-24T23:34:54.372844+0000","last_unstale":"2024-04-24T23:35:00.540654+0000","last_undegraded":"2024-04-24T23:35:00.540654+0000","last_fullsized":"2024-04-24T23:35:00.540654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:49.372040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396779+0000","last_change":"2024-04-24T23:34:58.399224+0000","last_active":"2024-04-24T23:34:59.396779+0000","last_peered":"2024-04-24T23:34:59.396779+0000","last_clean":"2024-04-24T23:34:59.396779+0000","last_became_active":"2024-04-24T23:34:58.397817+0000","last_became_peered":"2024-04-24T23:34:58.397817+0000","last_unstale":"2024-04-24T23:34:59.396779+0000","last_undegraded":"2024-04-24T23:34:59.396779+0000","last_fullsized":"2024-04-24T23:34:59.396779+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:53.085366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501568+0000","last_change":"2024-04-24T23:34:53.362994+0000","last_active":"2024-04-24T23:34:59.501568+0000","last_peered":"2024-04-24T23:34:59.501568+0000","last_clean":"2024-04-24T23:34:59.501568+0000","last_became_active":"2024-04-24T23:34:53.362337+0000","last_became_peered":"2024-04-24T23:34:53.362337+0000","last_unstale":"2024-04-24T23:34:59.501568+0000","last_undegraded":"2024-04-24T23:34:59.501568+0000","last_fullsized":"2024-04-24T23:34:59.501568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:12.258211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.541217+0000","last_change":"2024-04-24T23:34:54.379218+0000","last_active":"2024-04-24T23:35:00.541217+0000","last_peered":"2024-04-24T23:35:00.541217+0000","last_clean":"2024-04-24T23:35:00.541217+0000","last_became_active":"2024-04-24T23:34:54.376019+0000","last_became_peered":"2024-04-24T23:34:54.376019+0000","last_unstale":"2024-04-24T23:35:00.541217+0000","last_undegraded":"2024-04-24T23:35:00.541217+0000","last_fullsized":"2024-04-24T23:35:00.541217+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:41:48.207542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396752+0000","last_change":"2024-04-24T23:34:58.397152+0000","last_active":"2024-04-24T23:34:59.396752+0000","last_peered":"2024-04-24T23:34:59.396752+0000","last_clean":"2024-04-24T23:34:59.396752+0000","last_became_active":"2024-04-24T23:34:58.396658+0000","last_became_peered":"2024-04-24T23:34:58.396658+0000","last_unstale":"2024-04-24T23:34:59.396752+0000","last_undegraded":"2024-04-24T23:34:59.396752+0000","last_fullsized":"2024-04-24T23:34:59.396752+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:41.360776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540212+0000","last_change":"2024-04-24T23:34:54.377859+0000","last_active":"2024-04-24T23:35:00.540212+0000","last_peered":"2024-04-24T23:35:00.540212+0000","last_clean":"2024-04-24T23:35:00.540212+0000","last_became_active":"2024-04-24T23:34:54.377576+0000","last_became_peered":"2024-04-24T23:34:54.377576+0000","last_unstale":"2024-04-24T23:35:00.540212+0000","last_undegraded":"2024-04-24T23:35:00.540212+0000","last_fullsized":"2024-04-24T23:35:00.540212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:47.446459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501625+0000","last_change":"2024-04-24T23:34:53.360903+0000","last_active":"2024-04-24T23:34:59.501625+0000","last_peered":"2024-04-24T23:34:59.501625+0000","last_clean":"2024-04-24T23:34:59.501625+0000","last_became_active":"2024-04-24T23:34:53.360635+0000","last_became_peered":"2024-04-24T23:34:53.360635+0000","last_unstale":"2024-04-24T23:34:59.501625+0000","last_undegraded":"2024-04-24T23:34:59.501625+0000","last_fullsized":"2024-04-24T23:34:59.501625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:42.205997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397132+0000","last_change":"2024-04-24T23:34:58.407146+0000","last_active":"2024-04-24T23:34:59.397132+0000","last_peered":"2024-04-24T23:34:59.397132+0000","last_clean":"2024-04-24T23:34:59.397132+0000","last_became_active":"2024-04-24T23:34:58.406844+0000","last_became_peered":"2024-04-24T23:34:58.406844+0000","last_unstale":"2024-04-24T23:34:59.397132+0000","last_undegraded":"2024-04-24T23:34:59.397132+0000","last_fullsized":"2024-04-24T23:34:59.397132+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:41:52.244767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396642+0000","last_change":"2024-04-24T23:34:54.366724+0000","last_active":"2024-04-24T23:34:59.396642+0000","last_peered":"2024-04-24T23:34:59.396642+0000","last_clean":"2024-04-24T23:34:59.396642+0000","last_became_active":"2024-04-24T23:34:54.366426+0000","last_became_peered":"2024-04-24T23:34:54.366426+0000","last_unstale":"2024-04-24T23:34:59.396642+0000","last_undegraded":"2024-04-24T23:34:59.396642+0000","last_fullsized":"2024-04-24T23:34:59.396642+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:47.035356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.881182+0000","last_change":"2024-04-24T23:34:53.359140+0000","last_active":"2024-04-24T23:34:59.881182+0000","last_peered":"2024-04-24T23:34:59.881182+0000","last_clean":"2024-04-24T23:34:59.881182+0000","last_became_active":"2024-04-24T23:34:53.358841+0000","last_became_peered":"2024-04-24T23:34:53.358841+0000","last_unstale":"2024-04-24T23:34:59.881182+0000","last_undegraded":"2024-04-24T23:34:59.881182+0000","last_fullsized":"2024-04-24T23:34:59.881182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:41.422591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397033+0000","last_change":"2024-04-24T23:34:58.400128+0000","last_active":"2024-04-24T23:34:59.397033+0000","last_peered":"2024-04-24T23:34:59.397033+0000","last_clean":"2024-04-24T23:34:59.397033+0000","last_became_active":"2024-04-24T23:34:58.399375+0000","last_became_peered":"2024-04-24T23:34:58.399375+0000","last_unstale":"2024-04-24T23:34:59.397033+0000","last_undegraded":"2024-04-24T23:34:59.397033+0000","last_fullsized":"2024-04-24T23:34:59.397033+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:53.409265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.395824+0000","last_change":"2024-04-24T23:34:54.365012+0000","last_active":"2024-04-24T23:34:59.395824+0000","last_peered":"2024-04-24T23:34:59.395824+0000","last_clean":"2024-04-24T23:34:59.395824+0000","last_became_active":"2024-04-24T23:34:54.364744+0000","last_became_peered":"2024-04-24T23:34:54.364744+0000","last_unstale":"2024-04-24T23:34:59.395824+0000","last_undegraded":"2024-04-24T23:34:59.395824+0000","last_fullsized":"2024-04-24T23:34:59.395824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:35.774122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.880884+0000","last_change":"2024-04-24T23:34:53.360843+0000","last_active":"2024-04-24T23:34:59.880884+0000","last_peered":"2024-04-24T23:34:59.880884+0000","last_clean":"2024-04-24T23:34:59.880884+0000","last_became_active":"2024-04-24T23:34:53.360570+0000","last_became_peered":"2024-04-24T23:34:53.360570+0000","last_unstale":"2024-04-24T23:34:59.880884+0000","last_undegraded":"2024-04-24T23:34:59.880884+0000","last_fullsized":"2024-04-24T23:34:59.880884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:21.089467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.881510+0000","last_change":"2024-04-24T23:34:58.731835+0000","last_active":"2024-04-24T23:34:59.881510+0000","last_peered":"2024-04-24T23:34:59.881510+0000","last_clean":"2024-04-24T23:34:59.881510+0000","last_became_active":"2024-04-24T23:34:58.731485+0000","last_became_peered":"2024-04-24T23:34:58.731485+0000","last_unstale":"2024-04-24T23:34:59.881510+0000","last_undegraded":"2024-04-24T23:34:59.881510+0000","last_fullsized":"2024-04-24T23:34:59.881510+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:56.609480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397598+0000","last_change":"2024-04-24T23:34:54.362447+0000","last_active":"2024-04-24T23:34:59.397598+0000","last_peered":"2024-04-24T23:34:59.397598+0000","last_clean":"2024-04-24T23:34:59.397598+0000","last_became_active":"2024-04-24T23:34:54.362141+0000","last_became_peered":"2024-04-24T23:34:54.362141+0000","last_unstale":"2024-04-24T23:34:59.397598+0000","last_undegraded":"2024-04-24T23:34:59.397598+0000","last_fullsized":"2024-04-24T23:34:59.397598+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:51.227333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501234+0000","last_change":"2024-04-24T23:34:53.366452+0000","last_active":"2024-04-24T23:34:59.501234+0000","last_peered":"2024-04-24T23:34:59.501234+0000","last_clean":"2024-04-24T23:34:59.501234+0000","last_became_active":"2024-04-24T23:34:53.366110+0000","last_became_peered":"2024-04-24T23:34:53.366110+0000","last_unstale":"2024-04-24T23:34:59.501234+0000","last_undegraded":"2024-04-24T23:34:59.501234+0000","last_fullsized":"2024-04-24T23:34:59.501234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:14.716741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396886+0000","last_change":"2024-04-24T23:34:58.402849+0000","last_active":"2024-04-24T23:34:59.396886+0000","last_peered":"2024-04-24T23:34:59.396886+0000","last_clean":"2024-04-24T23:34:59.396886+0000","last_became_active":"2024-04-24T23:34:58.402365+0000","last_became_peered":"2024-04-24T23:34:58.402365+0000","last_unstale":"2024-04-24T23:34:59.396886+0000","last_undegraded":"2024-04-24T23:34:59.396886+0000","last_fullsized":"2024-04-24T23:34:59.396886+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:07:32.822427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.428004+0000","last_change":"2024-04-24T23:34:54.370300+0000","last_active":"2024-04-24T23:34:59.428004+0000","last_peered":"2024-04-24T23:34:59.428004+0000","last_clean":"2024-04-24T23:34:59.428004+0000","last_became_active":"2024-04-24T23:34:54.369835+0000","last_became_peered":"2024-04-24T23:34:54.369835+0000","last_unstale":"2024-04-24T23:34:59.428004+0000","last_undegraded":"2024-04-24T23:34:59.428004+0000","last_fullsized":"2024-04-24T23:34:59.428004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:42.199856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.541551+0000","last_change":"2024-04-24T23:34:53.366110+0000","last_active":"2024-04-24T23:35:00.541551+0000","last_peered":"2024-04-24T23:35:00.541551+0000","last_clean":"2024-04-24T23:35:00.541551+0000","last_became_active":"2024-04-24T23:34:53.365783+0000","last_became_peered":"2024-04-24T23:34:53.365783+0000","last_unstale":"2024-04-24T23:35:00.541551+0000","last_undegraded":"2024-04-24T23:35:00.541551+0000","last_fullsized":"2024-04-24T23:35:00.541551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:41.422591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397333+0000","last_change":"2024-04-24T23:34:58.728191+0000","last_active":"2024-04-24T23:34:59.397333+0000","last_peered":"2024-04-24T23:34:59.397333+0000","last_clean":"2024-04-24T23:34:59.397333+0000","last_became_active":"2024-04-24T23:34:58.727653+0000","last_became_peered":"2024-04-24T23:34:58.727653+0000","last_unstale":"2024-04-24T23:34:59.397333+0000","last_undegraded":"2024-04-24T23:34:59.397333+0000","last_fullsized":"2024-04-24T23:34:59.397333+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:30.299011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397474+0000","last_change":"2024-04-24T23:34:54.368278+0000","last_active":"2024-04-24T23:34:59.397474+0000","last_peered":"2024-04-24T23:34:59.397474+0000","last_clean":"2024-04-24T23:34:59.397474+0000","last_became_active":"2024-04-24T23:34:54.367991+0000","last_became_peered":"2024-04-24T23:34:54.367991+0000","last_unstale":"2024-04-24T23:34:59.397474+0000","last_undegraded":"2024-04-24T23:34:59.397474+0000","last_fullsized":"2024-04-24T23:34:59.397474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:49:31.018719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.881400+0000","last_change":"2024-04-24T23:34:53.364189+0000","last_active":"2024-04-24T23:34:59.881400+0000","last_peered":"2024-04-24T23:34:59.881400+0000","last_clean":"2024-04-24T23:34:59.881400+0000","last_became_active":"2024-04-24T23:34:53.363842+0000","last_became_peered":"2024-04-24T23:34:53.363842+0000","last_unstale":"2024-04-24T23:34:59.881400+0000","last_undegraded":"2024-04-24T23:34:59.881400+0000","last_fullsized":"2024-04-24T23:34:59.881400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:12.258211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.395610+0000","last_change":"2024-04-24T23:34:58.729450+0000","last_active":"2024-04-24T23:34:59.395610+0000","last_peered":"2024-04-24T23:34:59.395610+0000","last_clean":"2024-04-24T23:34:59.395610+0000","last_became_active":"2024-04-24T23:34:58.729082+0000","last_became_peered":"2024-04-24T23:34:58.729082+0000","last_unstale":"2024-04-24T23:34:59.395610+0000","last_undegraded":"2024-04-24T23:34:59.395610+0000","last_fullsized":"2024-04-24T23:34:59.395610+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:45.372053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.881249+0000","last_change":"2024-04-24T23:34:54.357246+0000","last_active":"2024-04-24T23:34:59.881249+0000","last_peered":"2024-04-24T23:34:59.881249+0000","last_clean":"2024-04-24T23:34:59.881249+0000","last_became_active":"2024-04-24T23:34:54.356560+0000","last_became_peered":"2024-04-24T23:34:54.356560+0000","last_unstale":"2024-04-24T23:34:59.881249+0000","last_undegraded":"2024-04-24T23:34:59.881249+0000","last_fullsized":"2024-04-24T23:34:59.881249+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:42.199856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.502011+0000","last_change":"2024-04-24T23:34:53.366296+0000","last_active":"2024-04-24T23:34:59.502011+0000","last_peered":"2024-04-24T23:34:59.502011+0000","last_clean":"2024-04-24T23:34:59.502011+0000","last_became_active":"2024-04-24T23:34:53.365636+0000","last_became_peered":"2024-04-24T23:34:53.365636+0000","last_unstale":"2024-04-24T23:34:59.502011+0000","last_undegraded":"2024-04-24T23:34:59.502011+0000","last_fullsized":"2024-04-24T23:34:59.502011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:50.222015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.427745+0000","last_change":"2024-04-24T23:34:58.728271+0000","last_active":"2024-04-24T23:34:59.427745+0000","last_peered":"2024-04-24T23:34:59.427745+0000","last_clean":"2024-04-24T23:34:59.427745+0000","last_became_active":"2024-04-24T23:34:58.727885+0000","last_became_peered":"2024-04-24T23:34:58.727885+0000","last_unstale":"2024-04-24T23:34:59.427745+0000","last_undegraded":"2024-04-24T23:34:59.427745+0000","last_fullsized":"2024-04-24T23:34:59.427745+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:49.461271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.541247+0000","last_change":"2024-04-24T23:34:54.372245+0000","last_active":"2024-04-24T23:35:00.541247+0000","last_peered":"2024-04-24T23:35:00.541247+0000","last_clean":"2024-04-24T23:35:00.541247+0000","last_became_active":"2024-04-24T23:34:54.371947+0000","last_became_peered":"2024-04-24T23:34:54.371947+0000","last_unstale":"2024-04-24T23:35:00.541247+0000","last_undegraded":"2024-04-24T23:35:00.541247+0000","last_fullsized":"2024-04-24T23:35:00.541247+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:47:52.460508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.881345+0000","last_change":"2024-04-24T23:34:53.360352+0000","last_active":"2024-04-24T23:34:59.881345+0000","last_peered":"2024-04-24T23:34:59.881345+0000","last_clean":"2024-04-24T23:34:59.881345+0000","last_became_active":"2024-04-24T23:34:53.360082+0000","last_became_peered":"2024-04-24T23:34:53.360082+0000","last_unstale":"2024-04-24T23:34:59.881345+0000","last_undegraded":"2024-04-24T23:34:59.881345+0000","last_fullsized":"2024-04-24T23:34:59.881345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:50.222015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.881021+0000","last_change":"2024-04-24T23:34:58.404176+0000","last_active":"2024-04-24T23:34:59.881021+0000","last_peered":"2024-04-24T23:34:59.881021+0000","last_clean":"2024-04-24T23:34:59.881021+0000","last_became_active":"2024-04-24T23:34:58.403735+0000","last_became_peered":"2024-04-24T23:34:58.403735+0000","last_unstale":"2024-04-24T23:34:59.881021+0000","last_undegraded":"2024-04-24T23:34:59.881021+0000","last_fullsized":"2024-04-24T23:34:59.881021+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:49.461271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397652+0000","last_change":"2024-04-24T23:34:54.367638+0000","last_active":"2024-04-24T23:34:59.397652+0000","last_peered":"2024-04-24T23:34:59.397652+0000","last_clean":"2024-04-24T23:34:59.397652+0000","last_became_active":"2024-04-24T23:34:54.367355+0000","last_became_peered":"2024-04-24T23:34:54.367355+0000","last_unstale":"2024-04-24T23:34:59.397652+0000","last_undegraded":"2024-04-24T23:34:59.397652+0000","last_fullsized":"2024-04-24T23:34:59.397652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:15.722059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540545+0000","last_change":"2024-04-24T23:34:53.356111+0000","last_active":"2024-04-24T23:35:00.540545+0000","last_peered":"2024-04-24T23:35:00.540545+0000","last_clean":"2024-04-24T23:35:00.540545+0000","last_became_active":"2024-04-24T23:34:53.355710+0000","last_became_peered":"2024-04-24T23:34:53.355710+0000","last_unstale":"2024-04-24T23:35:00.540545+0000","last_undegraded":"2024-04-24T23:35:00.540545+0000","last_fullsized":"2024-04-24T23:35:00.540545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:21.089467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.881283+0000","last_change":"2024-04-24T23:34:58.403016+0000","last_active":"2024-04-24T23:34:59.881283+0000","last_peered":"2024-04-24T23:34:59.881283+0000","last_clean":"2024-04-24T23:34:59.881283+0000","last_became_active":"2024-04-24T23:34:58.402598+0000","last_became_peered":"2024-04-24T23:34:58.402598+0000","last_unstale":"2024-04-24T23:34:59.881283+0000","last_undegraded":"2024-04-24T23:34:59.881283+0000","last_fullsized":"2024-04-24T23:34:59.881283+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:47:56.497734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.880863+0000","last_change":"2024-04-24T23:34:54.356601+0000","last_active":"2024-04-24T23:34:59.880863+0000","last_peered":"2024-04-24T23:34:59.880863+0000","last_clean":"2024-04-24T23:34:59.880863+0000","last_became_active":"2024-04-24T23:34:54.356326+0000","last_became_peered":"2024-04-24T23:34:54.356326+0000","last_unstale":"2024-04-24T23:34:59.880863+0000","last_undegraded":"2024-04-24T23:34:59.880863+0000","last_fullsized":"2024-04-24T23:34:59.880863+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:47.446459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396563+0000","last_change":"2024-04-24T23:34:53.359238+0000","last_active":"2024-04-24T23:34:59.396563+0000","last_peered":"2024-04-24T23:34:59.396563+0000","last_clean":"2024-04-24T23:34:59.396563+0000","last_became_active":"2024-04-24T23:34:53.358937+0000","last_became_peered":"2024-04-24T23:34:53.358937+0000","last_unstale":"2024-04-24T23:34:59.396563+0000","last_undegraded":"2024-04-24T23:34:59.396563+0000","last_fullsized":"2024-04-24T23:34:59.396563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:50.222015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.881551+0000","last_change":"2024-04-24T23:34:58.403258+0000","last_active":"2024-04-24T23:34:59.881551+0000","last_peered":"2024-04-24T23:34:59.881551+0000","last_clean":"2024-04-24T23:34:59.881551+0000","last_became_active":"2024-04-24T23:34:58.402569+0000","last_became_peered":"2024-04-24T23:34:58.402569+0000","last_unstale":"2024-04-24T23:34:59.881551+0000","last_undegraded":"2024-04-24T23:34:59.881551+0000","last_fullsized":"2024-04-24T23:34:59.881551+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:32:08.782070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.427468+0000","last_change":"2024-04-24T23:34:54.377543+0000","last_active":"2024-04-24T23:34:59.427468+0000","last_peered":"2024-04-24T23:34:59.427468+0000","last_clean":"2024-04-24T23:34:59.427468+0000","last_became_active":"2024-04-24T23:34:54.377228+0000","last_became_peered":"2024-04-24T23:34:54.377228+0000","last_unstale":"2024-04-24T23:34:59.427468+0000","last_undegraded":"2024-04-24T23:34:59.427468+0000","last_fullsized":"2024-04-24T23:34:59.427468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:42.255038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501238+0000","last_change":"2024-04-24T23:34:53.362675+0000","last_active":"2024-04-24T23:34:59.501238+0000","last_peered":"2024-04-24T23:34:59.501238+0000","last_clean":"2024-04-24T23:34:59.501238+0000","last_became_active":"2024-04-24T23:34:53.362425+0000","last_became_peered":"2024-04-24T23:34:53.362425+0000","last_unstale":"2024-04-24T23:34:59.501238+0000","last_undegraded":"2024-04-24T23:34:59.501238+0000","last_fullsized":"2024-04-24T23:34:59.501238+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:21.089467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.427884+0000","last_change":"2024-04-24T23:34:58.728573+0000","last_active":"2024-04-24T23:34:59.427884+0000","last_peered":"2024-04-24T23:34:59.427884+0000","last_clean":"2024-04-24T23:34:59.427884+0000","last_became_active":"2024-04-24T23:34:58.727284+0000","last_became_peered":"2024-04-24T23:34:58.727284+0000","last_unstale":"2024-04-24T23:34:59.427884+0000","last_undegraded":"2024-04-24T23:34:59.427884+0000","last_fullsized":"2024-04-24T23:34:59.427884+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:32:08.782070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396198+0000","last_change":"2024-04-24T23:34:54.382820+0000","last_active":"2024-04-24T23:34:59.396198+0000","last_peered":"2024-04-24T23:34:59.396198+0000","last_clean":"2024-04-24T23:34:59.396198+0000","last_became_active":"2024-04-24T23:34:54.382564+0000","last_became_peered":"2024-04-24T23:34:54.382564+0000","last_unstale":"2024-04-24T23:34:59.396198+0000","last_undegraded":"2024-04-24T23:34:59.396198+0000","last_fullsized":"2024-04-24T23:34:59.396198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:45.622611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.541368+0000","last_change":"2024-04-24T23:34:53.357743+0000","last_active":"2024-04-24T23:35:00.541368+0000","last_peered":"2024-04-24T23:35:00.541368+0000","last_clean":"2024-04-24T23:35:00.541368+0000","last_became_active":"2024-04-24T23:34:53.357456+0000","last_became_peered":"2024-04-24T23:34:53.357456+0000","last_unstale":"2024-04-24T23:35:00.541368+0000","last_undegraded":"2024-04-24T23:35:00.541368+0000","last_fullsized":"2024-04-24T23:35:00.541368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:57:14.381644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":177,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501380+0000","last_change":"2024-04-24T23:34:49.778966+0000","last_active":"2024-04-24T23:34:59.501380+0000","last_peered":"2024-04-24T23:34:59.501380+0000","last_clean":"2024-04-24T23:34:59.501380+0000","last_became_active":"2024-04-24T23:34:49.778083+0000","last_became_peered":"2024-04-24T23:34:49.778083+0000","last_unstale":"2024-04-24T23:34:59.501380+0000","last_undegraded":"2024-04-24T23:34:59.501380+0000","last_fullsized":"2024-04-24T23:34:59.501380+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:48.587078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:48.587078+0000","last_clean_scrub_stamp":"2024-04-24T23:34:48.587078+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:46:59.471089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540517+0000","last_change":"2024-04-24T23:34:58.729779+0000","last_active":"2024-04-24T23:35:00.540517+0000","last_peered":"2024-04-24T23:35:00.540517+0000","last_clean":"2024-04-24T23:35:00.540517+0000","last_became_active":"2024-04-24T23:34:58.729370+0000","last_became_peered":"2024-04-24T23:34:58.729370+0000","last_unstale":"2024-04-24T23:35:00.540517+0000","last_undegraded":"2024-04-24T23:35:00.540517+0000","last_fullsized":"2024-04-24T23:35:00.540517+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:30.299011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501756+0000","last_change":"2024-04-24T23:34:54.384106+0000","last_active":"2024-04-24T23:34:59.501756+0000","last_peered":"2024-04-24T23:34:59.501756+0000","last_clean":"2024-04-24T23:34:59.501756+0000","last_became_active":"2024-04-24T23:34:54.383589+0000","last_became_peered":"2024-04-24T23:34:54.383589+0000","last_unstale":"2024-04-24T23:34:59.501756+0000","last_undegraded":"2024-04-24T23:34:59.501756+0000","last_fullsized":"2024-04-24T23:34:59.501756+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:41:48.207542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397450+0000","last_change":"2024-04-24T23:34:53.354077+0000","last_active":"2024-04-24T23:34:59.397450+0000","last_peered":"2024-04-24T23:34:59.397450+0000","last_clean":"2024-04-24T23:34:59.397450+0000","last_became_active":"2024-04-24T23:34:53.353804+0000","last_became_peered":"2024-04-24T23:34:53.353804+0000","last_unstale":"2024-04-24T23:34:59.397450+0000","last_undegraded":"2024-04-24T23:34:59.397450+0000","last_fullsized":"2024-04-24T23:34:59.397450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:41.422591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540103+0000","last_change":"2024-04-24T23:34:58.728722+0000","last_active":"2024-04-24T23:35:00.540103+0000","last_peered":"2024-04-24T23:35:00.540103+0000","last_clean":"2024-04-24T23:35:00.540103+0000","last_became_active":"2024-04-24T23:34:58.727649+0000","last_became_peered":"2024-04-24T23:34:58.727649+0000","last_unstale":"2024-04-24T23:35:00.540103+0000","last_undegraded":"2024-04-24T23:35:00.540103+0000","last_fullsized":"2024-04-24T23:35:00.540103+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:41:52.244767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540678+0000","last_change":"2024-04-24T23:34:54.373371+0000","last_active":"2024-04-24T23:35:00.540678+0000","last_peered":"2024-04-24T23:35:00.540678+0000","last_clean":"2024-04-24T23:35:00.540678+0000","last_became_active":"2024-04-24T23:34:54.372553+0000","last_became_peered":"2024-04-24T23:34:54.372553+0000","last_unstale":"2024-04-24T23:35:00.540678+0000","last_undegraded":"2024-04-24T23:35:00.540678+0000","last_fullsized":"2024-04-24T23:35:00.540678+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:26.261786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.881287+0000","last_change":"2024-04-24T23:34:53.353505+0000","last_active":"2024-04-24T23:34:59.881287+0000","last_peered":"2024-04-24T23:34:59.881287+0000","last_clean":"2024-04-24T23:34:59.881287+0000","last_became_active":"2024-04-24T23:34:53.353186+0000","last_became_peered":"2024-04-24T23:34:53.353186+0000","last_unstale":"2024-04-24T23:34:59.881287+0000","last_undegraded":"2024-04-24T23:34:59.881287+0000","last_fullsized":"2024-04-24T23:34:59.881287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:42.205997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.427805+0000","last_change":"2024-04-24T23:34:58.400988+0000","last_active":"2024-04-24T23:34:59.427805+0000","last_peered":"2024-04-24T23:34:59.427805+0000","last_clean":"2024-04-24T23:34:59.427805+0000","last_became_active":"2024-04-24T23:34:58.400549+0000","last_became_peered":"2024-04-24T23:34:58.400549+0000","last_unstale":"2024-04-24T23:34:59.427805+0000","last_undegraded":"2024-04-24T23:34:59.427805+0000","last_fullsized":"2024-04-24T23:34:59.427805+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:47:56.497734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.502060+0000","last_change":"2024-04-24T23:34:54.375234+0000","last_active":"2024-04-24T23:34:59.502060+0000","last_peered":"2024-04-24T23:34:59.502060+0000","last_clean":"2024-04-24T23:34:59.502060+0000","last_became_active":"2024-04-24T23:34:54.374963+0000","last_became_peered":"2024-04-24T23:34:54.374963+0000","last_unstale":"2024-04-24T23:34:59.502060+0000","last_undegraded":"2024-04-24T23:34:59.502060+0000","last_fullsized":"2024-04-24T23:34:59.502060+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:47:52.460508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396918+0000","last_change":"2024-04-24T23:34:53.360531+0000","last_active":"2024-04-24T23:34:59.396918+0000","last_peered":"2024-04-24T23:34:59.396918+0000","last_clean":"2024-04-24T23:34:59.396918+0000","last_became_active":"2024-04-24T23:34:53.360247+0000","last_became_peered":"2024-04-24T23:34:53.360247+0000","last_unstale":"2024-04-24T23:34:59.396918+0000","last_undegraded":"2024-04-24T23:34:59.396918+0000","last_fullsized":"2024-04-24T23:34:59.396918+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:57:14.381644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540673+0000","last_change":"2024-04-24T23:34:58.729798+0000","last_active":"2024-04-24T23:35:00.540673+0000","last_peered":"2024-04-24T23:35:00.540673+0000","last_clean":"2024-04-24T23:35:00.540673+0000","last_became_active":"2024-04-24T23:34:58.729345+0000","last_became_peered":"2024-04-24T23:34:58.729345+0000","last_unstale":"2024-04-24T23:35:00.540673+0000","last_undegraded":"2024-04-24T23:35:00.540673+0000","last_fullsized":"2024-04-24T23:35:00.540673+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:06.670518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501916+0000","last_change":"2024-04-24T23:34:54.377593+0000","last_active":"2024-04-24T23:34:59.501916+0000","last_peered":"2024-04-24T23:34:59.501916+0000","last_clean":"2024-04-24T23:34:59.501916+0000","last_became_active":"2024-04-24T23:34:54.377270+0000","last_became_peered":"2024-04-24T23:34:54.377270+0000","last_unstale":"2024-04-24T23:34:59.501916+0000","last_undegraded":"2024-04-24T23:34:59.501916+0000","last_fullsized":"2024-04-24T23:34:59.501916+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:26.261786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397587+0000","last_change":"2024-04-24T23:34:53.352981+0000","last_active":"2024-04-24T23:34:59.397587+0000","last_peered":"2024-04-24T23:34:59.397587+0000","last_clean":"2024-04-24T23:34:59.397587+0000","last_became_active":"2024-04-24T23:34:53.352669+0000","last_became_peered":"2024-04-24T23:34:53.352669+0000","last_unstale":"2024-04-24T23:34:59.397587+0000","last_undegraded":"2024-04-24T23:34:59.397587+0000","last_fullsized":"2024-04-24T23:34:59.397587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:09:49.550050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396650+0000","last_change":"2024-04-24T23:34:58.399234+0000","last_active":"2024-04-24T23:34:59.396650+0000","last_peered":"2024-04-24T23:34:59.396650+0000","last_clean":"2024-04-24T23:34:59.396650+0000","last_became_active":"2024-04-24T23:34:58.398824+0000","last_became_peered":"2024-04-24T23:34:58.398824+0000","last_unstale":"2024-04-24T23:34:59.396650+0000","last_undegraded":"2024-04-24T23:34:59.396650+0000","last_fullsized":"2024-04-24T23:34:59.396650+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:22.244493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501466+0000","last_change":"2024-04-24T23:34:54.376800+0000","last_active":"2024-04-24T23:34:59.501466+0000","last_peered":"2024-04-24T23:34:59.501466+0000","last_clean":"2024-04-24T23:34:59.501466+0000","last_became_active":"2024-04-24T23:34:54.376512+0000","last_became_peered":"2024-04-24T23:34:54.376512+0000","last_unstale":"2024-04-24T23:34:59.501466+0000","last_undegraded":"2024-04-24T23:34:59.501466+0000","last_fullsized":"2024-04-24T23:34:59.501466+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:49.048141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396460+0000","last_change":"2024-04-24T23:34:53.355307+0000","last_active":"2024-04-24T23:34:59.396460+0000","last_peered":"2024-04-24T23:34:59.396460+0000","last_clean":"2024-04-24T23:34:59.396460+0000","last_became_active":"2024-04-24T23:34:53.354988+0000","last_became_peered":"2024-04-24T23:34:53.354988+0000","last_unstale":"2024-04-24T23:34:59.396460+0000","last_undegraded":"2024-04-24T23:34:59.396460+0000","last_fullsized":"2024-04-24T23:34:59.396460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:39.457116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.427781+0000","last_change":"2024-04-24T23:34:58.399188+0000","last_active":"2024-04-24T23:34:59.427781+0000","last_peered":"2024-04-24T23:34:59.427781+0000","last_clean":"2024-04-24T23:34:59.427781+0000","last_became_active":"2024-04-24T23:34:58.398457+0000","last_became_peered":"2024-04-24T23:34:58.398457+0000","last_unstale":"2024-04-24T23:34:59.427781+0000","last_undegraded":"2024-04-24T23:34:59.427781+0000","last_fullsized":"2024-04-24T23:34:59.427781+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:50:21.658540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540120+0000","last_change":"2024-04-24T23:34:54.372150+0000","last_active":"2024-04-24T23:35:00.540120+0000","last_peered":"2024-04-24T23:35:00.540120+0000","last_clean":"2024-04-24T23:35:00.540120+0000","last_became_active":"2024-04-24T23:34:54.371703+0000","last_became_peered":"2024-04-24T23:34:54.371703+0000","last_unstale":"2024-04-24T23:35:00.540120+0000","last_undegraded":"2024-04-24T23:35:00.540120+0000","last_fullsized":"2024-04-24T23:35:00.540120+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:15.477061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540199+0000","last_change":"2024-04-24T23:34:53.363787+0000","last_active":"2024-04-24T23:35:00.540199+0000","last_peered":"2024-04-24T23:35:00.540199+0000","last_clean":"2024-04-24T23:35:00.540199+0000","last_became_active":"2024-04-24T23:34:53.363515+0000","last_became_peered":"2024-04-24T23:34:53.363515+0000","last_unstale":"2024-04-24T23:35:00.540199+0000","last_undegraded":"2024-04-24T23:35:00.540199+0000","last_fullsized":"2024-04-24T23:35:00.540199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:50.222015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397429+0000","last_change":"2024-04-24T23:34:58.728420+0000","last_active":"2024-04-24T23:34:59.397429+0000","last_peered":"2024-04-24T23:34:59.397429+0000","last_clean":"2024-04-24T23:34:59.397429+0000","last_became_active":"2024-04-24T23:34:58.727425+0000","last_became_peered":"2024-04-24T23:34:58.727425+0000","last_unstale":"2024-04-24T23:34:59.397429+0000","last_undegraded":"2024-04-24T23:34:59.397429+0000","last_fullsized":"2024-04-24T23:34:59.397429+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:19:14.487689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501575+0000","last_change":"2024-04-24T23:34:54.379296+0000","last_active":"2024-04-24T23:34:59.501575+0000","last_peered":"2024-04-24T23:34:59.501575+0000","last_clean":"2024-04-24T23:34:59.501575+0000","last_became_active":"2024-04-24T23:34:54.379016+0000","last_became_peered":"2024-04-24T23:34:54.379016+0000","last_unstale":"2024-04-24T23:34:59.501575+0000","last_undegraded":"2024-04-24T23:34:59.501575+0000","last_fullsized":"2024-04-24T23:34:59.501575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:37.323551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540560+0000","last_change":"2024-04-24T23:34:53.360210+0000","last_active":"2024-04-24T23:35:00.540560+0000","last_peered":"2024-04-24T23:35:00.540560+0000","last_clean":"2024-04-24T23:35:00.540560+0000","last_became_active":"2024-04-24T23:34:53.359581+0000","last_became_peered":"2024-04-24T23:34:53.359581+0000","last_unstale":"2024-04-24T23:35:00.540560+0000","last_undegraded":"2024-04-24T23:35:00.540560+0000","last_fullsized":"2024-04-24T23:35:00.540560+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:14.716741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397168+0000","last_change":"2024-04-24T23:34:58.728491+0000","last_active":"2024-04-24T23:34:59.397168+0000","last_peered":"2024-04-24T23:34:59.397168+0000","last_clean":"2024-04-24T23:34:59.397168+0000","last_became_active":"2024-04-24T23:34:58.727686+0000","last_became_peered":"2024-04-24T23:34:58.727686+0000","last_unstale":"2024-04-24T23:34:59.397168+0000","last_undegraded":"2024-04-24T23:34:59.397168+0000","last_fullsized":"2024-04-24T23:34:59.397168+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:46.282712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.427650+0000","last_change":"2024-04-24T23:34:54.372009+0000","last_active":"2024-04-24T23:34:59.427650+0000","last_peered":"2024-04-24T23:34:59.427650+0000","last_clean":"2024-04-24T23:34:59.427650+0000","last_became_active":"2024-04-24T23:34:54.371720+0000","last_became_peered":"2024-04-24T23:34:54.371720+0000","last_unstale":"2024-04-24T23:34:59.427650+0000","last_undegraded":"2024-04-24T23:34:59.427650+0000","last_fullsized":"2024-04-24T23:34:59.427650+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:04:07.487367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.428005+0000","last_change":"2024-04-24T23:34:53.353738+0000","last_active":"2024-04-24T23:34:59.428005+0000","last_peered":"2024-04-24T23:34:59.428005+0000","last_clean":"2024-04-24T23:34:59.428005+0000","last_became_active":"2024-04-24T23:34:53.353452+0000","last_became_peered":"2024-04-24T23:34:53.353452+0000","last_unstale":"2024-04-24T23:34:59.428005+0000","last_undegraded":"2024-04-24T23:34:59.428005+0000","last_fullsized":"2024-04-24T23:34:59.428005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:33:43.693095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.541104+0000","last_change":"2024-04-24T23:34:58.728503+0000","last_active":"2024-04-24T23:35:00.541104+0000","last_peered":"2024-04-24T23:35:00.541104+0000","last_clean":"2024-04-24T23:35:00.541104+0000","last_became_active":"2024-04-24T23:34:58.727921+0000","last_became_peered":"2024-04-24T23:34:58.727921+0000","last_unstale":"2024-04-24T23:35:00.541104+0000","last_undegraded":"2024-04-24T23:35:00.541104+0000","last_fullsized":"2024-04-24T23:35:00.541104+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:19:40.928966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501636+0000","last_change":"2024-04-24T23:34:54.368532+0000","last_active":"2024-04-24T23:34:59.501636+0000","last_peered":"2024-04-24T23:34:59.501636+0000","last_clean":"2024-04-24T23:34:59.501636+0000","last_became_active":"2024-04-24T23:34:54.367903+0000","last_became_peered":"2024-04-24T23:34:54.367903+0000","last_unstale":"2024-04-24T23:34:59.501636+0000","last_undegraded":"2024-04-24T23:34:59.501636+0000","last_fullsized":"2024-04-24T23:34:59.501636+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:49.372040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:03.674709+0000","last_change":"2024-04-24T23:34:53.362758+0000","last_active":"2024-04-24T23:35:03.674709+0000","last_peered":"2024-04-24T23:35:03.674709+0000","last_clean":"2024-04-24T23:35:03.674709+0000","last_became_active":"2024-04-24T23:34:53.362230+0000","last_became_peered":"2024-04-24T23:34:53.362230+0000","last_unstale":"2024-04-24T23:35:03.674709+0000","last_undegraded":"2024-04-24T23:35:03.674709+0000","last_fullsized":"2024-04-24T23:35:03.674709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:14.716741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.881050+0000","last_change":"2024-04-24T23:34:58.400881+0000","last_active":"2024-04-24T23:34:59.881050+0000","last_peered":"2024-04-24T23:34:59.881050+0000","last_clean":"2024-04-24T23:34:59.881050+0000","last_became_active":"2024-04-24T23:34:58.400429+0000","last_became_peered":"2024-04-24T23:34:58.400429+0000","last_unstale":"2024-04-24T23:34:59.881050+0000","last_undegraded":"2024-04-24T23:34:59.881050+0000","last_fullsized":"2024-04-24T23:34:59.881050+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:50:21.658540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397180+0000","last_change":"2024-04-24T23:34:54.373603+0000","last_active":"2024-04-24T23:34:59.397180+0000","last_peered":"2024-04-24T23:34:59.397180+0000","last_clean":"2024-04-24T23:34:59.397180+0000","last_became_active":"2024-04-24T23:34:54.373347+0000","last_became_peered":"2024-04-24T23:34:54.373347+0000","last_unstale":"2024-04-24T23:34:59.397180+0000","last_undegraded":"2024-04-24T23:34:59.397180+0000","last_fullsized":"2024-04-24T23:34:59.397180+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:13.263529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.881250+0000","last_change":"2024-04-24T23:34:53.354673+0000","last_active":"2024-04-24T23:34:59.881250+0000","last_peered":"2024-04-24T23:34:59.881250+0000","last_clean":"2024-04-24T23:34:59.881250+0000","last_became_active":"2024-04-24T23:34:53.354394+0000","last_became_peered":"2024-04-24T23:34:53.354394+0000","last_unstale":"2024-04-24T23:34:59.881250+0000","last_undegraded":"2024-04-24T23:34:59.881250+0000","last_fullsized":"2024-04-24T23:34:59.881250+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:14.716741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397330+0000","last_change":"2024-04-24T23:34:58.397739+0000","last_active":"2024-04-24T23:34:59.397330+0000","last_peered":"2024-04-24T23:34:59.397330+0000","last_clean":"2024-04-24T23:34:59.397330+0000","last_became_active":"2024-04-24T23:34:58.397353+0000","last_became_peered":"2024-04-24T23:34:58.397353+0000","last_unstale":"2024-04-24T23:34:59.397330+0000","last_undegraded":"2024-04-24T23:34:59.397330+0000","last_fullsized":"2024-04-24T23:34:59.397330+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:55.672101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.427967+0000","last_change":"2024-04-24T23:34:54.383348+0000","last_active":"2024-04-24T23:34:59.427967+0000","last_peered":"2024-04-24T23:34:59.427967+0000","last_clean":"2024-04-24T23:34:59.427967+0000","last_became_active":"2024-04-24T23:34:54.383082+0000","last_became_peered":"2024-04-24T23:34:54.383082+0000","last_unstale":"2024-04-24T23:34:59.427967+0000","last_undegraded":"2024-04-24T23:34:59.427967+0000","last_fullsized":"2024-04-24T23:34:59.427967+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:56:17.763159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540591+0000","last_change":"2024-04-24T23:34:53.364540+0000","last_active":"2024-04-24T23:35:00.540591+0000","last_peered":"2024-04-24T23:35:00.540591+0000","last_clean":"2024-04-24T23:35:00.540591+0000","last_became_active":"2024-04-24T23:34:53.364259+0000","last_became_peered":"2024-04-24T23:34:53.364259+0000","last_unstale":"2024-04-24T23:35:00.540591+0000","last_undegraded":"2024-04-24T23:35:00.540591+0000","last_fullsized":"2024-04-24T23:35:00.540591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:41.422591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.395771+0000","last_change":"2024-04-24T23:34:58.405865+0000","last_active":"2024-04-24T23:34:59.395771+0000","last_peered":"2024-04-24T23:34:59.395771+0000","last_clean":"2024-04-24T23:34:59.395771+0000","last_became_active":"2024-04-24T23:34:58.405032+0000","last_became_peered":"2024-04-24T23:34:58.405032+0000","last_unstale":"2024-04-24T23:34:59.395771+0000","last_undegraded":"2024-04-24T23:34:59.395771+0000","last_fullsized":"2024-04-24T23:34:59.395771+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:54.012389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540495+0000","last_change":"2024-04-24T23:34:54.383631+0000","last_active":"2024-04-24T23:35:00.540495+0000","last_peered":"2024-04-24T23:35:00.540495+0000","last_clean":"2024-04-24T23:35:00.540495+0000","last_became_active":"2024-04-24T23:34:54.383354+0000","last_became_peered":"2024-04-24T23:34:54.383354+0000","last_unstale":"2024-04-24T23:35:00.540495+0000","last_undegraded":"2024-04-24T23:35:00.540495+0000","last_fullsized":"2024-04-24T23:35:00.540495+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:50:17.621315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.541421+0000","last_change":"2024-04-24T23:34:53.362409+0000","last_active":"2024-04-24T23:35:00.541421+0000","last_peered":"2024-04-24T23:35:00.541421+0000","last_clean":"2024-04-24T23:35:00.541421+0000","last_became_active":"2024-04-24T23:34:53.362103+0000","last_became_peered":"2024-04-24T23:34:53.362103+0000","last_unstale":"2024-04-24T23:35:00.541421+0000","last_undegraded":"2024-04-24T23:35:00.541421+0000","last_fullsized":"2024-04-24T23:35:00.541421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:57:14.381644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.395785+0000","last_change":"2024-04-24T23:34:58.730088+0000","last_active":"2024-04-24T23:34:59.395785+0000","last_peered":"2024-04-24T23:34:59.395785+0000","last_clean":"2024-04-24T23:34:59.395785+0000","last_became_active":"2024-04-24T23:34:58.729217+0000","last_became_peered":"2024-04-24T23:34:58.729217+0000","last_unstale":"2024-04-24T23:34:59.395785+0000","last_undegraded":"2024-04-24T23:34:59.395785+0000","last_fullsized":"2024-04-24T23:34:59.395785+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:12:14.243770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.427823+0000","last_change":"2024-04-24T23:34:58.728892+0000","last_active":"2024-04-24T23:34:59.427823+0000","last_peered":"2024-04-24T23:34:59.427823+0000","last_clean":"2024-04-24T23:34:59.427823+0000","last_became_active":"2024-04-24T23:34:58.727914+0000","last_became_peered":"2024-04-24T23:34:58.727914+0000","last_unstale":"2024-04-24T23:34:59.427823+0000","last_undegraded":"2024-04-24T23:34:59.427823+0000","last_fullsized":"2024-04-24T23:34:59.427823+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:56.609480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.880773+0000","last_change":"2024-04-24T23:34:54.363973+0000","last_active":"2024-04-24T23:34:59.880773+0000","last_peered":"2024-04-24T23:34:59.880773+0000","last_clean":"2024-04-24T23:34:59.880773+0000","last_became_active":"2024-04-24T23:34:54.363724+0000","last_became_peered":"2024-04-24T23:34:54.363724+0000","last_unstale":"2024-04-24T23:34:59.880773+0000","last_undegraded":"2024-04-24T23:34:59.880773+0000","last_fullsized":"2024-04-24T23:34:59.880773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:03:24.745895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.541110+0000","last_change":"2024-04-24T23:34:53.360041+0000","last_active":"2024-04-24T23:35:00.541110+0000","last_peered":"2024-04-24T23:35:00.541110+0000","last_clean":"2024-04-24T23:35:00.541110+0000","last_became_active":"2024-04-24T23:34:53.359707+0000","last_became_peered":"2024-04-24T23:34:53.359707+0000","last_unstale":"2024-04-24T23:35:00.541110+0000","last_undegraded":"2024-04-24T23:35:00.541110+0000","last_fullsized":"2024-04-24T23:35:00.541110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:50.222015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.541453+0000","last_change":"2024-04-24T23:34:58.399618+0000","last_active":"2024-04-24T23:35:00.541453+0000","last_peered":"2024-04-24T23:35:00.541453+0000","last_clean":"2024-04-24T23:35:00.541453+0000","last_became_active":"2024-04-24T23:34:58.399182+0000","last_became_peered":"2024-04-24T23:34:58.399182+0000","last_unstale":"2024-04-24T23:35:00.541453+0000","last_undegraded":"2024-04-24T23:35:00.541453+0000","last_fullsized":"2024-04-24T23:35:00.541453+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:55.672101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.395731+0000","last_change":"2024-04-24T23:34:54.369729+0000","last_active":"2024-04-24T23:34:59.395731+0000","last_peered":"2024-04-24T23:34:59.395731+0000","last_clean":"2024-04-24T23:34:59.395731+0000","last_became_active":"2024-04-24T23:34:54.369155+0000","last_became_peered":"2024-04-24T23:34:54.369155+0000","last_unstale":"2024-04-24T23:34:59.395731+0000","last_undegraded":"2024-04-24T23:34:59.395731+0000","last_fullsized":"2024-04-24T23:34:59.395731+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:46:28.181996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.395890+0000","last_change":"2024-04-24T23:34:53.365328+0000","last_active":"2024-04-24T23:34:59.395890+0000","last_peered":"2024-04-24T23:34:59.395890+0000","last_clean":"2024-04-24T23:34:59.395890+0000","last_became_active":"2024-04-24T23:34:53.365029+0000","last_became_peered":"2024-04-24T23:34:53.365029+0000","last_unstale":"2024-04-24T23:34:59.395890+0000","last_undegraded":"2024-04-24T23:34:59.395890+0000","last_fullsized":"2024-04-24T23:34:59.395890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:48:30.961870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540306+0000","last_change":"2024-04-24T23:34:53.365123+0000","last_active":"2024-04-24T23:35:00.540306+0000","last_peered":"2024-04-24T23:35:00.540306+0000","last_clean":"2024-04-24T23:35:00.540306+0000","last_became_active":"2024-04-24T23:34:53.364821+0000","last_became_peered":"2024-04-24T23:34:53.364821+0000","last_unstale":"2024-04-24T23:35:00.540306+0000","last_undegraded":"2024-04-24T23:35:00.540306+0000","last_fullsized":"2024-04-24T23:35:00.540306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:39.457116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.427901+0000","last_change":"2024-04-24T23:34:54.376037+0000","last_active":"2024-04-24T23:34:59.427901+0000","last_peered":"2024-04-24T23:34:59.427901+0000","last_clean":"2024-04-24T23:34:59.427901+0000","last_became_active":"2024-04-24T23:34:54.375689+0000","last_became_peered":"2024-04-24T23:34:54.375689+0000","last_unstale":"2024-04-24T23:34:59.427901+0000","last_undegraded":"2024-04-24T23:34:59.427901+0000","last_fullsized":"2024-04-24T23:34:59.427901+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:47.446459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397192+0000","last_change":"2024-04-24T23:34:58.404439+0000","last_active":"2024-04-24T23:34:59.397192+0000","last_peered":"2024-04-24T23:34:59.397192+0000","last_clean":"2024-04-24T23:34:59.397192+0000","last_became_active":"2024-04-24T23:34:58.404053+0000","last_became_peered":"2024-04-24T23:34:58.404053+0000","last_unstale":"2024-04-24T23:34:59.397192+0000","last_undegraded":"2024-04-24T23:34:59.397192+0000","last_fullsized":"2024-04-24T23:34:59.397192+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:46.292263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396770+0000","last_change":"2024-04-24T23:34:53.363066+0000","last_active":"2024-04-24T23:34:59.396770+0000","last_peered":"2024-04-24T23:34:59.396770+0000","last_clean":"2024-04-24T23:34:59.396770+0000","last_became_active":"2024-04-24T23:34:53.362548+0000","last_became_peered":"2024-04-24T23:34:53.362548+0000","last_unstale":"2024-04-24T23:34:59.396770+0000","last_undegraded":"2024-04-24T23:34:59.396770+0000","last_fullsized":"2024-04-24T23:34:59.396770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:21:13.338964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397349+0000","last_change":"2024-04-24T23:34:54.365704+0000","last_active":"2024-04-24T23:34:59.397349+0000","last_peered":"2024-04-24T23:34:59.397349+0000","last_clean":"2024-04-24T23:34:59.397349+0000","last_became_active":"2024-04-24T23:34:54.364814+0000","last_became_peered":"2024-04-24T23:34:54.364814+0000","last_unstale":"2024-04-24T23:34:59.397349+0000","last_undegraded":"2024-04-24T23:34:59.397349+0000","last_fullsized":"2024-04-24T23:34:59.397349+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:22.094785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397247+0000","last_change":"2024-04-24T23:34:58.407186+0000","last_active":"2024-04-24T23:34:59.397247+0000","last_peered":"2024-04-24T23:34:59.397247+0000","last_clean":"2024-04-24T23:34:59.397247+0000","last_became_active":"2024-04-24T23:34:58.406811+0000","last_became_peered":"2024-04-24T23:34:58.406811+0000","last_unstale":"2024-04-24T23:34:59.397247+0000","last_undegraded":"2024-04-24T23:34:59.397247+0000","last_fullsized":"2024-04-24T23:34:59.397247+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:55.861555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.539983+0000","last_change":"2024-04-24T23:34:54.375142+0000","last_active":"2024-04-24T23:35:00.539983+0000","last_peered":"2024-04-24T23:35:00.539983+0000","last_clean":"2024-04-24T23:35:00.539983+0000","last_became_active":"2024-04-24T23:34:54.374844+0000","last_became_peered":"2024-04-24T23:34:54.374844+0000","last_unstale":"2024-04-24T23:35:00.539983+0000","last_undegraded":"2024-04-24T23:35:00.539983+0000","last_fullsized":"2024-04-24T23:35:00.539983+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:29:07.593498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396456+0000","last_change":"2024-04-24T23:34:54.383574+0000","last_active":"2024-04-24T23:34:59.396456+0000","last_peered":"2024-04-24T23:34:59.396456+0000","last_clean":"2024-04-24T23:34:59.396456+0000","last_became_active":"2024-04-24T23:34:54.383253+0000","last_became_peered":"2024-04-24T23:34:54.383253+0000","last_unstale":"2024-04-24T23:34:59.396456+0000","last_undegraded":"2024-04-24T23:34:59.396456+0000","last_fullsized":"2024-04-24T23:34:59.396456+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:43.211314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540447+0000","last_change":"2024-04-24T23:34:54.372208+0000","last_active":"2024-04-24T23:35:00.540447+0000","last_peered":"2024-04-24T23:35:00.540447+0000","last_clean":"2024-04-24T23:35:00.540447+0000","last_became_active":"2024-04-24T23:34:54.371910+0000","last_became_peered":"2024-04-24T23:34:54.371910+0000","last_unstale":"2024-04-24T23:35:00.540447+0000","last_undegraded":"2024-04-24T23:35:00.540447+0000","last_fullsized":"2024-04-24T23:35:00.540447+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:07:28.785202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.427860+0000","last_change":"2024-04-24T23:34:54.366271+0000","last_active":"2024-04-24T23:34:59.427860+0000","last_peered":"2024-04-24T23:34:59.427860+0000","last_clean":"2024-04-24T23:34:59.427860+0000","last_became_active":"2024-04-24T23:34:54.366005+0000","last_became_peered":"2024-04-24T23:34:54.366005+0000","last_unstale":"2024-04-24T23:34:59.427860+0000","last_undegraded":"2024-04-24T23:34:59.427860+0000","last_fullsized":"2024-04-24T23:34:59.427860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:03:24.745895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397136+0000","last_change":"2024-04-24T23:34:54.360529+0000","last_active":"2024-04-24T23:34:59.397136+0000","last_peered":"2024-04-24T23:34:59.397136+0000","last_clean":"2024-04-24T23:34:59.397136+0000","last_became_active":"2024-04-24T23:34:54.360266+0000","last_became_peered":"2024-04-24T23:34:54.360266+0000","last_unstale":"2024-04-24T23:34:59.397136+0000","last_undegraded":"2024-04-24T23:34:59.397136+0000","last_fullsized":"2024-04-24T23:34:59.397136+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:49:47.246193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.881089+0000","last_change":"2024-04-24T23:34:54.365744+0000","last_active":"2024-04-24T23:34:59.881089+0000","last_peered":"2024-04-24T23:34:59.881089+0000","last_clean":"2024-04-24T23:34:59.881089+0000","last_became_active":"2024-04-24T23:34:54.365493+0000","last_became_peered":"2024-04-24T23:34:54.365493+0000","last_unstale":"2024-04-24T23:34:59.881089+0000","last_undegraded":"2024-04-24T23:34:59.881089+0000","last_fullsized":"2024-04-24T23:34:59.881089+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:15.477061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.541221+0000","last_change":"2024-04-24T23:34:54.379642+0000","last_active":"2024-04-24T23:35:00.541221+0000","last_peered":"2024-04-24T23:35:00.541221+0000","last_clean":"2024-04-24T23:35:00.541221+0000","last_became_active":"2024-04-24T23:34:54.378568+0000","last_became_peered":"2024-04-24T23:34:54.378568+0000","last_unstale":"2024-04-24T23:35:00.541221+0000","last_undegraded":"2024-04-24T23:35:00.541221+0000","last_fullsized":"2024-04-24T23:35:00.541221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:42.199856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397390+0000","last_change":"2024-04-24T23:34:54.363700+0000","last_active":"2024-04-24T23:34:59.397390+0000","last_peered":"2024-04-24T23:34:59.397390+0000","last_clean":"2024-04-24T23:34:59.397390+0000","last_became_active":"2024-04-24T23:34:54.363439+0000","last_became_peered":"2024-04-24T23:34:54.363439+0000","last_unstale":"2024-04-24T23:34:59.397390+0000","last_undegraded":"2024-04-24T23:34:59.397390+0000","last_fullsized":"2024-04-24T23:34:59.397390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:28:56.048178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.541000+0000","last_change":"2024-04-24T23:34:54.375460+0000","last_active":"2024-04-24T23:35:00.541000+0000","last_peered":"2024-04-24T23:35:00.541000+0000","last_clean":"2024-04-24T23:35:00.541000+0000","last_became_active":"2024-04-24T23:34:54.375123+0000","last_became_peered":"2024-04-24T23:34:54.375123+0000","last_unstale":"2024-04-24T23:35:00.541000+0000","last_undegraded":"2024-04-24T23:35:00.541000+0000","last_fullsized":"2024-04-24T23:35:00.541000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:18.207267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396028+0000","last_change":"2024-04-24T23:34:54.369509+0000","last_active":"2024-04-24T23:34:59.396028+0000","last_peered":"2024-04-24T23:34:59.396028+0000","last_clean":"2024-04-24T23:34:59.396028+0000","last_became_active":"2024-04-24T23:34:54.369219+0000","last_became_peered":"2024-04-24T23:34:54.369219+0000","last_unstale":"2024-04-24T23:34:59.396028+0000","last_undegraded":"2024-04-24T23:34:59.396028+0000","last_fullsized":"2024-04-24T23:34:59.396028+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:09:37.649503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540426+0000","last_change":"2024-04-24T23:34:54.384240+0000","last_active":"2024-04-24T23:35:00.540426+0000","last_peered":"2024-04-24T23:35:00.540426+0000","last_clean":"2024-04-24T23:35:00.540426+0000","last_became_active":"2024-04-24T23:34:54.383780+0000","last_became_peered":"2024-04-24T23:34:54.383780+0000","last_unstale":"2024-04-24T23:35:00.540426+0000","last_undegraded":"2024-04-24T23:35:00.540426+0000","last_fullsized":"2024-04-24T23:35:00.540426+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:49.048141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.427635+0000","last_change":"2024-04-24T23:34:54.371052+0000","last_active":"2024-04-24T23:34:59.427635+0000","last_peered":"2024-04-24T23:34:59.427635+0000","last_clean":"2024-04-24T23:34:59.427635+0000","last_became_active":"2024-04-24T23:34:54.370793+0000","last_became_peered":"2024-04-24T23:34:54.370793+0000","last_unstale":"2024-04-24T23:34:59.427635+0000","last_undegraded":"2024-04-24T23:34:59.427635+0000","last_fullsized":"2024-04-24T23:34:59.427635+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:29:07.593498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540693+0000","last_change":"2024-04-24T23:34:54.377190+0000","last_active":"2024-04-24T23:35:00.540693+0000","last_peered":"2024-04-24T23:35:00.540693+0000","last_clean":"2024-04-24T23:35:00.540693+0000","last_became_active":"2024-04-24T23:34:54.376940+0000","last_became_peered":"2024-04-24T23:34:54.376940+0000","last_unstale":"2024-04-24T23:35:00.540693+0000","last_undegraded":"2024-04-24T23:35:00.540693+0000","last_fullsized":"2024-04-24T23:35:00.540693+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:03:24.745895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501917+0000","last_change":"2024-04-24T23:34:54.368770+0000","last_active":"2024-04-24T23:34:59.501917+0000","last_peered":"2024-04-24T23:34:59.501917+0000","last_clean":"2024-04-24T23:34:59.501917+0000","last_became_active":"2024-04-24T23:34:54.368228+0000","last_became_peered":"2024-04-24T23:34:54.368228+0000","last_unstale":"2024-04-24T23:34:59.501917+0000","last_undegraded":"2024-04-24T23:34:59.501917+0000","last_fullsized":"2024-04-24T23:34:59.501917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:51.634876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396819+0000","last_change":"2024-04-24T23:34:54.373038+0000","last_active":"2024-04-24T23:34:59.396819+0000","last_peered":"2024-04-24T23:34:59.396819+0000","last_clean":"2024-04-24T23:34:59.396819+0000","last_became_active":"2024-04-24T23:34:54.372766+0000","last_became_peered":"2024-04-24T23:34:54.372766+0000","last_unstale":"2024-04-24T23:34:59.396819+0000","last_undegraded":"2024-04-24T23:34:59.396819+0000","last_fullsized":"2024-04-24T23:34:59.396819+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:13.263529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.541061+0000","last_change":"2024-04-24T23:34:54.383733+0000","last_active":"2024-04-24T23:35:00.541061+0000","last_peered":"2024-04-24T23:35:00.541061+0000","last_clean":"2024-04-24T23:35:00.541061+0000","last_became_active":"2024-04-24T23:34:54.383463+0000","last_became_peered":"2024-04-24T23:34:54.383463+0000","last_unstale":"2024-04-24T23:35:00.541061+0000","last_undegraded":"2024-04-24T23:35:00.541061+0000","last_fullsized":"2024-04-24T23:35:00.541061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:51.634876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540883+0000","last_change":"2024-04-24T23:34:54.380959+0000","last_active":"2024-04-24T23:35:00.540883+0000","last_peered":"2024-04-24T23:35:00.540883+0000","last_clean":"2024-04-24T23:35:00.540883+0000","last_became_active":"2024-04-24T23:34:54.380702+0000","last_became_peered":"2024-04-24T23:34:54.380702+0000","last_unstale":"2024-04-24T23:35:00.540883+0000","last_undegraded":"2024-04-24T23:35:00.540883+0000","last_fullsized":"2024-04-24T23:35:00.540883+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:45.424046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.880840+0000","last_change":"2024-04-24T23:34:54.358461+0000","last_active":"2024-04-24T23:34:59.880840+0000","last_peered":"2024-04-24T23:34:59.880840+0000","last_clean":"2024-04-24T23:34:59.880840+0000","last_became_active":"2024-04-24T23:34:54.358203+0000","last_became_peered":"2024-04-24T23:34:54.358203+0000","last_unstale":"2024-04-24T23:34:59.880840+0000","last_undegraded":"2024-04-24T23:34:59.880840+0000","last_fullsized":"2024-04-24T23:34:59.880840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:18.207267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.427594+0000","last_change":"2024-04-24T23:34:54.367860+0000","last_active":"2024-04-24T23:34:59.427594+0000","last_peered":"2024-04-24T23:34:59.427594+0000","last_clean":"2024-04-24T23:34:59.427594+0000","last_became_active":"2024-04-24T23:34:54.367595+0000","last_became_peered":"2024-04-24T23:34:54.367595+0000","last_unstale":"2024-04-24T23:34:59.427594+0000","last_undegraded":"2024-04-24T23:34:59.427594+0000","last_fullsized":"2024-04-24T23:34:59.427594+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:40:40.928386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396558+0000","last_change":"2024-04-24T23:34:54.373640+0000","last_active":"2024-04-24T23:34:59.396558+0000","last_peered":"2024-04-24T23:34:59.396558+0000","last_clean":"2024-04-24T23:34:59.396558+0000","last_became_active":"2024-04-24T23:34:54.373404+0000","last_became_peered":"2024-04-24T23:34:54.373404+0000","last_unstale":"2024-04-24T23:34:59.396558+0000","last_undegraded":"2024-04-24T23:34:59.396558+0000","last_fullsized":"2024-04-24T23:34:59.396558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:41:21.841042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397052+0000","last_change":"2024-04-24T23:34:54.377573+0000","last_active":"2024-04-24T23:34:59.397052+0000","last_peered":"2024-04-24T23:34:59.397052+0000","last_clean":"2024-04-24T23:34:59.397052+0000","last_became_active":"2024-04-24T23:34:54.377277+0000","last_became_peered":"2024-04-24T23:34:54.377277+0000","last_unstale":"2024-04-24T23:34:59.397052+0000","last_undegraded":"2024-04-24T23:34:59.397052+0000","last_fullsized":"2024-04-24T23:34:59.397052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:43.211314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540693+0000","last_change":"2024-04-24T23:34:54.388134+0000","last_active":"2024-04-24T23:35:00.540693+0000","last_peered":"2024-04-24T23:35:00.540693+0000","last_clean":"2024-04-24T23:35:00.540693+0000","last_became_active":"2024-04-24T23:34:54.387781+0000","last_became_peered":"2024-04-24T23:34:54.387781+0000","last_unstale":"2024-04-24T23:35:00.540693+0000","last_undegraded":"2024-04-24T23:35:00.540693+0000","last_fullsized":"2024-04-24T23:35:00.540693+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:56:17.763159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.541198+0000","last_change":"2024-04-24T23:34:54.388182+0000","last_active":"2024-04-24T23:35:00.541198+0000","last_peered":"2024-04-24T23:35:00.541198+0000","last_clean":"2024-04-24T23:35:00.541198+0000","last_became_active":"2024-04-24T23:34:54.387913+0000","last_became_peered":"2024-04-24T23:34:54.387913+0000","last_unstale":"2024-04-24T23:35:00.541198+0000","last_undegraded":"2024-04-24T23:35:00.541198+0000","last_fullsized":"2024-04-24T23:35:00.541198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:18.207267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.502012+0000","last_change":"2024-04-24T23:34:54.383303+0000","last_active":"2024-04-24T23:34:59.502012+0000","last_peered":"2024-04-24T23:34:59.502012+0000","last_clean":"2024-04-24T23:34:59.502012+0000","last_became_active":"2024-04-24T23:34:54.383024+0000","last_became_peered":"2024-04-24T23:34:54.383024+0000","last_unstale":"2024-04-24T23:34:59.502012+0000","last_undegraded":"2024-04-24T23:34:59.502012+0000","last_fullsized":"2024-04-24T23:34:59.502012+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:45.424046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.395629+0000","last_change":"2024-04-24T23:34:54.382803+0000","last_active":"2024-04-24T23:34:59.395629+0000","last_peered":"2024-04-24T23:34:59.395629+0000","last_clean":"2024-04-24T23:34:59.395629+0000","last_became_active":"2024-04-24T23:34:54.382482+0000","last_became_peered":"2024-04-24T23:34:54.382482+0000","last_unstale":"2024-04-24T23:34:59.395629+0000","last_undegraded":"2024-04-24T23:34:59.395629+0000","last_fullsized":"2024-04-24T23:34:59.395629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:27:33.721584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501319+0000","last_change":"2024-04-24T23:34:54.369167+0000","last_active":"2024-04-24T23:34:59.501319+0000","last_peered":"2024-04-24T23:34:59.501319+0000","last_clean":"2024-04-24T23:34:59.501319+0000","last_became_active":"2024-04-24T23:34:54.368663+0000","last_became_peered":"2024-04-24T23:34:54.368663+0000","last_unstale":"2024-04-24T23:34:59.501319+0000","last_undegraded":"2024-04-24T23:34:59.501319+0000","last_fullsized":"2024-04-24T23:34:59.501319+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:18.207267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540351+0000","last_change":"2024-04-24T23:34:54.380345+0000","last_active":"2024-04-24T23:35:00.540351+0000","last_peered":"2024-04-24T23:35:00.540351+0000","last_clean":"2024-04-24T23:35:00.540351+0000","last_became_active":"2024-04-24T23:34:54.380052+0000","last_became_peered":"2024-04-24T23:34:54.380052+0000","last_unstale":"2024-04-24T23:35:00.540351+0000","last_undegraded":"2024-04-24T23:35:00.540351+0000","last_fullsized":"2024-04-24T23:35:00.540351+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:42.199856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501438+0000","last_change":"2024-04-24T23:34:54.365114+0000","last_active":"2024-04-24T23:34:59.501438+0000","last_peered":"2024-04-24T23:34:59.501438+0000","last_clean":"2024-04-24T23:34:59.501438+0000","last_became_active":"2024-04-24T23:34:54.364605+0000","last_became_peered":"2024-04-24T23:34:54.364605+0000","last_unstale":"2024-04-24T23:34:59.501438+0000","last_undegraded":"2024-04-24T23:34:59.501438+0000","last_fullsized":"2024-04-24T23:34:59.501438+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:07:28.785202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540539+0000","last_change":"2024-04-24T23:34:54.374843+0000","last_active":"2024-04-24T23:35:00.540539+0000","last_peered":"2024-04-24T23:35:00.540539+0000","last_clean":"2024-04-24T23:35:00.540539+0000","last_became_active":"2024-04-24T23:34:54.373117+0000","last_became_peered":"2024-04-24T23:34:54.373117+0000","last_unstale":"2024-04-24T23:35:00.540539+0000","last_undegraded":"2024-04-24T23:35:00.540539+0000","last_fullsized":"2024-04-24T23:35:00.540539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:37.323551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.427656+0000","last_change":"2024-04-24T23:34:54.366079+0000","last_active":"2024-04-24T23:34:59.427656+0000","last_peered":"2024-04-24T23:34:59.427656+0000","last_clean":"2024-04-24T23:34:59.427656+0000","last_became_active":"2024-04-24T23:34:54.365814+0000","last_became_peered":"2024-04-24T23:34:54.365814+0000","last_unstale":"2024-04-24T23:34:59.427656+0000","last_undegraded":"2024-04-24T23:34:59.427656+0000","last_fullsized":"2024-04-24T23:34:59.427656+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:18.207267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.881393+0000","last_change":"2024-04-24T23:34:54.368976+0000","last_active":"2024-04-24T23:34:59.881393+0000","last_peered":"2024-04-24T23:34:59.881393+0000","last_clean":"2024-04-24T23:34:59.881393+0000","last_became_active":"2024-04-24T23:34:54.368663+0000","last_became_peered":"2024-04-24T23:34:54.368663+0000","last_unstale":"2024-04-24T23:34:59.881393+0000","last_undegraded":"2024-04-24T23:34:59.881393+0000","last_fullsized":"2024-04-24T23:34:59.881393+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:56:17.763159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501905+0000","last_change":"2024-04-24T23:34:54.373810+0000","last_active":"2024-04-24T23:34:59.501905+0000","last_peered":"2024-04-24T23:34:59.501905+0000","last_clean":"2024-04-24T23:34:59.501905+0000","last_became_active":"2024-04-24T23:34:54.373531+0000","last_became_peered":"2024-04-24T23:34:54.373531+0000","last_unstale":"2024-04-24T23:34:59.501905+0000","last_undegraded":"2024-04-24T23:34:59.501905+0000","last_fullsized":"2024-04-24T23:34:59.501905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:50:17.621315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65248,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542152,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":331776,"data_stored":694125,"data_compressed":32624,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542153,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":274432,"data_stored":107017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542152,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":103265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542152,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":103301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542152,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":262144,"data_stored":103175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542152,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31444,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340396,"statfs":{"total":96636764160,"available":96604565504,"internally_reserved":0,"allocated":270336,"data_stored":103335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542152,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31520,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340320,"statfs":{"total":96636764160,"available":96604487680,"internally_reserved":0,"allocated":339968,"data_stored":694181,"data_compressed":32624,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31444,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340396,"statfs":{"total":96636764160,"available":96604565504,"internally_reserved":0,"allocated":274432,"data_stored":107017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32624,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32624,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:35:21.740 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:35:21.872 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.871+0000 7ff2033f4640 1 Processor -- start 2024-04-24T23:35:21.873 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.872+0000 7ff2033f4640 1 -- start start 2024-04-24T23:35:21.875 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.872+0000 7ff2033f4640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1fc152b40 0x7ff1fc0c5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:21.875 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.872+0000 7ff2033f4640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff1fc0be360 0x7ff1fc155350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:21.875 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.872+0000 7ff2033f4640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1fc0c8bb0 con 0x7ff1fc0c5b70 2024-04-24T23:35:21.875 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.872+0000 7ff2033f4640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7ff1fc0c8e90 con 0x7ff1fc0be360 2024-04-24T23:35:21.875 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.872+0000 7ff2033f4640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff1fc159480 con 0x7ff1fc152b40 2024-04-24T23:35:21.875 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.873+0000 7ff20098f640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff1fc0be360 0x7ff1fc155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:21.875 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.873+0000 7ff201190640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1fc152b40 0x7ff1fc0c5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:21.875 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.873+0000 7ff20098f640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff1fc0be360 0x7ff1fc155350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.157:45244/0 (socket says 172.21.15.157:45244) 2024-04-24T23:35:21.876 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.873+0000 7ff20098f640 1 -- 172.21.15.157:0/1199853194 learned_addr learned my addr 172.21.15.157:0/1199853194 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:21.876 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.873+0000 7ff201190640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1fc152b40 0x7ff1fc0c5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.157:53054/0 (socket says 172.21.15.157:53054) 2024-04-24T23:35:21.876 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.873+0000 7ff20098f640 1 -- 172.21.15.157:0/1199853194 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1fc152b40 msgr2=0x7ff1fc0c5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:21.876 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.873+0000 7ff20098f640 1 --2- 172.21.15.157:0/1199853194 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1fc152b40 0x7ff1fc0c5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:21.876 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.873+0000 7ff20098f640 1 -- 172.21.15.157:0/1199853194 >> v1:172.21.15.169:6789/0 conn(0x7ff1fc0c5b70 legacy=0x7ff1fc155890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:21.876 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.873+0000 7ff20098f640 1 -- 172.21.15.157:0/1199853194 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1fc159760 con 0x7ff1fc0be360 2024-04-24T23:35:21.876 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.873+0000 7ff201190640 1 --2- 172.21.15.157:0/1199853194 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1fc152b40 0x7ff1fc0c5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:35:21.876 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.874+0000 7ff20098f640 1 --2- 172.21.15.157:0/1199853194 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1fc0be360 0x7ff1fc155350 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec009870 tx=0x7ff1ec02a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84dea790581d7375 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:21.876 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.874+0000 7ff1f3fff640 1 -- 172.21.15.157:0/1199853194 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec037070 con 0x7ff1fc0be360 2024-04-24T23:35:21.876 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.874+0000 7ff1f3fff640 1 -- 172.21.15.157:0/1199853194 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1ec032440 con 0x7ff1fc0be360 2024-04-24T23:35:21.876 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.874+0000 7ff1f3fff640 1 -- 172.21.15.157:0/1199853194 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec032810 con 0x7ff1fc0be360 2024-04-24T23:35:21.876 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.874+0000 7ff2033f4640 1 -- 172.21.15.157:0/1199853194 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1fc0be360 msgr2=0x7ff1fc155350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:21.876 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.874+0000 7ff2033f4640 1 --2- 172.21.15.157:0/1199853194 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1fc0be360 0x7ff1fc155350 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec009870 tx=0x7ff1ec02a130 comp rx=0 tx=0).stop 2024-04-24T23:35:21.877 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.874+0000 7ff2033f4640 1 -- 172.21.15.157:0/1199853194 shutdown_connections 2024-04-24T23:35:21.877 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.874+0000 7ff2033f4640 1 --2- 172.21.15.157:0/1199853194 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1fc0be360 0x7ff1fc155350 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:21.877 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.874+0000 7ff2033f4640 1 --2- 172.21.15.157:0/1199853194 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1fc152b40 0x7ff1fc0c5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:21.877 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.874+0000 7ff2033f4640 1 -- 172.21.15.157:0/1199853194 >> 172.21.15.157:0/1199853194 conn(0x7ff1fc0672c0 msgr2=0x7ff1fc0676d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:21.877 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.874+0000 7ff2033f4640 1 -- 172.21.15.157:0/1199853194 shutdown_connections 2024-04-24T23:35:21.877 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.874+0000 7ff2033f4640 1 -- 172.21.15.157:0/1199853194 wait complete. 2024-04-24T23:35:21.879 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.875+0000 7ff2033f4640 1 Processor -- start 2024-04-24T23:35:21.879 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.876+0000 7ff2033f4640 1 -- start start 2024-04-24T23:35:21.879 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.876+0000 7ff2033f4640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1fc0be360 0x7ff1fc0ab0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:21.879 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.876+0000 7ff2033f4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1fc0c5b70 0x7ff1fc1dfd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:21.879 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.876+0000 7ff2033f4640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1fc152b40 0x7ff1fc147d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:21.879 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.876+0000 7ff2033f4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff1fc0caa00 con 0x7ff1fc0c5b70 2024-04-24T23:35:21.879 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.876+0000 7ff2033f4640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff1fc0cb210 con 0x7ff1fc152b40 2024-04-24T23:35:21.879 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.876+0000 7ff2033f4640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff1fc0cba40 con 0x7ff1fc0be360 2024-04-24T23:35:21.880 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.876+0000 7ff20098f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1fc0c5b70 0x7ff1fc1dfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:21.880 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.876+0000 7ff20098f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1fc0c5b70 0x7ff1fc1dfd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43024/0 (socket says 172.21.15.157:43024) 2024-04-24T23:35:21.880 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.876+0000 7ff201190640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1fc0be360 0x7ff1fc0ab0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:21.880 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.876+0000 7ff201991640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1fc152b40 0x7ff1fc147d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:21.880 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.876+0000 7ff20098f640 1 -- 172.21.15.157:0/1038317765 learned_addr learned my addr 172.21.15.157:0/1038317765 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:21.880 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.877+0000 7ff20098f640 1 -- 172.21.15.157:0/1038317765 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1fc0be360 msgr2=0x7ff1fc0ab0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:21.880 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.877+0000 7ff20098f640 1 --2- 172.21.15.157:0/1038317765 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1fc0be360 0x7ff1fc0ab0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:21.880 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.877+0000 7ff20098f640 1 -- 172.21.15.157:0/1038317765 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1fc152b40 msgr2=0x7ff1fc147d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:21.880 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.877+0000 7ff20098f640 1 --2- 172.21.15.157:0/1038317765 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1fc152b40 0x7ff1fc147d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:21.880 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.877+0000 7ff20098f640 1 -- 172.21.15.157:0/1038317765 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1fc0ab900 con 0x7ff1fc0c5b70 2024-04-24T23:35:21.880 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.877+0000 7ff201190640 1 --2- 172.21.15.157:0/1038317765 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1fc0be360 0x7ff1fc0ab0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:35:21.880 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.877+0000 7ff201991640 1 --2- 172.21.15.157:0/1038317765 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1fc152b40 0x7ff1fc147d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:35:21.880 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.877+0000 7ff20098f640 1 --2- 172.21.15.157:0/1038317765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1fc0c5b70 0x7ff1fc1dfd30 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec000c00 tx=0x7ff1ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:21.881 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.877+0000 7ff1f1ffb640 1 -- 172.21.15.157:0/1038317765 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec037040 con 0x7ff1fc0c5b70 2024-04-24T23:35:21.881 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.877+0000 7ff1f1ffb640 1 -- 172.21.15.157:0/1038317765 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1ec042070 con 0x7ff1fc0c5b70 2024-04-24T23:35:21.881 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.877+0000 7ff2033f4640 1 -- 172.21.15.157:0/1038317765 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1fc0abbe0 con 0x7ff1fc0c5b70 2024-04-24T23:35:21.881 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.877+0000 7ff1f1ffb640 1 -- 172.21.15.157:0/1038317765 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec030880 con 0x7ff1fc0c5b70 2024-04-24T23:35:21.881 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.877+0000 7ff2033f4640 1 -- 172.21.15.157:0/1038317765 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1fc1ec840 con 0x7ff1fc0c5b70 2024-04-24T23:35:21.881 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.878+0000 7ff2033f4640 1 -- 172.21.15.157:0/1038317765 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1fc06c830 con 0x7ff1fc0c5b70 2024-04-24T23:35:21.881 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.879+0000 7ff1f1ffb640 1 -- 172.21.15.157:0/1038317765 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff1ec030a20 con 0x7ff1fc0c5b70 2024-04-24T23:35:21.882 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.879+0000 7ff1f1ffb640 1 --2- 172.21.15.157:0/1038317765 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7ff1e006be60 0x7ff1e006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:21.882 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.879+0000 7ff1f1ffb640 1 -- 172.21.15.157:0/1038317765 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ff1ec0ab860 con 0x7ff1fc0c5b70 2024-04-24T23:35:21.882 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.879+0000 7ff201190640 1 --2- 172.21.15.157:0/1038317765 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7ff1e006be60 0x7ff1e006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:21.882 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.880+0000 7ff201190640 1 --2- 172.21.15.157:0/1038317765 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7ff1e006be60 0x7ff1e006e320 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4009fd0 tx=0x7ff1f400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:21.884 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:21.883+0000 7ff1f1ffb640 1 -- 172.21.15.157:0/1038317765 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff1fc06c830 con 0x7ff1fc0c5b70 2024-04-24T23:35:22.053 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.053+0000 7ff2033f4640 1 -- 172.21.15.157:0/1038317765 --> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff1fc14cbb0 con 0x7ff1e006be60 2024-04-24T23:35:22.062 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.061+0000 7ff1f1ffb640 1 -- 172.21.15.157:0/1038317765 <== mgr.4105 v2:172.21.15.169:6832/662378289 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332608 (secure 0 0 0) 0x7ff1fc14cbb0 con 0x7ff1e006be60 2024-04-24T23:35:22.066 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:35:22.068 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2024-04-24T23:35:22.069 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.065+0000 7ff2033f4640 1 -- 172.21.15.157:0/1038317765 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7ff1e006be60 msgr2=0x7ff1e006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:22.070 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.065+0000 7ff2033f4640 1 --2- 172.21.15.157:0/1038317765 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7ff1e006be60 0x7ff1e006e320 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4009fd0 tx=0x7ff1f400b040 comp rx=0 tx=0).stop 2024-04-24T23:35:22.070 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.065+0000 7ff2033f4640 1 -- 172.21.15.157:0/1038317765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1fc0c5b70 msgr2=0x7ff1fc1dfd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:22.070 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.065+0000 7ff2033f4640 1 --2- 172.21.15.157:0/1038317765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1fc0c5b70 0x7ff1fc1dfd30 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec000c00 tx=0x7ff1ec0046c0 comp rx=0 tx=0).stop 2024-04-24T23:35:22.070 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.065+0000 7ff2033f4640 1 -- 172.21.15.157:0/1038317765 shutdown_connections 2024-04-24T23:35:22.070 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.065+0000 7ff2033f4640 1 --2- 172.21.15.157:0/1038317765 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7ff1e006be60 0x7ff1e006e320 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:22.070 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.065+0000 7ff2033f4640 1 --2- 172.21.15.157:0/1038317765 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1fc152b40 0x7ff1fc147d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:22.070 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.065+0000 7ff2033f4640 1 --2- 172.21.15.157:0/1038317765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1fc0c5b70 0x7ff1fc1dfd30 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:22.070 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.065+0000 7ff2033f4640 1 --2- 172.21.15.157:0/1038317765 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1fc0be360 0x7ff1fc0ab0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:22.070 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.065+0000 7ff2033f4640 1 -- 172.21.15.157:0/1038317765 >> 172.21.15.157:0/1038317765 conn(0x7ff1fc0672c0 msgr2=0x7ff1fc14af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:22.071 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.065+0000 7ff2033f4640 1 -- 172.21.15.157:0/1038317765 shutdown_connections 2024-04-24T23:35:22.071 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.065+0000 7ff2033f4640 1 -- 172.21.15.157:0/1038317765 wait complete. 2024-04-24T23:35:22.076 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-04-24T23:35:21.773459+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235052,"kb_used_data":2240,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739668,"statfs":{"total":773094113280,"available":772853420032,"internally_reserved":0,"allocated":2293760,"data_stored":2015416,"data_compressed":65248,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003530"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.428046+0000","last_change":"2024-04-24T23:34:53.362409+0000","last_active":"2024-04-24T23:34:59.428046+0000","last_peered":"2024-04-24T23:34:59.428046+0000","last_clean":"2024-04-24T23:34:59.428046+0000","last_became_active":"2024-04-24T23:34:53.362104+0000","last_became_peered":"2024-04-24T23:34:53.362104+0000","last_unstale":"2024-04-24T23:34:59.428046+0000","last_undegraded":"2024-04-24T23:34:59.428046+0000","last_fullsized":"2024-04-24T23:34:59.428046+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:53.159219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396596+0000","last_change":"2024-04-24T23:34:54.373560+0000","last_active":"2024-04-24T23:34:59.396596+0000","last_peered":"2024-04-24T23:34:59.396596+0000","last_clean":"2024-04-24T23:34:59.396596+0000","last_became_active":"2024-04-24T23:34:54.373289+0000","last_became_peered":"2024-04-24T23:34:54.373289+0000","last_unstale":"2024-04-24T23:34:59.396596+0000","last_undegraded":"2024-04-24T23:34:59.396596+0000","last_fullsized":"2024-04-24T23:34:59.396596+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:31.447899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.395661+0000","last_change":"2024-04-24T23:34:58.730165+0000","last_active":"2024-04-24T23:34:59.395661+0000","last_peered":"2024-04-24T23:34:59.395661+0000","last_clean":"2024-04-24T23:34:59.395661+0000","last_became_active":"2024-04-24T23:34:58.729816+0000","last_became_peered":"2024-04-24T23:34:58.729816+0000","last_unstale":"2024-04-24T23:34:59.395661+0000","last_undegraded":"2024-04-24T23:34:59.395661+0000","last_fullsized":"2024-04-24T23:34:59.395661+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:02:02.335135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396776+0000","last_change":"2024-04-24T23:34:53.361650+0000","last_active":"2024-04-24T23:34:59.396776+0000","last_peered":"2024-04-24T23:34:59.396776+0000","last_clean":"2024-04-24T23:34:59.396776+0000","last_became_active":"2024-04-24T23:34:53.361366+0000","last_became_peered":"2024-04-24T23:34:53.361366+0000","last_unstale":"2024-04-24T23:34:59.396776+0000","last_undegraded":"2024-04-24T23:34:59.396776+0000","last_fullsized":"2024-04-24T23:34:59.396776+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:56:15.784223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.427859+0000","last_change":"2024-04-24T23:34:54.383602+0000","last_active":"2024-04-24T23:34:59.427859+0000","last_peered":"2024-04-24T23:34:59.427859+0000","last_clean":"2024-04-24T23:34:59.427859+0000","last_became_active":"2024-04-24T23:34:54.383160+0000","last_became_peered":"2024-04-24T23:34:54.383160+0000","last_unstale":"2024-04-24T23:34:59.427859+0000","last_undegraded":"2024-04-24T23:34:59.427859+0000","last_fullsized":"2024-04-24T23:34:59.427859+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:15.477061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396642+0000","last_change":"2024-04-24T23:34:58.404416+0000","last_active":"2024-04-24T23:34:59.396642+0000","last_peered":"2024-04-24T23:34:59.396642+0000","last_clean":"2024-04-24T23:34:59.396642+0000","last_became_active":"2024-04-24T23:34:58.403984+0000","last_became_peered":"2024-04-24T23:34:58.403984+0000","last_unstale":"2024-04-24T23:34:59.396642+0000","last_undegraded":"2024-04-24T23:34:59.396642+0000","last_fullsized":"2024-04-24T23:34:59.396642+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:46.237082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501886+0000","last_change":"2024-04-24T23:34:53.366730+0000","last_active":"2024-04-24T23:34:59.501886+0000","last_peered":"2024-04-24T23:34:59.501886+0000","last_clean":"2024-04-24T23:34:59.501886+0000","last_became_active":"2024-04-24T23:34:53.366430+0000","last_became_peered":"2024-04-24T23:34:53.366430+0000","last_unstale":"2024-04-24T23:34:59.501886+0000","last_undegraded":"2024-04-24T23:34:59.501886+0000","last_fullsized":"2024-04-24T23:34:59.501886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:41.422591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540654+0000","last_change":"2024-04-24T23:34:54.373592+0000","last_active":"2024-04-24T23:35:00.540654+0000","last_peered":"2024-04-24T23:35:00.540654+0000","last_clean":"2024-04-24T23:35:00.540654+0000","last_became_active":"2024-04-24T23:34:54.372844+0000","last_became_peered":"2024-04-24T23:34:54.372844+0000","last_unstale":"2024-04-24T23:35:00.540654+0000","last_undegraded":"2024-04-24T23:35:00.540654+0000","last_fullsized":"2024-04-24T23:35:00.540654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:49.372040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396779+0000","last_change":"2024-04-24T23:34:58.399224+0000","last_active":"2024-04-24T23:34:59.396779+0000","last_peered":"2024-04-24T23:34:59.396779+0000","last_clean":"2024-04-24T23:34:59.396779+0000","last_became_active":"2024-04-24T23:34:58.397817+0000","last_became_peered":"2024-04-24T23:34:58.397817+0000","last_unstale":"2024-04-24T23:34:59.396779+0000","last_undegraded":"2024-04-24T23:34:59.396779+0000","last_fullsized":"2024-04-24T23:34:59.396779+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:53.085366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501568+0000","last_change":"2024-04-24T23:34:53.362994+0000","last_active":"2024-04-24T23:34:59.501568+0000","last_peered":"2024-04-24T23:34:59.501568+0000","last_clean":"2024-04-24T23:34:59.501568+0000","last_became_active":"2024-04-24T23:34:53.362337+0000","last_became_peered":"2024-04-24T23:34:53.362337+0000","last_unstale":"2024-04-24T23:34:59.501568+0000","last_undegraded":"2024-04-24T23:34:59.501568+0000","last_fullsized":"2024-04-24T23:34:59.501568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:12.258211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.541217+0000","last_change":"2024-04-24T23:34:54.379218+0000","last_active":"2024-04-24T23:35:00.541217+0000","last_peered":"2024-04-24T23:35:00.541217+0000","last_clean":"2024-04-24T23:35:00.541217+0000","last_became_active":"2024-04-24T23:34:54.376019+0000","last_became_peered":"2024-04-24T23:34:54.376019+0000","last_unstale":"2024-04-24T23:35:00.541217+0000","last_undegraded":"2024-04-24T23:35:00.541217+0000","last_fullsized":"2024-04-24T23:35:00.541217+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:41:48.207542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396752+0000","last_change":"2024-04-24T23:34:58.397152+0000","last_active":"2024-04-24T23:34:59.396752+0000","last_peered":"2024-04-24T23:34:59.396752+0000","last_clean":"2024-04-24T23:34:59.396752+0000","last_became_active":"2024-04-24T23:34:58.396658+0000","last_became_peered":"2024-04-24T23:34:58.396658+0000","last_unstale":"2024-04-24T23:34:59.396752+0000","last_undegraded":"2024-04-24T23:34:59.396752+0000","last_fullsized":"2024-04-24T23:34:59.396752+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:41.360776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540212+0000","last_change":"2024-04-24T23:34:54.377859+0000","last_active":"2024-04-24T23:35:00.540212+0000","last_peered":"2024-04-24T23:35:00.540212+0000","last_clean":"2024-04-24T23:35:00.540212+0000","last_became_active":"2024-04-24T23:34:54.377576+0000","last_became_peered":"2024-04-24T23:34:54.377576+0000","last_unstale":"2024-04-24T23:35:00.540212+0000","last_undegraded":"2024-04-24T23:35:00.540212+0000","last_fullsized":"2024-04-24T23:35:00.540212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:47.446459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501625+0000","last_change":"2024-04-24T23:34:53.360903+0000","last_active":"2024-04-24T23:34:59.501625+0000","last_peered":"2024-04-24T23:34:59.501625+0000","last_clean":"2024-04-24T23:34:59.501625+0000","last_became_active":"2024-04-24T23:34:53.360635+0000","last_became_peered":"2024-04-24T23:34:53.360635+0000","last_unstale":"2024-04-24T23:34:59.501625+0000","last_undegraded":"2024-04-24T23:34:59.501625+0000","last_fullsized":"2024-04-24T23:34:59.501625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:42.205997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397132+0000","last_change":"2024-04-24T23:34:58.407146+0000","last_active":"2024-04-24T23:34:59.397132+0000","last_peered":"2024-04-24T23:34:59.397132+0000","last_clean":"2024-04-24T23:34:59.397132+0000","last_became_active":"2024-04-24T23:34:58.406844+0000","last_became_peered":"2024-04-24T23:34:58.406844+0000","last_unstale":"2024-04-24T23:34:59.397132+0000","last_undegraded":"2024-04-24T23:34:59.397132+0000","last_fullsized":"2024-04-24T23:34:59.397132+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:41:52.244767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396642+0000","last_change":"2024-04-24T23:34:54.366724+0000","last_active":"2024-04-24T23:34:59.396642+0000","last_peered":"2024-04-24T23:34:59.396642+0000","last_clean":"2024-04-24T23:34:59.396642+0000","last_became_active":"2024-04-24T23:34:54.366426+0000","last_became_peered":"2024-04-24T23:34:54.366426+0000","last_unstale":"2024-04-24T23:34:59.396642+0000","last_undegraded":"2024-04-24T23:34:59.396642+0000","last_fullsized":"2024-04-24T23:34:59.396642+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:47.035356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.881182+0000","last_change":"2024-04-24T23:34:53.359140+0000","last_active":"2024-04-24T23:34:59.881182+0000","last_peered":"2024-04-24T23:34:59.881182+0000","last_clean":"2024-04-24T23:34:59.881182+0000","last_became_active":"2024-04-24T23:34:53.358841+0000","last_became_peered":"2024-04-24T23:34:53.358841+0000","last_unstale":"2024-04-24T23:34:59.881182+0000","last_undegraded":"2024-04-24T23:34:59.881182+0000","last_fullsized":"2024-04-24T23:34:59.881182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:41.422591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397033+0000","last_change":"2024-04-24T23:34:58.400128+0000","last_active":"2024-04-24T23:34:59.397033+0000","last_peered":"2024-04-24T23:34:59.397033+0000","last_clean":"2024-04-24T23:34:59.397033+0000","last_became_active":"2024-04-24T23:34:58.399375+0000","last_became_peered":"2024-04-24T23:34:58.399375+0000","last_unstale":"2024-04-24T23:34:59.397033+0000","last_undegraded":"2024-04-24T23:34:59.397033+0000","last_fullsized":"2024-04-24T23:34:59.397033+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:53.409265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.395824+0000","last_change":"2024-04-24T23:34:54.365012+0000","last_active":"2024-04-24T23:34:59.395824+0000","last_peered":"2024-04-24T23:34:59.395824+0000","last_clean":"2024-04-24T23:34:59.395824+0000","last_became_active":"2024-04-24T23:34:54.364744+0000","last_became_peered":"2024-04-24T23:34:54.364744+0000","last_unstale":"2024-04-24T23:34:59.395824+0000","last_undegraded":"2024-04-24T23:34:59.395824+0000","last_fullsized":"2024-04-24T23:34:59.395824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:35.774122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.880884+0000","last_change":"2024-04-24T23:34:53.360843+0000","last_active":"2024-04-24T23:34:59.880884+0000","last_peered":"2024-04-24T23:34:59.880884+0000","last_clean":"2024-04-24T23:34:59.880884+0000","last_became_active":"2024-04-24T23:34:53.360570+0000","last_became_peered":"2024-04-24T23:34:53.360570+0000","last_unstale":"2024-04-24T23:34:59.880884+0000","last_undegraded":"2024-04-24T23:34:59.880884+0000","last_fullsized":"2024-04-24T23:34:59.880884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:21.089467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.881510+0000","last_change":"2024-04-24T23:34:58.731835+0000","last_active":"2024-04-24T23:34:59.881510+0000","last_peered":"2024-04-24T23:34:59.881510+0000","last_clean":"2024-04-24T23:34:59.881510+0000","last_became_active":"2024-04-24T23:34:58.731485+0000","last_became_peered":"2024-04-24T23:34:58.731485+0000","last_unstale":"2024-04-24T23:34:59.881510+0000","last_undegraded":"2024-04-24T23:34:59.881510+0000","last_fullsized":"2024-04-24T23:34:59.881510+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:56.609480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397598+0000","last_change":"2024-04-24T23:34:54.362447+0000","last_active":"2024-04-24T23:34:59.397598+0000","last_peered":"2024-04-24T23:34:59.397598+0000","last_clean":"2024-04-24T23:34:59.397598+0000","last_became_active":"2024-04-24T23:34:54.362141+0000","last_became_peered":"2024-04-24T23:34:54.362141+0000","last_unstale":"2024-04-24T23:34:59.397598+0000","last_undegraded":"2024-04-24T23:34:59.397598+0000","last_fullsized":"2024-04-24T23:34:59.397598+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:51.227333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501234+0000","last_change":"2024-04-24T23:34:53.366452+0000","last_active":"2024-04-24T23:34:59.501234+0000","last_peered":"2024-04-24T23:34:59.501234+0000","last_clean":"2024-04-24T23:34:59.501234+0000","last_became_active":"2024-04-24T23:34:53.366110+0000","last_became_peered":"2024-04-24T23:34:53.366110+0000","last_unstale":"2024-04-24T23:34:59.501234+0000","last_undegraded":"2024-04-24T23:34:59.501234+0000","last_fullsized":"2024-04-24T23:34:59.501234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:14.716741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396886+0000","last_change":"2024-04-24T23:34:58.402849+0000","last_active":"2024-04-24T23:34:59.396886+0000","last_peered":"2024-04-24T23:34:59.396886+0000","last_clean":"2024-04-24T23:34:59.396886+0000","last_became_active":"2024-04-24T23:34:58.402365+0000","last_became_peered":"2024-04-24T23:34:58.402365+0000","last_unstale":"2024-04-24T23:34:59.396886+0000","last_undegraded":"2024-04-24T23:34:59.396886+0000","last_fullsized":"2024-04-24T23:34:59.396886+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:07:32.822427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.428004+0000","last_change":"2024-04-24T23:34:54.370300+0000","last_active":"2024-04-24T23:34:59.428004+0000","last_peered":"2024-04-24T23:34:59.428004+0000","last_clean":"2024-04-24T23:34:59.428004+0000","last_became_active":"2024-04-24T23:34:54.369835+0000","last_became_peered":"2024-04-24T23:34:54.369835+0000","last_unstale":"2024-04-24T23:34:59.428004+0000","last_undegraded":"2024-04-24T23:34:59.428004+0000","last_fullsized":"2024-04-24T23:34:59.428004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:42.199856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.541551+0000","last_change":"2024-04-24T23:34:53.366110+0000","last_active":"2024-04-24T23:35:00.541551+0000","last_peered":"2024-04-24T23:35:00.541551+0000","last_clean":"2024-04-24T23:35:00.541551+0000","last_became_active":"2024-04-24T23:34:53.365783+0000","last_became_peered":"2024-04-24T23:34:53.365783+0000","last_unstale":"2024-04-24T23:35:00.541551+0000","last_undegraded":"2024-04-24T23:35:00.541551+0000","last_fullsized":"2024-04-24T23:35:00.541551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:41.422591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397333+0000","last_change":"2024-04-24T23:34:58.728191+0000","last_active":"2024-04-24T23:34:59.397333+0000","last_peered":"2024-04-24T23:34:59.397333+0000","last_clean":"2024-04-24T23:34:59.397333+0000","last_became_active":"2024-04-24T23:34:58.727653+0000","last_became_peered":"2024-04-24T23:34:58.727653+0000","last_unstale":"2024-04-24T23:34:59.397333+0000","last_undegraded":"2024-04-24T23:34:59.397333+0000","last_fullsized":"2024-04-24T23:34:59.397333+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:30.299011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397474+0000","last_change":"2024-04-24T23:34:54.368278+0000","last_active":"2024-04-24T23:34:59.397474+0000","last_peered":"2024-04-24T23:34:59.397474+0000","last_clean":"2024-04-24T23:34:59.397474+0000","last_became_active":"2024-04-24T23:34:54.367991+0000","last_became_peered":"2024-04-24T23:34:54.367991+0000","last_unstale":"2024-04-24T23:34:59.397474+0000","last_undegraded":"2024-04-24T23:34:59.397474+0000","last_fullsized":"2024-04-24T23:34:59.397474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:49:31.018719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.881400+0000","last_change":"2024-04-24T23:34:53.364189+0000","last_active":"2024-04-24T23:34:59.881400+0000","last_peered":"2024-04-24T23:34:59.881400+0000","last_clean":"2024-04-24T23:34:59.881400+0000","last_became_active":"2024-04-24T23:34:53.363842+0000","last_became_peered":"2024-04-24T23:34:53.363842+0000","last_unstale":"2024-04-24T23:34:59.881400+0000","last_undegraded":"2024-04-24T23:34:59.881400+0000","last_fullsized":"2024-04-24T23:34:59.881400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:12.258211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.395610+0000","last_change":"2024-04-24T23:34:58.729450+0000","last_active":"2024-04-24T23:34:59.395610+0000","last_peered":"2024-04-24T23:34:59.395610+0000","last_clean":"2024-04-24T23:34:59.395610+0000","last_became_active":"2024-04-24T23:34:58.729082+0000","last_became_peered":"2024-04-24T23:34:58.729082+0000","last_unstale":"2024-04-24T23:34:59.395610+0000","last_undegraded":"2024-04-24T23:34:59.395610+0000","last_fullsized":"2024-04-24T23:34:59.395610+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:45.372053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.881249+0000","last_change":"2024-04-24T23:34:54.357246+0000","last_active":"2024-04-24T23:34:59.881249+0000","last_peered":"2024-04-24T23:34:59.881249+0000","last_clean":"2024-04-24T23:34:59.881249+0000","last_became_active":"2024-04-24T23:34:54.356560+0000","last_became_peered":"2024-04-24T23:34:54.356560+0000","last_unstale":"2024-04-24T23:34:59.881249+0000","last_undegraded":"2024-04-24T23:34:59.881249+0000","last_fullsized":"2024-04-24T23:34:59.881249+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:42.199856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.502011+0000","last_change":"2024-04-24T23:34:53.366296+0000","last_active":"2024-04-24T23:34:59.502011+0000","last_peered":"2024-04-24T23:34:59.502011+0000","last_clean":"2024-04-24T23:34:59.502011+0000","last_became_active":"2024-04-24T23:34:53.365636+0000","last_became_peered":"2024-04-24T23:34:53.365636+0000","last_unstale":"2024-04-24T23:34:59.502011+0000","last_undegraded":"2024-04-24T23:34:59.502011+0000","last_fullsized":"2024-04-24T23:34:59.502011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:50.222015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.427745+0000","last_change":"2024-04-24T23:34:58.728271+0000","last_active":"2024-04-24T23:34:59.427745+0000","last_peered":"2024-04-24T23:34:59.427745+0000","last_clean":"2024-04-24T23:34:59.427745+0000","last_became_active":"2024-04-24T23:34:58.727885+0000","last_became_peered":"2024-04-24T23:34:58.727885+0000","last_unstale":"2024-04-24T23:34:59.427745+0000","last_undegraded":"2024-04-24T23:34:59.427745+0000","last_fullsized":"2024-04-24T23:34:59.427745+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:49.461271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.541247+0000","last_change":"2024-04-24T23:34:54.372245+0000","last_active":"2024-04-24T23:35:00.541247+0000","last_peered":"2024-04-24T23:35:00.541247+0000","last_clean":"2024-04-24T23:35:00.541247+0000","last_became_active":"2024-04-24T23:34:54.371947+0000","last_became_peered":"2024-04-24T23:34:54.371947+0000","last_unstale":"2024-04-24T23:35:00.541247+0000","last_undegraded":"2024-04-24T23:35:00.541247+0000","last_fullsized":"2024-04-24T23:35:00.541247+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:47:52.460508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.881345+0000","last_change":"2024-04-24T23:34:53.360352+0000","last_active":"2024-04-24T23:34:59.881345+0000","last_peered":"2024-04-24T23:34:59.881345+0000","last_clean":"2024-04-24T23:34:59.881345+0000","last_became_active":"2024-04-24T23:34:53.360082+0000","last_became_peered":"2024-04-24T23:34:53.360082+0000","last_unstale":"2024-04-24T23:34:59.881345+0000","last_undegraded":"2024-04-24T23:34:59.881345+0000","last_fullsized":"2024-04-24T23:34:59.881345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:50.222015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.881021+0000","last_change":"2024-04-24T23:34:58.404176+0000","last_active":"2024-04-24T23:34:59.881021+0000","last_peered":"2024-04-24T23:34:59.881021+0000","last_clean":"2024-04-24T23:34:59.881021+0000","last_became_active":"2024-04-24T23:34:58.403735+0000","last_became_peered":"2024-04-24T23:34:58.403735+0000","last_unstale":"2024-04-24T23:34:59.881021+0000","last_undegraded":"2024-04-24T23:34:59.881021+0000","last_fullsized":"2024-04-24T23:34:59.881021+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:49.461271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397652+0000","last_change":"2024-04-24T23:34:54.367638+0000","last_active":"2024-04-24T23:34:59.397652+0000","last_peered":"2024-04-24T23:34:59.397652+0000","last_clean":"2024-04-24T23:34:59.397652+0000","last_became_active":"2024-04-24T23:34:54.367355+0000","last_became_peered":"2024-04-24T23:34:54.367355+0000","last_unstale":"2024-04-24T23:34:59.397652+0000","last_undegraded":"2024-04-24T23:34:59.397652+0000","last_fullsized":"2024-04-24T23:34:59.397652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:15.722059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540545+0000","last_change":"2024-04-24T23:34:53.356111+0000","last_active":"2024-04-24T23:35:00.540545+0000","last_peered":"2024-04-24T23:35:00.540545+0000","last_clean":"2024-04-24T23:35:00.540545+0000","last_became_active":"2024-04-24T23:34:53.355710+0000","last_became_peered":"2024-04-24T23:34:53.355710+0000","last_unstale":"2024-04-24T23:35:00.540545+0000","last_undegraded":"2024-04-24T23:35:00.540545+0000","last_fullsized":"2024-04-24T23:35:00.540545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:21.089467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.881283+0000","last_change":"2024-04-24T23:34:58.403016+0000","last_active":"2024-04-24T23:34:59.881283+0000","last_peered":"2024-04-24T23:34:59.881283+0000","last_clean":"2024-04-24T23:34:59.881283+0000","last_became_active":"2024-04-24T23:34:58.402598+0000","last_became_peered":"2024-04-24T23:34:58.402598+0000","last_unstale":"2024-04-24T23:34:59.881283+0000","last_undegraded":"2024-04-24T23:34:59.881283+0000","last_fullsized":"2024-04-24T23:34:59.881283+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:47:56.497734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.880863+0000","last_change":"2024-04-24T23:34:54.356601+0000","last_active":"2024-04-24T23:34:59.880863+0000","last_peered":"2024-04-24T23:34:59.880863+0000","last_clean":"2024-04-24T23:34:59.880863+0000","last_became_active":"2024-04-24T23:34:54.356326+0000","last_became_peered":"2024-04-24T23:34:54.356326+0000","last_unstale":"2024-04-24T23:34:59.880863+0000","last_undegraded":"2024-04-24T23:34:59.880863+0000","last_fullsized":"2024-04-24T23:34:59.880863+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:47.446459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396563+0000","last_change":"2024-04-24T23:34:53.359238+0000","last_active":"2024-04-24T23:34:59.396563+0000","last_peered":"2024-04-24T23:34:59.396563+0000","last_clean":"2024-04-24T23:34:59.396563+0000","last_became_active":"2024-04-24T23:34:53.358937+0000","last_became_peered":"2024-04-24T23:34:53.358937+0000","last_unstale":"2024-04-24T23:34:59.396563+0000","last_undegraded":"2024-04-24T23:34:59.396563+0000","last_fullsized":"2024-04-24T23:34:59.396563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:50.222015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.881551+0000","last_change":"2024-04-24T23:34:58.403258+0000","last_active":"2024-04-24T23:34:59.881551+0000","last_peered":"2024-04-24T23:34:59.881551+0000","last_clean":"2024-04-24T23:34:59.881551+0000","last_became_active":"2024-04-24T23:34:58.402569+0000","last_became_peered":"2024-04-24T23:34:58.402569+0000","last_unstale":"2024-04-24T23:34:59.881551+0000","last_undegraded":"2024-04-24T23:34:59.881551+0000","last_fullsized":"2024-04-24T23:34:59.881551+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:32:08.782070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.427468+0000","last_change":"2024-04-24T23:34:54.377543+0000","last_active":"2024-04-24T23:34:59.427468+0000","last_peered":"2024-04-24T23:34:59.427468+0000","last_clean":"2024-04-24T23:34:59.427468+0000","last_became_active":"2024-04-24T23:34:54.377228+0000","last_became_peered":"2024-04-24T23:34:54.377228+0000","last_unstale":"2024-04-24T23:34:59.427468+0000","last_undegraded":"2024-04-24T23:34:59.427468+0000","last_fullsized":"2024-04-24T23:34:59.427468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:42.255038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501238+0000","last_change":"2024-04-24T23:34:53.362675+0000","last_active":"2024-04-24T23:34:59.501238+0000","last_peered":"2024-04-24T23:34:59.501238+0000","last_clean":"2024-04-24T23:34:59.501238+0000","last_became_active":"2024-04-24T23:34:53.362425+0000","last_became_peered":"2024-04-24T23:34:53.362425+0000","last_unstale":"2024-04-24T23:34:59.501238+0000","last_undegraded":"2024-04-24T23:34:59.501238+0000","last_fullsized":"2024-04-24T23:34:59.501238+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:21.089467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.427884+0000","last_change":"2024-04-24T23:34:58.728573+0000","last_active":"2024-04-24T23:34:59.427884+0000","last_peered":"2024-04-24T23:34:59.427884+0000","last_clean":"2024-04-24T23:34:59.427884+0000","last_became_active":"2024-04-24T23:34:58.727284+0000","last_became_peered":"2024-04-24T23:34:58.727284+0000","last_unstale":"2024-04-24T23:34:59.427884+0000","last_undegraded":"2024-04-24T23:34:59.427884+0000","last_fullsized":"2024-04-24T23:34:59.427884+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:32:08.782070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396198+0000","last_change":"2024-04-24T23:34:54.382820+0000","last_active":"2024-04-24T23:34:59.396198+0000","last_peered":"2024-04-24T23:34:59.396198+0000","last_clean":"2024-04-24T23:34:59.396198+0000","last_became_active":"2024-04-24T23:34:54.382564+0000","last_became_peered":"2024-04-24T23:34:54.382564+0000","last_unstale":"2024-04-24T23:34:59.396198+0000","last_undegraded":"2024-04-24T23:34:59.396198+0000","last_fullsized":"2024-04-24T23:34:59.396198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:45.622611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.541368+0000","last_change":"2024-04-24T23:34:53.357743+0000","last_active":"2024-04-24T23:35:00.541368+0000","last_peered":"2024-04-24T23:35:00.541368+0000","last_clean":"2024-04-24T23:35:00.541368+0000","last_became_active":"2024-04-24T23:34:53.357456+0000","last_became_peered":"2024-04-24T23:34:53.357456+0000","last_unstale":"2024-04-24T23:35:00.541368+0000","last_undegraded":"2024-04-24T23:35:00.541368+0000","last_fullsized":"2024-04-24T23:35:00.541368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:57:14.381644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":177,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501380+0000","last_change":"2024-04-24T23:34:49.778966+0000","last_active":"2024-04-24T23:34:59.501380+0000","last_peered":"2024-04-24T23:34:59.501380+0000","last_clean":"2024-04-24T23:34:59.501380+0000","last_became_active":"2024-04-24T23:34:49.778083+0000","last_became_peered":"2024-04-24T23:34:49.778083+0000","last_unstale":"2024-04-24T23:34:59.501380+0000","last_undegraded":"2024-04-24T23:34:59.501380+0000","last_fullsized":"2024-04-24T23:34:59.501380+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:48.587078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:48.587078+0000","last_clean_scrub_stamp":"2024-04-24T23:34:48.587078+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:46:59.471089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540517+0000","last_change":"2024-04-24T23:34:58.729779+0000","last_active":"2024-04-24T23:35:00.540517+0000","last_peered":"2024-04-24T23:35:00.540517+0000","last_clean":"2024-04-24T23:35:00.540517+0000","last_became_active":"2024-04-24T23:34:58.729370+0000","last_became_peered":"2024-04-24T23:34:58.729370+0000","last_unstale":"2024-04-24T23:35:00.540517+0000","last_undegraded":"2024-04-24T23:35:00.540517+0000","last_fullsized":"2024-04-24T23:35:00.540517+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:30.299011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501756+0000","last_change":"2024-04-24T23:34:54.384106+0000","last_active":"2024-04-24T23:34:59.501756+0000","last_peered":"2024-04-24T23:34:59.501756+0000","last_clean":"2024-04-24T23:34:59.501756+0000","last_became_active":"2024-04-24T23:34:54.383589+0000","last_became_peered":"2024-04-24T23:34:54.383589+0000","last_unstale":"2024-04-24T23:34:59.501756+0000","last_undegraded":"2024-04-24T23:34:59.501756+0000","last_fullsized":"2024-04-24T23:34:59.501756+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:41:48.207542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397450+0000","last_change":"2024-04-24T23:34:53.354077+0000","last_active":"2024-04-24T23:34:59.397450+0000","last_peered":"2024-04-24T23:34:59.397450+0000","last_clean":"2024-04-24T23:34:59.397450+0000","last_became_active":"2024-04-24T23:34:53.353804+0000","last_became_peered":"2024-04-24T23:34:53.353804+0000","last_unstale":"2024-04-24T23:34:59.397450+0000","last_undegraded":"2024-04-24T23:34:59.397450+0000","last_fullsized":"2024-04-24T23:34:59.397450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:41.422591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540103+0000","last_change":"2024-04-24T23:34:58.728722+0000","last_active":"2024-04-24T23:35:00.540103+0000","last_peered":"2024-04-24T23:35:00.540103+0000","last_clean":"2024-04-24T23:35:00.540103+0000","last_became_active":"2024-04-24T23:34:58.727649+0000","last_became_peered":"2024-04-24T23:34:58.727649+0000","last_unstale":"2024-04-24T23:35:00.540103+0000","last_undegraded":"2024-04-24T23:35:00.540103+0000","last_fullsized":"2024-04-24T23:35:00.540103+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:41:52.244767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540678+0000","last_change":"2024-04-24T23:34:54.373371+0000","last_active":"2024-04-24T23:35:00.540678+0000","last_peered":"2024-04-24T23:35:00.540678+0000","last_clean":"2024-04-24T23:35:00.540678+0000","last_became_active":"2024-04-24T23:34:54.372553+0000","last_became_peered":"2024-04-24T23:34:54.372553+0000","last_unstale":"2024-04-24T23:35:00.540678+0000","last_undegraded":"2024-04-24T23:35:00.540678+0000","last_fullsized":"2024-04-24T23:35:00.540678+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:26.261786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.881287+0000","last_change":"2024-04-24T23:34:53.353505+0000","last_active":"2024-04-24T23:34:59.881287+0000","last_peered":"2024-04-24T23:34:59.881287+0000","last_clean":"2024-04-24T23:34:59.881287+0000","last_became_active":"2024-04-24T23:34:53.353186+0000","last_became_peered":"2024-04-24T23:34:53.353186+0000","last_unstale":"2024-04-24T23:34:59.881287+0000","last_undegraded":"2024-04-24T23:34:59.881287+0000","last_fullsized":"2024-04-24T23:34:59.881287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:42.205997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.427805+0000","last_change":"2024-04-24T23:34:58.400988+0000","last_active":"2024-04-24T23:34:59.427805+0000","last_peered":"2024-04-24T23:34:59.427805+0000","last_clean":"2024-04-24T23:34:59.427805+0000","last_became_active":"2024-04-24T23:34:58.400549+0000","last_became_peered":"2024-04-24T23:34:58.400549+0000","last_unstale":"2024-04-24T23:34:59.427805+0000","last_undegraded":"2024-04-24T23:34:59.427805+0000","last_fullsized":"2024-04-24T23:34:59.427805+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:47:56.497734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.502060+0000","last_change":"2024-04-24T23:34:54.375234+0000","last_active":"2024-04-24T23:34:59.502060+0000","last_peered":"2024-04-24T23:34:59.502060+0000","last_clean":"2024-04-24T23:34:59.502060+0000","last_became_active":"2024-04-24T23:34:54.374963+0000","last_became_peered":"2024-04-24T23:34:54.374963+0000","last_unstale":"2024-04-24T23:34:59.502060+0000","last_undegraded":"2024-04-24T23:34:59.502060+0000","last_fullsized":"2024-04-24T23:34:59.502060+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:47:52.460508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396918+0000","last_change":"2024-04-24T23:34:53.360531+0000","last_active":"2024-04-24T23:34:59.396918+0000","last_peered":"2024-04-24T23:34:59.396918+0000","last_clean":"2024-04-24T23:34:59.396918+0000","last_became_active":"2024-04-24T23:34:53.360247+0000","last_became_peered":"2024-04-24T23:34:53.360247+0000","last_unstale":"2024-04-24T23:34:59.396918+0000","last_undegraded":"2024-04-24T23:34:59.396918+0000","last_fullsized":"2024-04-24T23:34:59.396918+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:57:14.381644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540673+0000","last_change":"2024-04-24T23:34:58.729798+0000","last_active":"2024-04-24T23:35:00.540673+0000","last_peered":"2024-04-24T23:35:00.540673+0000","last_clean":"2024-04-24T23:35:00.540673+0000","last_became_active":"2024-04-24T23:34:58.729345+0000","last_became_peered":"2024-04-24T23:34:58.729345+0000","last_unstale":"2024-04-24T23:35:00.540673+0000","last_undegraded":"2024-04-24T23:35:00.540673+0000","last_fullsized":"2024-04-24T23:35:00.540673+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:06.670518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501916+0000","last_change":"2024-04-24T23:34:54.377593+0000","last_active":"2024-04-24T23:34:59.501916+0000","last_peered":"2024-04-24T23:34:59.501916+0000","last_clean":"2024-04-24T23:34:59.501916+0000","last_became_active":"2024-04-24T23:34:54.377270+0000","last_became_peered":"2024-04-24T23:34:54.377270+0000","last_unstale":"2024-04-24T23:34:59.501916+0000","last_undegraded":"2024-04-24T23:34:59.501916+0000","last_fullsized":"2024-04-24T23:34:59.501916+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:26.261786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397587+0000","last_change":"2024-04-24T23:34:53.352981+0000","last_active":"2024-04-24T23:34:59.397587+0000","last_peered":"2024-04-24T23:34:59.397587+0000","last_clean":"2024-04-24T23:34:59.397587+0000","last_became_active":"2024-04-24T23:34:53.352669+0000","last_became_peered":"2024-04-24T23:34:53.352669+0000","last_unstale":"2024-04-24T23:34:59.397587+0000","last_undegraded":"2024-04-24T23:34:59.397587+0000","last_fullsized":"2024-04-24T23:34:59.397587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:09:49.550050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396650+0000","last_change":"2024-04-24T23:34:58.399234+0000","last_active":"2024-04-24T23:34:59.396650+0000","last_peered":"2024-04-24T23:34:59.396650+0000","last_clean":"2024-04-24T23:34:59.396650+0000","last_became_active":"2024-04-24T23:34:58.398824+0000","last_became_peered":"2024-04-24T23:34:58.398824+0000","last_unstale":"2024-04-24T23:34:59.396650+0000","last_undegraded":"2024-04-24T23:34:59.396650+0000","last_fullsized":"2024-04-24T23:34:59.396650+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:22.244493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501466+0000","last_change":"2024-04-24T23:34:54.376800+0000","last_active":"2024-04-24T23:34:59.501466+0000","last_peered":"2024-04-24T23:34:59.501466+0000","last_clean":"2024-04-24T23:34:59.501466+0000","last_became_active":"2024-04-24T23:34:54.376512+0000","last_became_peered":"2024-04-24T23:34:54.376512+0000","last_unstale":"2024-04-24T23:34:59.501466+0000","last_undegraded":"2024-04-24T23:34:59.501466+0000","last_fullsized":"2024-04-24T23:34:59.501466+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:49.048141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396460+0000","last_change":"2024-04-24T23:34:53.355307+0000","last_active":"2024-04-24T23:34:59.396460+0000","last_peered":"2024-04-24T23:34:59.396460+0000","last_clean":"2024-04-24T23:34:59.396460+0000","last_became_active":"2024-04-24T23:34:53.354988+0000","last_became_peered":"2024-04-24T23:34:53.354988+0000","last_unstale":"2024-04-24T23:34:59.396460+0000","last_undegraded":"2024-04-24T23:34:59.396460+0000","last_fullsized":"2024-04-24T23:34:59.396460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:39.457116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.427781+0000","last_change":"2024-04-24T23:34:58.399188+0000","last_active":"2024-04-24T23:34:59.427781+0000","last_peered":"2024-04-24T23:34:59.427781+0000","last_clean":"2024-04-24T23:34:59.427781+0000","last_became_active":"2024-04-24T23:34:58.398457+0000","last_became_peered":"2024-04-24T23:34:58.398457+0000","last_unstale":"2024-04-24T23:34:59.427781+0000","last_undegraded":"2024-04-24T23:34:59.427781+0000","last_fullsized":"2024-04-24T23:34:59.427781+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:50:21.658540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540120+0000","last_change":"2024-04-24T23:34:54.372150+0000","last_active":"2024-04-24T23:35:00.540120+0000","last_peered":"2024-04-24T23:35:00.540120+0000","last_clean":"2024-04-24T23:35:00.540120+0000","last_became_active":"2024-04-24T23:34:54.371703+0000","last_became_peered":"2024-04-24T23:34:54.371703+0000","last_unstale":"2024-04-24T23:35:00.540120+0000","last_undegraded":"2024-04-24T23:35:00.540120+0000","last_fullsized":"2024-04-24T23:35:00.540120+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:15.477061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540199+0000","last_change":"2024-04-24T23:34:53.363787+0000","last_active":"2024-04-24T23:35:00.540199+0000","last_peered":"2024-04-24T23:35:00.540199+0000","last_clean":"2024-04-24T23:35:00.540199+0000","last_became_active":"2024-04-24T23:34:53.363515+0000","last_became_peered":"2024-04-24T23:34:53.363515+0000","last_unstale":"2024-04-24T23:35:00.540199+0000","last_undegraded":"2024-04-24T23:35:00.540199+0000","last_fullsized":"2024-04-24T23:35:00.540199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:50.222015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397429+0000","last_change":"2024-04-24T23:34:58.728420+0000","last_active":"2024-04-24T23:34:59.397429+0000","last_peered":"2024-04-24T23:34:59.397429+0000","last_clean":"2024-04-24T23:34:59.397429+0000","last_became_active":"2024-04-24T23:34:58.727425+0000","last_became_peered":"2024-04-24T23:34:58.727425+0000","last_unstale":"2024-04-24T23:34:59.397429+0000","last_undegraded":"2024-04-24T23:34:59.397429+0000","last_fullsized":"2024-04-24T23:34:59.397429+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:19:14.487689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501575+0000","last_change":"2024-04-24T23:34:54.379296+0000","last_active":"2024-04-24T23:34:59.501575+0000","last_peered":"2024-04-24T23:34:59.501575+0000","last_clean":"2024-04-24T23:34:59.501575+0000","last_became_active":"2024-04-24T23:34:54.379016+0000","last_became_peered":"2024-04-24T23:34:54.379016+0000","last_unstale":"2024-04-24T23:34:59.501575+0000","last_undegraded":"2024-04-24T23:34:59.501575+0000","last_fullsized":"2024-04-24T23:34:59.501575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:37.323551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540560+0000","last_change":"2024-04-24T23:34:53.360210+0000","last_active":"2024-04-24T23:35:00.540560+0000","last_peered":"2024-04-24T23:35:00.540560+0000","last_clean":"2024-04-24T23:35:00.540560+0000","last_became_active":"2024-04-24T23:34:53.359581+0000","last_became_peered":"2024-04-24T23:34:53.359581+0000","last_unstale":"2024-04-24T23:35:00.540560+0000","last_undegraded":"2024-04-24T23:35:00.540560+0000","last_fullsized":"2024-04-24T23:35:00.540560+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:14.716741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397168+0000","last_change":"2024-04-24T23:34:58.728491+0000","last_active":"2024-04-24T23:34:59.397168+0000","last_peered":"2024-04-24T23:34:59.397168+0000","last_clean":"2024-04-24T23:34:59.397168+0000","last_became_active":"2024-04-24T23:34:58.727686+0000","last_became_peered":"2024-04-24T23:34:58.727686+0000","last_unstale":"2024-04-24T23:34:59.397168+0000","last_undegraded":"2024-04-24T23:34:59.397168+0000","last_fullsized":"2024-04-24T23:34:59.397168+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:46.282712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.427650+0000","last_change":"2024-04-24T23:34:54.372009+0000","last_active":"2024-04-24T23:34:59.427650+0000","last_peered":"2024-04-24T23:34:59.427650+0000","last_clean":"2024-04-24T23:34:59.427650+0000","last_became_active":"2024-04-24T23:34:54.371720+0000","last_became_peered":"2024-04-24T23:34:54.371720+0000","last_unstale":"2024-04-24T23:34:59.427650+0000","last_undegraded":"2024-04-24T23:34:59.427650+0000","last_fullsized":"2024-04-24T23:34:59.427650+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:04:07.487367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.428005+0000","last_change":"2024-04-24T23:34:53.353738+0000","last_active":"2024-04-24T23:34:59.428005+0000","last_peered":"2024-04-24T23:34:59.428005+0000","last_clean":"2024-04-24T23:34:59.428005+0000","last_became_active":"2024-04-24T23:34:53.353452+0000","last_became_peered":"2024-04-24T23:34:53.353452+0000","last_unstale":"2024-04-24T23:34:59.428005+0000","last_undegraded":"2024-04-24T23:34:59.428005+0000","last_fullsized":"2024-04-24T23:34:59.428005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:33:43.693095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.541104+0000","last_change":"2024-04-24T23:34:58.728503+0000","last_active":"2024-04-24T23:35:00.541104+0000","last_peered":"2024-04-24T23:35:00.541104+0000","last_clean":"2024-04-24T23:35:00.541104+0000","last_became_active":"2024-04-24T23:34:58.727921+0000","last_became_peered":"2024-04-24T23:34:58.727921+0000","last_unstale":"2024-04-24T23:35:00.541104+0000","last_undegraded":"2024-04-24T23:35:00.541104+0000","last_fullsized":"2024-04-24T23:35:00.541104+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:19:40.928966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501636+0000","last_change":"2024-04-24T23:34:54.368532+0000","last_active":"2024-04-24T23:34:59.501636+0000","last_peered":"2024-04-24T23:34:59.501636+0000","last_clean":"2024-04-24T23:34:59.501636+0000","last_became_active":"2024-04-24T23:34:54.367903+0000","last_became_peered":"2024-04-24T23:34:54.367903+0000","last_unstale":"2024-04-24T23:34:59.501636+0000","last_undegraded":"2024-04-24T23:34:59.501636+0000","last_fullsized":"2024-04-24T23:34:59.501636+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:49.372040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:03.674709+0000","last_change":"2024-04-24T23:34:53.362758+0000","last_active":"2024-04-24T23:35:03.674709+0000","last_peered":"2024-04-24T23:35:03.674709+0000","last_clean":"2024-04-24T23:35:03.674709+0000","last_became_active":"2024-04-24T23:34:53.362230+0000","last_became_peered":"2024-04-24T23:34:53.362230+0000","last_unstale":"2024-04-24T23:35:03.674709+0000","last_undegraded":"2024-04-24T23:35:03.674709+0000","last_fullsized":"2024-04-24T23:35:03.674709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:14.716741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.881050+0000","last_change":"2024-04-24T23:34:58.400881+0000","last_active":"2024-04-24T23:34:59.881050+0000","last_peered":"2024-04-24T23:34:59.881050+0000","last_clean":"2024-04-24T23:34:59.881050+0000","last_became_active":"2024-04-24T23:34:58.400429+0000","last_became_peered":"2024-04-24T23:34:58.400429+0000","last_unstale":"2024-04-24T23:34:59.881050+0000","last_undegraded":"2024-04-24T23:34:59.881050+0000","last_fullsized":"2024-04-24T23:34:59.881050+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:50:21.658540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397180+0000","last_change":"2024-04-24T23:34:54.373603+0000","last_active":"2024-04-24T23:34:59.397180+0000","last_peered":"2024-04-24T23:34:59.397180+0000","last_clean":"2024-04-24T23:34:59.397180+0000","last_became_active":"2024-04-24T23:34:54.373347+0000","last_became_peered":"2024-04-24T23:34:54.373347+0000","last_unstale":"2024-04-24T23:34:59.397180+0000","last_undegraded":"2024-04-24T23:34:59.397180+0000","last_fullsized":"2024-04-24T23:34:59.397180+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:13.263529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.881250+0000","last_change":"2024-04-24T23:34:53.354673+0000","last_active":"2024-04-24T23:34:59.881250+0000","last_peered":"2024-04-24T23:34:59.881250+0000","last_clean":"2024-04-24T23:34:59.881250+0000","last_became_active":"2024-04-24T23:34:53.354394+0000","last_became_peered":"2024-04-24T23:34:53.354394+0000","last_unstale":"2024-04-24T23:34:59.881250+0000","last_undegraded":"2024-04-24T23:34:59.881250+0000","last_fullsized":"2024-04-24T23:34:59.881250+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:14.716741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397330+0000","last_change":"2024-04-24T23:34:58.397739+0000","last_active":"2024-04-24T23:34:59.397330+0000","last_peered":"2024-04-24T23:34:59.397330+0000","last_clean":"2024-04-24T23:34:59.397330+0000","last_became_active":"2024-04-24T23:34:58.397353+0000","last_became_peered":"2024-04-24T23:34:58.397353+0000","last_unstale":"2024-04-24T23:34:59.397330+0000","last_undegraded":"2024-04-24T23:34:59.397330+0000","last_fullsized":"2024-04-24T23:34:59.397330+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:55.672101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.427967+0000","last_change":"2024-04-24T23:34:54.383348+0000","last_active":"2024-04-24T23:34:59.427967+0000","last_peered":"2024-04-24T23:34:59.427967+0000","last_clean":"2024-04-24T23:34:59.427967+0000","last_became_active":"2024-04-24T23:34:54.383082+0000","last_became_peered":"2024-04-24T23:34:54.383082+0000","last_unstale":"2024-04-24T23:34:59.427967+0000","last_undegraded":"2024-04-24T23:34:59.427967+0000","last_fullsized":"2024-04-24T23:34:59.427967+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:56:17.763159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540591+0000","last_change":"2024-04-24T23:34:53.364540+0000","last_active":"2024-04-24T23:35:00.540591+0000","last_peered":"2024-04-24T23:35:00.540591+0000","last_clean":"2024-04-24T23:35:00.540591+0000","last_became_active":"2024-04-24T23:34:53.364259+0000","last_became_peered":"2024-04-24T23:34:53.364259+0000","last_unstale":"2024-04-24T23:35:00.540591+0000","last_undegraded":"2024-04-24T23:35:00.540591+0000","last_fullsized":"2024-04-24T23:35:00.540591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:41.422591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.395771+0000","last_change":"2024-04-24T23:34:58.405865+0000","last_active":"2024-04-24T23:34:59.395771+0000","last_peered":"2024-04-24T23:34:59.395771+0000","last_clean":"2024-04-24T23:34:59.395771+0000","last_became_active":"2024-04-24T23:34:58.405032+0000","last_became_peered":"2024-04-24T23:34:58.405032+0000","last_unstale":"2024-04-24T23:34:59.395771+0000","last_undegraded":"2024-04-24T23:34:59.395771+0000","last_fullsized":"2024-04-24T23:34:59.395771+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:54.012389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540495+0000","last_change":"2024-04-24T23:34:54.383631+0000","last_active":"2024-04-24T23:35:00.540495+0000","last_peered":"2024-04-24T23:35:00.540495+0000","last_clean":"2024-04-24T23:35:00.540495+0000","last_became_active":"2024-04-24T23:34:54.383354+0000","last_became_peered":"2024-04-24T23:34:54.383354+0000","last_unstale":"2024-04-24T23:35:00.540495+0000","last_undegraded":"2024-04-24T23:35:00.540495+0000","last_fullsized":"2024-04-24T23:35:00.540495+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:50:17.621315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.541421+0000","last_change":"2024-04-24T23:34:53.362409+0000","last_active":"2024-04-24T23:35:00.541421+0000","last_peered":"2024-04-24T23:35:00.541421+0000","last_clean":"2024-04-24T23:35:00.541421+0000","last_became_active":"2024-04-24T23:34:53.362103+0000","last_became_peered":"2024-04-24T23:34:53.362103+0000","last_unstale":"2024-04-24T23:35:00.541421+0000","last_undegraded":"2024-04-24T23:35:00.541421+0000","last_fullsized":"2024-04-24T23:35:00.541421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:57:14.381644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.395785+0000","last_change":"2024-04-24T23:34:58.730088+0000","last_active":"2024-04-24T23:34:59.395785+0000","last_peered":"2024-04-24T23:34:59.395785+0000","last_clean":"2024-04-24T23:34:59.395785+0000","last_became_active":"2024-04-24T23:34:58.729217+0000","last_became_peered":"2024-04-24T23:34:58.729217+0000","last_unstale":"2024-04-24T23:34:59.395785+0000","last_undegraded":"2024-04-24T23:34:59.395785+0000","last_fullsized":"2024-04-24T23:34:59.395785+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:12:14.243770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.427823+0000","last_change":"2024-04-24T23:34:58.728892+0000","last_active":"2024-04-24T23:34:59.427823+0000","last_peered":"2024-04-24T23:34:59.427823+0000","last_clean":"2024-04-24T23:34:59.427823+0000","last_became_active":"2024-04-24T23:34:58.727914+0000","last_became_peered":"2024-04-24T23:34:58.727914+0000","last_unstale":"2024-04-24T23:34:59.427823+0000","last_undegraded":"2024-04-24T23:34:59.427823+0000","last_fullsized":"2024-04-24T23:34:59.427823+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:56.609480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.880773+0000","last_change":"2024-04-24T23:34:54.363973+0000","last_active":"2024-04-24T23:34:59.880773+0000","last_peered":"2024-04-24T23:34:59.880773+0000","last_clean":"2024-04-24T23:34:59.880773+0000","last_became_active":"2024-04-24T23:34:54.363724+0000","last_became_peered":"2024-04-24T23:34:54.363724+0000","last_unstale":"2024-04-24T23:34:59.880773+0000","last_undegraded":"2024-04-24T23:34:59.880773+0000","last_fullsized":"2024-04-24T23:34:59.880773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:03:24.745895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.541110+0000","last_change":"2024-04-24T23:34:53.360041+0000","last_active":"2024-04-24T23:35:00.541110+0000","last_peered":"2024-04-24T23:35:00.541110+0000","last_clean":"2024-04-24T23:35:00.541110+0000","last_became_active":"2024-04-24T23:34:53.359707+0000","last_became_peered":"2024-04-24T23:34:53.359707+0000","last_unstale":"2024-04-24T23:35:00.541110+0000","last_undegraded":"2024-04-24T23:35:00.541110+0000","last_fullsized":"2024-04-24T23:35:00.541110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:50.222015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.541453+0000","last_change":"2024-04-24T23:34:58.399618+0000","last_active":"2024-04-24T23:35:00.541453+0000","last_peered":"2024-04-24T23:35:00.541453+0000","last_clean":"2024-04-24T23:35:00.541453+0000","last_became_active":"2024-04-24T23:34:58.399182+0000","last_became_peered":"2024-04-24T23:34:58.399182+0000","last_unstale":"2024-04-24T23:35:00.541453+0000","last_undegraded":"2024-04-24T23:35:00.541453+0000","last_fullsized":"2024-04-24T23:35:00.541453+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:55.672101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.395731+0000","last_change":"2024-04-24T23:34:54.369729+0000","last_active":"2024-04-24T23:34:59.395731+0000","last_peered":"2024-04-24T23:34:59.395731+0000","last_clean":"2024-04-24T23:34:59.395731+0000","last_became_active":"2024-04-24T23:34:54.369155+0000","last_became_peered":"2024-04-24T23:34:54.369155+0000","last_unstale":"2024-04-24T23:34:59.395731+0000","last_undegraded":"2024-04-24T23:34:59.395731+0000","last_fullsized":"2024-04-24T23:34:59.395731+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:46:28.181996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.395890+0000","last_change":"2024-04-24T23:34:53.365328+0000","last_active":"2024-04-24T23:34:59.395890+0000","last_peered":"2024-04-24T23:34:59.395890+0000","last_clean":"2024-04-24T23:34:59.395890+0000","last_became_active":"2024-04-24T23:34:53.365029+0000","last_became_peered":"2024-04-24T23:34:53.365029+0000","last_unstale":"2024-04-24T23:34:59.395890+0000","last_undegraded":"2024-04-24T23:34:59.395890+0000","last_fullsized":"2024-04-24T23:34:59.395890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:48:30.961870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540306+0000","last_change":"2024-04-24T23:34:53.365123+0000","last_active":"2024-04-24T23:35:00.540306+0000","last_peered":"2024-04-24T23:35:00.540306+0000","last_clean":"2024-04-24T23:35:00.540306+0000","last_became_active":"2024-04-24T23:34:53.364821+0000","last_became_peered":"2024-04-24T23:34:53.364821+0000","last_unstale":"2024-04-24T23:35:00.540306+0000","last_undegraded":"2024-04-24T23:35:00.540306+0000","last_fullsized":"2024-04-24T23:35:00.540306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:39.457116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.427901+0000","last_change":"2024-04-24T23:34:54.376037+0000","last_active":"2024-04-24T23:34:59.427901+0000","last_peered":"2024-04-24T23:34:59.427901+0000","last_clean":"2024-04-24T23:34:59.427901+0000","last_became_active":"2024-04-24T23:34:54.375689+0000","last_became_peered":"2024-04-24T23:34:54.375689+0000","last_unstale":"2024-04-24T23:34:59.427901+0000","last_undegraded":"2024-04-24T23:34:59.427901+0000","last_fullsized":"2024-04-24T23:34:59.427901+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:47.446459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397192+0000","last_change":"2024-04-24T23:34:58.404439+0000","last_active":"2024-04-24T23:34:59.397192+0000","last_peered":"2024-04-24T23:34:59.397192+0000","last_clean":"2024-04-24T23:34:59.397192+0000","last_became_active":"2024-04-24T23:34:58.404053+0000","last_became_peered":"2024-04-24T23:34:58.404053+0000","last_unstale":"2024-04-24T23:34:59.397192+0000","last_undegraded":"2024-04-24T23:34:59.397192+0000","last_fullsized":"2024-04-24T23:34:59.397192+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:46.292263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396770+0000","last_change":"2024-04-24T23:34:53.363066+0000","last_active":"2024-04-24T23:34:59.396770+0000","last_peered":"2024-04-24T23:34:59.396770+0000","last_clean":"2024-04-24T23:34:59.396770+0000","last_became_active":"2024-04-24T23:34:53.362548+0000","last_became_peered":"2024-04-24T23:34:53.362548+0000","last_unstale":"2024-04-24T23:34:59.396770+0000","last_undegraded":"2024-04-24T23:34:59.396770+0000","last_fullsized":"2024-04-24T23:34:59.396770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:21:13.338964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397349+0000","last_change":"2024-04-24T23:34:54.365704+0000","last_active":"2024-04-24T23:34:59.397349+0000","last_peered":"2024-04-24T23:34:59.397349+0000","last_clean":"2024-04-24T23:34:59.397349+0000","last_became_active":"2024-04-24T23:34:54.364814+0000","last_became_peered":"2024-04-24T23:34:54.364814+0000","last_unstale":"2024-04-24T23:34:59.397349+0000","last_undegraded":"2024-04-24T23:34:59.397349+0000","last_fullsized":"2024-04-24T23:34:59.397349+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:22.094785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397247+0000","last_change":"2024-04-24T23:34:58.407186+0000","last_active":"2024-04-24T23:34:59.397247+0000","last_peered":"2024-04-24T23:34:59.397247+0000","last_clean":"2024-04-24T23:34:59.397247+0000","last_became_active":"2024-04-24T23:34:58.406811+0000","last_became_peered":"2024-04-24T23:34:58.406811+0000","last_unstale":"2024-04-24T23:34:59.397247+0000","last_undegraded":"2024-04-24T23:34:59.397247+0000","last_fullsized":"2024-04-24T23:34:59.397247+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:55.861555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.539983+0000","last_change":"2024-04-24T23:34:54.375142+0000","last_active":"2024-04-24T23:35:00.539983+0000","last_peered":"2024-04-24T23:35:00.539983+0000","last_clean":"2024-04-24T23:35:00.539983+0000","last_became_active":"2024-04-24T23:34:54.374844+0000","last_became_peered":"2024-04-24T23:34:54.374844+0000","last_unstale":"2024-04-24T23:35:00.539983+0000","last_undegraded":"2024-04-24T23:35:00.539983+0000","last_fullsized":"2024-04-24T23:35:00.539983+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:29:07.593498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396456+0000","last_change":"2024-04-24T23:34:54.383574+0000","last_active":"2024-04-24T23:34:59.396456+0000","last_peered":"2024-04-24T23:34:59.396456+0000","last_clean":"2024-04-24T23:34:59.396456+0000","last_became_active":"2024-04-24T23:34:54.383253+0000","last_became_peered":"2024-04-24T23:34:54.383253+0000","last_unstale":"2024-04-24T23:34:59.396456+0000","last_undegraded":"2024-04-24T23:34:59.396456+0000","last_fullsized":"2024-04-24T23:34:59.396456+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:43.211314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540447+0000","last_change":"2024-04-24T23:34:54.372208+0000","last_active":"2024-04-24T23:35:00.540447+0000","last_peered":"2024-04-24T23:35:00.540447+0000","last_clean":"2024-04-24T23:35:00.540447+0000","last_became_active":"2024-04-24T23:34:54.371910+0000","last_became_peered":"2024-04-24T23:34:54.371910+0000","last_unstale":"2024-04-24T23:35:00.540447+0000","last_undegraded":"2024-04-24T23:35:00.540447+0000","last_fullsized":"2024-04-24T23:35:00.540447+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:07:28.785202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.427860+0000","last_change":"2024-04-24T23:34:54.366271+0000","last_active":"2024-04-24T23:34:59.427860+0000","last_peered":"2024-04-24T23:34:59.427860+0000","last_clean":"2024-04-24T23:34:59.427860+0000","last_became_active":"2024-04-24T23:34:54.366005+0000","last_became_peered":"2024-04-24T23:34:54.366005+0000","last_unstale":"2024-04-24T23:34:59.427860+0000","last_undegraded":"2024-04-24T23:34:59.427860+0000","last_fullsized":"2024-04-24T23:34:59.427860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:03:24.745895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397136+0000","last_change":"2024-04-24T23:34:54.360529+0000","last_active":"2024-04-24T23:34:59.397136+0000","last_peered":"2024-04-24T23:34:59.397136+0000","last_clean":"2024-04-24T23:34:59.397136+0000","last_became_active":"2024-04-24T23:34:54.360266+0000","last_became_peered":"2024-04-24T23:34:54.360266+0000","last_unstale":"2024-04-24T23:34:59.397136+0000","last_undegraded":"2024-04-24T23:34:59.397136+0000","last_fullsized":"2024-04-24T23:34:59.397136+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:49:47.246193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.881089+0000","last_change":"2024-04-24T23:34:54.365744+0000","last_active":"2024-04-24T23:34:59.881089+0000","last_peered":"2024-04-24T23:34:59.881089+0000","last_clean":"2024-04-24T23:34:59.881089+0000","last_became_active":"2024-04-24T23:34:54.365493+0000","last_became_peered":"2024-04-24T23:34:54.365493+0000","last_unstale":"2024-04-24T23:34:59.881089+0000","last_undegraded":"2024-04-24T23:34:59.881089+0000","last_fullsized":"2024-04-24T23:34:59.881089+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:15.477061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.541221+0000","last_change":"2024-04-24T23:34:54.379642+0000","last_active":"2024-04-24T23:35:00.541221+0000","last_peered":"2024-04-24T23:35:00.541221+0000","last_clean":"2024-04-24T23:35:00.541221+0000","last_became_active":"2024-04-24T23:34:54.378568+0000","last_became_peered":"2024-04-24T23:34:54.378568+0000","last_unstale":"2024-04-24T23:35:00.541221+0000","last_undegraded":"2024-04-24T23:35:00.541221+0000","last_fullsized":"2024-04-24T23:35:00.541221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:42.199856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397390+0000","last_change":"2024-04-24T23:34:54.363700+0000","last_active":"2024-04-24T23:34:59.397390+0000","last_peered":"2024-04-24T23:34:59.397390+0000","last_clean":"2024-04-24T23:34:59.397390+0000","last_became_active":"2024-04-24T23:34:54.363439+0000","last_became_peered":"2024-04-24T23:34:54.363439+0000","last_unstale":"2024-04-24T23:34:59.397390+0000","last_undegraded":"2024-04-24T23:34:59.397390+0000","last_fullsized":"2024-04-24T23:34:59.397390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:28:56.048178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.541000+0000","last_change":"2024-04-24T23:34:54.375460+0000","last_active":"2024-04-24T23:35:00.541000+0000","last_peered":"2024-04-24T23:35:00.541000+0000","last_clean":"2024-04-24T23:35:00.541000+0000","last_became_active":"2024-04-24T23:34:54.375123+0000","last_became_peered":"2024-04-24T23:34:54.375123+0000","last_unstale":"2024-04-24T23:35:00.541000+0000","last_undegraded":"2024-04-24T23:35:00.541000+0000","last_fullsized":"2024-04-24T23:35:00.541000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:18.207267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396028+0000","last_change":"2024-04-24T23:34:54.369509+0000","last_active":"2024-04-24T23:34:59.396028+0000","last_peered":"2024-04-24T23:34:59.396028+0000","last_clean":"2024-04-24T23:34:59.396028+0000","last_became_active":"2024-04-24T23:34:54.369219+0000","last_became_peered":"2024-04-24T23:34:54.369219+0000","last_unstale":"2024-04-24T23:34:59.396028+0000","last_undegraded":"2024-04-24T23:34:59.396028+0000","last_fullsized":"2024-04-24T23:34:59.396028+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:09:37.649503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540426+0000","last_change":"2024-04-24T23:34:54.384240+0000","last_active":"2024-04-24T23:35:00.540426+0000","last_peered":"2024-04-24T23:35:00.540426+0000","last_clean":"2024-04-24T23:35:00.540426+0000","last_became_active":"2024-04-24T23:34:54.383780+0000","last_became_peered":"2024-04-24T23:34:54.383780+0000","last_unstale":"2024-04-24T23:35:00.540426+0000","last_undegraded":"2024-04-24T23:35:00.540426+0000","last_fullsized":"2024-04-24T23:35:00.540426+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:49.048141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.427635+0000","last_change":"2024-04-24T23:34:54.371052+0000","last_active":"2024-04-24T23:34:59.427635+0000","last_peered":"2024-04-24T23:34:59.427635+0000","last_clean":"2024-04-24T23:34:59.427635+0000","last_became_active":"2024-04-24T23:34:54.370793+0000","last_became_peered":"2024-04-24T23:34:54.370793+0000","last_unstale":"2024-04-24T23:34:59.427635+0000","last_undegraded":"2024-04-24T23:34:59.427635+0000","last_fullsized":"2024-04-24T23:34:59.427635+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:29:07.593498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540693+0000","last_change":"2024-04-24T23:34:54.377190+0000","last_active":"2024-04-24T23:35:00.540693+0000","last_peered":"2024-04-24T23:35:00.540693+0000","last_clean":"2024-04-24T23:35:00.540693+0000","last_became_active":"2024-04-24T23:34:54.376940+0000","last_became_peered":"2024-04-24T23:34:54.376940+0000","last_unstale":"2024-04-24T23:35:00.540693+0000","last_undegraded":"2024-04-24T23:35:00.540693+0000","last_fullsized":"2024-04-24T23:35:00.540693+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:03:24.745895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501917+0000","last_change":"2024-04-24T23:34:54.368770+0000","last_active":"2024-04-24T23:34:59.501917+0000","last_peered":"2024-04-24T23:34:59.501917+0000","last_clean":"2024-04-24T23:34:59.501917+0000","last_became_active":"2024-04-24T23:34:54.368228+0000","last_became_peered":"2024-04-24T23:34:54.368228+0000","last_unstale":"2024-04-24T23:34:59.501917+0000","last_undegraded":"2024-04-24T23:34:59.501917+0000","last_fullsized":"2024-04-24T23:34:59.501917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:51.634876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396819+0000","last_change":"2024-04-24T23:34:54.373038+0000","last_active":"2024-04-24T23:34:59.396819+0000","last_peered":"2024-04-24T23:34:59.396819+0000","last_clean":"2024-04-24T23:34:59.396819+0000","last_became_active":"2024-04-24T23:34:54.372766+0000","last_became_peered":"2024-04-24T23:34:54.372766+0000","last_unstale":"2024-04-24T23:34:59.396819+0000","last_undegraded":"2024-04-24T23:34:59.396819+0000","last_fullsized":"2024-04-24T23:34:59.396819+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:13.263529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.541061+0000","last_change":"2024-04-24T23:34:54.383733+0000","last_active":"2024-04-24T23:35:00.541061+0000","last_peered":"2024-04-24T23:35:00.541061+0000","last_clean":"2024-04-24T23:35:00.541061+0000","last_became_active":"2024-04-24T23:34:54.383463+0000","last_became_peered":"2024-04-24T23:34:54.383463+0000","last_unstale":"2024-04-24T23:35:00.541061+0000","last_undegraded":"2024-04-24T23:35:00.541061+0000","last_fullsized":"2024-04-24T23:35:00.541061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:51.634876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540883+0000","last_change":"2024-04-24T23:34:54.380959+0000","last_active":"2024-04-24T23:35:00.540883+0000","last_peered":"2024-04-24T23:35:00.540883+0000","last_clean":"2024-04-24T23:35:00.540883+0000","last_became_active":"2024-04-24T23:34:54.380702+0000","last_became_peered":"2024-04-24T23:34:54.380702+0000","last_unstale":"2024-04-24T23:35:00.540883+0000","last_undegraded":"2024-04-24T23:35:00.540883+0000","last_fullsized":"2024-04-24T23:35:00.540883+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:45.424046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.880840+0000","last_change":"2024-04-24T23:34:54.358461+0000","last_active":"2024-04-24T23:34:59.880840+0000","last_peered":"2024-04-24T23:34:59.880840+0000","last_clean":"2024-04-24T23:34:59.880840+0000","last_became_active":"2024-04-24T23:34:54.358203+0000","last_became_peered":"2024-04-24T23:34:54.358203+0000","last_unstale":"2024-04-24T23:34:59.880840+0000","last_undegraded":"2024-04-24T23:34:59.880840+0000","last_fullsized":"2024-04-24T23:34:59.880840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:18.207267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.427594+0000","last_change":"2024-04-24T23:34:54.367860+0000","last_active":"2024-04-24T23:34:59.427594+0000","last_peered":"2024-04-24T23:34:59.427594+0000","last_clean":"2024-04-24T23:34:59.427594+0000","last_became_active":"2024-04-24T23:34:54.367595+0000","last_became_peered":"2024-04-24T23:34:54.367595+0000","last_unstale":"2024-04-24T23:34:59.427594+0000","last_undegraded":"2024-04-24T23:34:59.427594+0000","last_fullsized":"2024-04-24T23:34:59.427594+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:40:40.928386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.396558+0000","last_change":"2024-04-24T23:34:54.373640+0000","last_active":"2024-04-24T23:34:59.396558+0000","last_peered":"2024-04-24T23:34:59.396558+0000","last_clean":"2024-04-24T23:34:59.396558+0000","last_became_active":"2024-04-24T23:34:54.373404+0000","last_became_peered":"2024-04-24T23:34:54.373404+0000","last_unstale":"2024-04-24T23:34:59.396558+0000","last_undegraded":"2024-04-24T23:34:59.396558+0000","last_fullsized":"2024-04-24T23:34:59.396558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:41:21.841042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.397052+0000","last_change":"2024-04-24T23:34:54.377573+0000","last_active":"2024-04-24T23:34:59.397052+0000","last_peered":"2024-04-24T23:34:59.397052+0000","last_clean":"2024-04-24T23:34:59.397052+0000","last_became_active":"2024-04-24T23:34:54.377277+0000","last_became_peered":"2024-04-24T23:34:54.377277+0000","last_unstale":"2024-04-24T23:34:59.397052+0000","last_undegraded":"2024-04-24T23:34:59.397052+0000","last_fullsized":"2024-04-24T23:34:59.397052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:43.211314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540693+0000","last_change":"2024-04-24T23:34:54.388134+0000","last_active":"2024-04-24T23:35:00.540693+0000","last_peered":"2024-04-24T23:35:00.540693+0000","last_clean":"2024-04-24T23:35:00.540693+0000","last_became_active":"2024-04-24T23:34:54.387781+0000","last_became_peered":"2024-04-24T23:34:54.387781+0000","last_unstale":"2024-04-24T23:35:00.540693+0000","last_undegraded":"2024-04-24T23:35:00.540693+0000","last_fullsized":"2024-04-24T23:35:00.540693+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:56:17.763159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.541198+0000","last_change":"2024-04-24T23:34:54.388182+0000","last_active":"2024-04-24T23:35:00.541198+0000","last_peered":"2024-04-24T23:35:00.541198+0000","last_clean":"2024-04-24T23:35:00.541198+0000","last_became_active":"2024-04-24T23:34:54.387913+0000","last_became_peered":"2024-04-24T23:34:54.387913+0000","last_unstale":"2024-04-24T23:35:00.541198+0000","last_undegraded":"2024-04-24T23:35:00.541198+0000","last_fullsized":"2024-04-24T23:35:00.541198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:18.207267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.502012+0000","last_change":"2024-04-24T23:34:54.383303+0000","last_active":"2024-04-24T23:34:59.502012+0000","last_peered":"2024-04-24T23:34:59.502012+0000","last_clean":"2024-04-24T23:34:59.502012+0000","last_became_active":"2024-04-24T23:34:54.383024+0000","last_became_peered":"2024-04-24T23:34:54.383024+0000","last_unstale":"2024-04-24T23:34:59.502012+0000","last_undegraded":"2024-04-24T23:34:59.502012+0000","last_fullsized":"2024-04-24T23:34:59.502012+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:45.424046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.395629+0000","last_change":"2024-04-24T23:34:54.382803+0000","last_active":"2024-04-24T23:34:59.395629+0000","last_peered":"2024-04-24T23:34:59.395629+0000","last_clean":"2024-04-24T23:34:59.395629+0000","last_became_active":"2024-04-24T23:34:54.382482+0000","last_became_peered":"2024-04-24T23:34:54.382482+0000","last_unstale":"2024-04-24T23:34:59.395629+0000","last_undegraded":"2024-04-24T23:34:59.395629+0000","last_fullsized":"2024-04-24T23:34:59.395629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:27:33.721584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501319+0000","last_change":"2024-04-24T23:34:54.369167+0000","last_active":"2024-04-24T23:34:59.501319+0000","last_peered":"2024-04-24T23:34:59.501319+0000","last_clean":"2024-04-24T23:34:59.501319+0000","last_became_active":"2024-04-24T23:34:54.368663+0000","last_became_peered":"2024-04-24T23:34:54.368663+0000","last_unstale":"2024-04-24T23:34:59.501319+0000","last_undegraded":"2024-04-24T23:34:59.501319+0000","last_fullsized":"2024-04-24T23:34:59.501319+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:18.207267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540351+0000","last_change":"2024-04-24T23:34:54.380345+0000","last_active":"2024-04-24T23:35:00.540351+0000","last_peered":"2024-04-24T23:35:00.540351+0000","last_clean":"2024-04-24T23:35:00.540351+0000","last_became_active":"2024-04-24T23:34:54.380052+0000","last_became_peered":"2024-04-24T23:34:54.380052+0000","last_unstale":"2024-04-24T23:35:00.540351+0000","last_undegraded":"2024-04-24T23:35:00.540351+0000","last_fullsized":"2024-04-24T23:35:00.540351+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:42.199856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501438+0000","last_change":"2024-04-24T23:34:54.365114+0000","last_active":"2024-04-24T23:34:59.501438+0000","last_peered":"2024-04-24T23:34:59.501438+0000","last_clean":"2024-04-24T23:34:59.501438+0000","last_became_active":"2024-04-24T23:34:54.364605+0000","last_became_peered":"2024-04-24T23:34:54.364605+0000","last_unstale":"2024-04-24T23:34:59.501438+0000","last_undegraded":"2024-04-24T23:34:59.501438+0000","last_fullsized":"2024-04-24T23:34:59.501438+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:07:28.785202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:35:00.540539+0000","last_change":"2024-04-24T23:34:54.374843+0000","last_active":"2024-04-24T23:35:00.540539+0000","last_peered":"2024-04-24T23:35:00.540539+0000","last_clean":"2024-04-24T23:35:00.540539+0000","last_became_active":"2024-04-24T23:34:54.373117+0000","last_became_peered":"2024-04-24T23:34:54.373117+0000","last_unstale":"2024-04-24T23:35:00.540539+0000","last_undegraded":"2024-04-24T23:35:00.540539+0000","last_fullsized":"2024-04-24T23:35:00.540539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:37.323551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.427656+0000","last_change":"2024-04-24T23:34:54.366079+0000","last_active":"2024-04-24T23:34:59.427656+0000","last_peered":"2024-04-24T23:34:59.427656+0000","last_clean":"2024-04-24T23:34:59.427656+0000","last_became_active":"2024-04-24T23:34:54.365814+0000","last_became_peered":"2024-04-24T23:34:54.365814+0000","last_unstale":"2024-04-24T23:34:59.427656+0000","last_undegraded":"2024-04-24T23:34:59.427656+0000","last_fullsized":"2024-04-24T23:34:59.427656+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:18.207267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.881393+0000","last_change":"2024-04-24T23:34:54.368976+0000","last_active":"2024-04-24T23:34:59.881393+0000","last_peered":"2024-04-24T23:34:59.881393+0000","last_clean":"2024-04-24T23:34:59.881393+0000","last_became_active":"2024-04-24T23:34:54.368663+0000","last_became_peered":"2024-04-24T23:34:54.368663+0000","last_unstale":"2024-04-24T23:34:59.881393+0000","last_undegraded":"2024-04-24T23:34:59.881393+0000","last_fullsized":"2024-04-24T23:34:59.881393+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:56:17.763159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T23:34:59.501905+0000","last_change":"2024-04-24T23:34:54.373810+0000","last_active":"2024-04-24T23:34:59.501905+0000","last_peered":"2024-04-24T23:34:59.501905+0000","last_clean":"2024-04-24T23:34:59.501905+0000","last_became_active":"2024-04-24T23:34:54.373531+0000","last_became_peered":"2024-04-24T23:34:54.373531+0000","last_unstale":"2024-04-24T23:34:59.501905+0000","last_undegraded":"2024-04-24T23:34:59.501905+0000","last_fullsized":"2024-04-24T23:34:59.501905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:50:17.621315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65248,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542152,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":331776,"data_stored":694125,"data_compressed":32624,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542153,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":274432,"data_stored":107017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542152,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":103265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542152,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":103301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542152,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":262144,"data_stored":103175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542152,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31444,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340396,"statfs":{"total":96636764160,"available":96604565504,"internally_reserved":0,"allocated":270336,"data_stored":103335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542152,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31520,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340320,"statfs":{"total":96636764160,"available":96604487680,"internally_reserved":0,"allocated":339968,"data_stored":694181,"data_compressed":32624,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31444,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340396,"statfs":{"total":96636764160,"available":96604565504,"internally_reserved":0,"allocated":274432,"data_stored":107017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32624,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32624,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:35:22.082 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-24T23:35:22.082 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-24T23:35:22.082 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-24T23:35:22.082 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-24T23:35:22.212 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.211+0000 7fdc50fab640 1 Processor -- start 2024-04-24T23:35:22.212 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.211+0000 7fdc50fab640 1 -- start start 2024-04-24T23:35:22.213 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.212+0000 7fdc50fab640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc4c0c4b20 0x7fdc4c0c2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:22.213 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.212+0000 7fdc50fab640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fdc4c14b720 0x7fdc4c14bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:22.213 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.212+0000 7fdc50fab640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc4c06d3c0 con 0x7fdc4c073130 2024-04-24T23:35:22.213 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.212+0000 7fdc50fab640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fdc4c06c400 con 0x7fdc4c14b720 2024-04-24T23:35:22.214 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.212+0000 7fdc50fab640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fdc4c073510 con 0x7fdc4c0c4b20 2024-04-24T23:35:22.214 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.212+0000 7fdc4a59c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fdc4c073130 0x7fdc4c06ccb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:47744/0 (socket says 172.21.15.157:47744) 2024-04-24T23:35:22.214 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.212+0000 7fdc4a59c640 1 -- 172.21.15.157:0/2796182191 learned_addr learned my addr 172.21.15.157:0/2796182191 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-24T23:35:22.214 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.212+0000 7fdc49d9b640 1 --2- 172.21.15.157:0/2796182191 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc4c0c4b20 0x7fdc4c0c2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:22.214 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.212+0000 7fdc4ad9d640 1 --2- 172.21.15.157:0/2796182191 >> v2:172.21.15.169:3300/0 conn(0x7fdc4c14b720 0x7fdc4c14bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:22.214 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.212+0000 7fdc4959a640 1 -- 172.21.15.157:0/2796182191 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1068131777 0 0) 0x7fdc4c06d3c0 con 0x7fdc4c073130 2024-04-24T23:35:22.214 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.212+0000 7fdc4959a640 1 -- 172.21.15.157:0/2796182191 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdc34003520 con 0x7fdc4c073130 2024-04-24T23:35:22.214 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.213+0000 7fdc49d9b640 1 -- 172.21.15.157:0/2796182191 >> v2:172.21.15.169:3300/0 conn(0x7fdc4c14b720 msgr2=0x7fdc4c14bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:22.214 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.213+0000 7fdc49d9b640 1 --2- 172.21.15.157:0/2796182191 >> v2:172.21.15.169:3300/0 conn(0x7fdc4c14b720 0x7fdc4c14bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:22.214 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.213+0000 7fdc4959a640 1 -- 172.21.15.157:0/2796182191 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1994020362 0 0) 0x7fdc34003520 con 0x7fdc4c073130 2024-04-24T23:35:22.214 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.213+0000 7fdc49d9b640 1 -- 172.21.15.157:0/2796182191 >> v1:172.21.15.157:6789/0 conn(0x7fdc4c073130 legacy=0x7fdc4c06ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:22.214 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.213+0000 7fdc49d9b640 1 -- 172.21.15.157:0/2796182191 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc4c06c6a0 con 0x7fdc4c0c4b20 2024-04-24T23:35:22.214 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.213+0000 7fdc4ad9d640 1 --2- 172.21.15.157:0/2796182191 >> v2:172.21.15.169:3300/0 conn(0x7fdc4c14b720 0x7fdc4c14bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:35:22.216 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.213+0000 7fdc49d9b640 1 --2- 172.21.15.157:0/2796182191 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc4c0c4b20 0x7fdc4c0c2990 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdc400097a0 tx=0x7fdc4002a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a09b93d809a2f65f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:22.216 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.213+0000 7fdc4959a640 1 -- 172.21.15.157:0/2796182191 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc40037070 con 0x7fdc4c0c4b20 2024-04-24T23:35:22.216 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.214+0000 7fdc4959a640 1 -- 172.21.15.157:0/2796182191 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc4002ab10 con 0x7fdc4c0c4b20 2024-04-24T23:35:22.216 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.214+0000 7fdc4959a640 1 -- 172.21.15.157:0/2796182191 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c001670 con 0x7fdc4c0c4b20 2024-04-24T23:35:22.216 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.214+0000 7fdc50fab640 1 -- 172.21.15.157:0/2796182191 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc4c0c4b20 msgr2=0x7fdc4c0c2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:22.216 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.214+0000 7fdc50fab640 1 --2- 172.21.15.157:0/2796182191 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc4c0c4b20 0x7fdc4c0c2990 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdc400097a0 tx=0x7fdc4002a080 comp rx=0 tx=0).stop 2024-04-24T23:35:22.216 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.214+0000 7fdc50fab640 1 -- 172.21.15.157:0/2796182191 shutdown_connections 2024-04-24T23:35:22.216 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.214+0000 7fdc50fab640 1 --2- 172.21.15.157:0/2796182191 >> v2:172.21.15.169:3300/0 conn(0x7fdc4c14b720 0x7fdc4c14bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:22.217 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.214+0000 7fdc50fab640 1 --2- 172.21.15.157:0/2796182191 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc4c0c4b20 0x7fdc4c0c2990 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:22.217 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.214+0000 7fdc50fab640 1 -- 172.21.15.157:0/2796182191 >> 172.21.15.157:0/2796182191 conn(0x7fdc4c0a8720 msgr2=0x7fdc4c0a8b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:22.217 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.214+0000 7fdc50fab640 1 -- 172.21.15.157:0/2796182191 shutdown_connections 2024-04-24T23:35:22.217 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.214+0000 7fdc50fab640 1 -- 172.21.15.157:0/2796182191 wait complete. 2024-04-24T23:35:22.217 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.215+0000 7fdc50fab640 1 Processor -- start 2024-04-24T23:35:22.217 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.215+0000 7fdc50fab640 1 -- start start 2024-04-24T23:35:22.219 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.216+0000 7fdc50fab640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc4c073130 0x7fdc4c1dd610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:22.221 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.216+0000 7fdc50fab640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc4c0c4b20 0x7fdc4c0ac090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:22.221 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.216+0000 7fdc50fab640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc4c14b720 0x7fdc4c0ac5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:22.221 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.216+0000 7fdc50fab640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdc4c0c2470 con 0x7fdc4c073130 2024-04-24T23:35:22.221 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.216+0000 7fdc50fab640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdc4c0c2190 con 0x7fdc4c0c4b20 2024-04-24T23:35:22.221 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.216+0000 7fdc50fab640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fdc4c14c0f0 con 0x7fdc4c14b720 2024-04-24T23:35:22.221 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.216+0000 7fdc4a59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc4c073130 0x7fdc4c1dd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:22.221 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.216+0000 7fdc4a59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc4c073130 0x7fdc4c1dd610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43038/0 (socket says 172.21.15.157:43038) 2024-04-24T23:35:22.221 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.216+0000 7fdc4a59c640 1 -- 172.21.15.157:0/3439999917 learned_addr learned my addr 172.21.15.157:0/3439999917 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:22.221 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.216+0000 7fdc49d9b640 1 --2- 172.21.15.157:0/3439999917 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc4c0c4b20 0x7fdc4c0ac090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:22.222 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.216+0000 7fdc4ad9d640 1 --2- 172.21.15.157:0/3439999917 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc4c14b720 0x7fdc4c0ac5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:22.222 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.217+0000 7fdc4a59c640 1 -- 172.21.15.157:0/3439999917 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc4c14b720 msgr2=0x7fdc4c0ac5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:22.222 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.217+0000 7fdc4a59c640 1 --2- 172.21.15.157:0/3439999917 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc4c14b720 0x7fdc4c0ac5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:22.222 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.217+0000 7fdc4a59c640 1 -- 172.21.15.157:0/3439999917 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc4c0c4b20 msgr2=0x7fdc4c0ac090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:22.222 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.217+0000 7fdc4a59c640 1 --2- 172.21.15.157:0/3439999917 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc4c0c4b20 0x7fdc4c0ac090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:22.222 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.217+0000 7fdc4a59c640 1 -- 172.21.15.157:0/3439999917 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc4c146fe0 con 0x7fdc4c073130 2024-04-24T23:35:22.222 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.217+0000 7fdc4a59c640 1 --2- 172.21.15.157:0/3439999917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc4c073130 0x7fdc4c1dd610 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c00c780 tx=0x7fdc3c00cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:22.222 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.217+0000 7fdc3affd640 1 -- 172.21.15.157:0/3439999917 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c017070 con 0x7fdc4c073130 2024-04-24T23:35:22.222 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.217+0000 7fdc3affd640 1 -- 172.21.15.157:0/3439999917 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc3c005ad0 con 0x7fdc4c073130 2024-04-24T23:35:22.222 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.217+0000 7fdc50fab640 1 -- 172.21.15.157:0/3439999917 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc4c150280 con 0x7fdc4c073130 2024-04-24T23:35:22.222 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.217+0000 7fdc3affd640 1 -- 172.21.15.157:0/3439999917 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c007070 con 0x7fdc4c073130 2024-04-24T23:35:22.223 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.217+0000 7fdc50fab640 1 -- 172.21.15.157:0/3439999917 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc4c147560 con 0x7fdc4c073130 2024-04-24T23:35:22.223 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.218+0000 7fdc50fab640 1 -- 172.21.15.157:0/3439999917 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc4c150450 con 0x7fdc4c073130 2024-04-24T23:35:22.223 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.218+0000 7fdc3affd640 1 -- 172.21.15.157:0/3439999917 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdc3c01a070 con 0x7fdc4c073130 2024-04-24T23:35:22.223 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.219+0000 7fdc3affd640 1 --2- 172.21.15.157:0/3439999917 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fdc2c06be30 0x7fdc2c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:22.223 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.219+0000 7fdc3affd640 1 -- 172.21.15.157:0/3439999917 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fdc3c08a510 con 0x7fdc4c073130 2024-04-24T23:35:22.223 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.219+0000 7fdc49d9b640 1 --2- 172.21.15.157:0/3439999917 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fdc2c06be30 0x7fdc2c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:22.223 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.220+0000 7fdc49d9b640 1 --2- 172.21.15.157:0/3439999917 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fdc2c06be30 0x7fdc2c06e2f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdc40034000 tx=0x7fdc4002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:22.224 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.223+0000 7fdc3affd640 1 -- 172.21.15.157:0/3439999917 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdc3c0585c0 con 0x7fdc4c073130 2024-04-24T23:35:22.438 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.438+0000 7fdc50fab640 1 -- 172.21.15.157:0/3439999917 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fdc4c072cc0 con 0x7fdc4c073130 2024-04-24T23:35:22.439 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.438+0000 7fdc3affd640 1 -- 172.21.15.157:0/3439999917 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fdc4c072cc0 con 0x7fdc4c073130 2024-04-24T23:35:22.439 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:35:22.439 INFO:teuthology.orchestra.run.smithi157.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-24T23:35:22.441 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.440+0000 7fdc50fab640 1 -- 172.21.15.157:0/3439999917 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fdc2c06be30 msgr2=0x7fdc2c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:22.441 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.440+0000 7fdc50fab640 1 --2- 172.21.15.157:0/3439999917 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fdc2c06be30 0x7fdc2c06e2f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdc40034000 tx=0x7fdc4002e000 comp rx=0 tx=0).stop 2024-04-24T23:35:22.441 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.440+0000 7fdc50fab640 1 -- 172.21.15.157:0/3439999917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc4c073130 msgr2=0x7fdc4c1dd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:22.441 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.440+0000 7fdc50fab640 1 --2- 172.21.15.157:0/3439999917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc4c073130 0x7fdc4c1dd610 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c00c780 tx=0x7fdc3c00cb80 comp rx=0 tx=0).stop 2024-04-24T23:35:22.442 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.440+0000 7fdc50fab640 1 -- 172.21.15.157:0/3439999917 shutdown_connections 2024-04-24T23:35:22.442 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.440+0000 7fdc50fab640 1 --2- 172.21.15.157:0/3439999917 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fdc2c06be30 0x7fdc2c06e2f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:22.442 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.440+0000 7fdc50fab640 1 --2- 172.21.15.157:0/3439999917 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc4c14b720 0x7fdc4c0ac5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:22.442 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.440+0000 7fdc50fab640 1 --2- 172.21.15.157:0/3439999917 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc4c0c4b20 0x7fdc4c0ac090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:22.442 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.440+0000 7fdc50fab640 1 --2- 172.21.15.157:0/3439999917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc4c073130 0x7fdc4c1dd610 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:22.442 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.440+0000 7fdc50fab640 1 -- 172.21.15.157:0/3439999917 >> 172.21.15.157:0/3439999917 conn(0x7fdc4c0a8720 msgr2=0x7fdc4c06e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:22.442 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.440+0000 7fdc50fab640 1 -- 172.21.15.157:0/3439999917 shutdown_connections 2024-04-24T23:35:22.442 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.440+0000 7fdc50fab640 1 -- 172.21.15.157:0/3439999917 wait complete. 2024-04-24T23:35:22.452 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-24T23:35:22.452 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:35:22.549 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.549+0000 7fcdad361640 1 Processor -- start 2024-04-24T23:35:22.549 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.549+0000 7fcdad361640 1 -- start start 2024-04-24T23:35:22.551 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.549+0000 7fcdad361640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcda80be360 0x7fcda8155350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:22.551 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.549+0000 7fcdad361640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcda8155890 0x7fcda815fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:22.551 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.549+0000 7fcdad361640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcda80c8bb0 con 0x7fcda8152b40 2024-04-24T23:35:22.551 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.549+0000 7fcdad361640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fcda80c8e90 con 0x7fcda80be360 2024-04-24T23:35:22.551 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.549+0000 7fcdad361640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcda8159480 con 0x7fcda8155890 2024-04-24T23:35:22.551 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.550+0000 7fcda67fc640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcda80be360 0x7fcda8155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:22.551 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.550+0000 7fcda77fe640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcda8155890 0x7fcda815fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:22.551 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.550+0000 7fcda6ffd640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fcda8152b40 0x7fcda80c5630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.157:42636/0 (socket says 172.21.15.157:42636) 2024-04-24T23:35:22.551 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.550+0000 7fcda77fe640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcda8155890 0x7fcda815fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.157:53094/0 (socket says 172.21.15.157:53094) 2024-04-24T23:35:22.552 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.550+0000 7fcda67fc640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcda80be360 0x7fcda8155350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.157:45292/0 (socket says 172.21.15.157:45292) 2024-04-24T23:35:22.552 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.550+0000 7fcda6ffd640 1 -- 172.21.15.157:0/3611691320 learned_addr learned my addr 172.21.15.157:0/3611691320 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-24T23:35:22.552 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.550+0000 7fcda67fc640 1 -- 172.21.15.157:0/3611691320 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcda8155890 msgr2=0x7fcda815fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:22.552 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.550+0000 7fcda67fc640 1 --2- 172.21.15.157:0/3611691320 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcda8155890 0x7fcda815fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:22.552 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.550+0000 7fcda67fc640 1 -- 172.21.15.157:0/3611691320 >> v1:172.21.15.169:6789/0 conn(0x7fcda8152b40 legacy=0x7fcda80c5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:22.552 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.550+0000 7fcda67fc640 1 -- 172.21.15.157:0/3611691320 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcda8159760 con 0x7fcda80be360 2024-04-24T23:35:22.552 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.550+0000 7fcda5ffb640 1 -- 172.21.15.157:0/3611691320 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1133316738 0 0) 0x7fcda80c8bb0 con 0x7fcda8152b40 2024-04-24T23:35:22.552 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.550+0000 7fcda77fe640 1 --2- 172.21.15.157:0/3611691320 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcda8155890 0x7fcda815fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:35:22.555 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.551+0000 7fcda67fc640 1 --2- 172.21.15.157:0/3611691320 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcda80be360 0x7fcda8155350 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c0097a0 tx=0x7fcd9c02a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=360c2e68db45e017 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:22.555 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.551+0000 7fcda5ffb640 1 -- 172.21.15.157:0/3611691320 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd9c037070 con 0x7fcda80be360 2024-04-24T23:35:22.555 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.551+0000 7fcda5ffb640 1 -- 172.21.15.157:0/3611691320 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd9c02adb0 con 0x7fcda80be360 2024-04-24T23:35:22.556 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.551+0000 7fcda5ffb640 1 -- 172.21.15.157:0/3611691320 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd9c032660 con 0x7fcda80be360 2024-04-24T23:35:22.556 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.551+0000 7fcdad361640 1 -- 172.21.15.157:0/3611691320 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcda80be360 msgr2=0x7fcda8155350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:22.556 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.551+0000 7fcdad361640 1 --2- 172.21.15.157:0/3611691320 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcda80be360 0x7fcda8155350 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c0097a0 tx=0x7fcd9c02a060 comp rx=0 tx=0).stop 2024-04-24T23:35:22.556 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.551+0000 7fcdad361640 1 -- 172.21.15.157:0/3611691320 shutdown_connections 2024-04-24T23:35:22.556 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.551+0000 7fcdad361640 1 --2- 172.21.15.157:0/3611691320 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcda8155890 0x7fcda815fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:22.556 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.551+0000 7fcdad361640 1 --2- 172.21.15.157:0/3611691320 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcda80be360 0x7fcda8155350 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:22.556 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.551+0000 7fcdad361640 1 -- 172.21.15.157:0/3611691320 >> 172.21.15.157:0/3611691320 conn(0x7fcda80672c0 msgr2=0x7fcda80676d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:22.556 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.551+0000 7fcdad361640 1 -- 172.21.15.157:0/3611691320 shutdown_connections 2024-04-24T23:35:22.556 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.551+0000 7fcdad361640 1 -- 172.21.15.157:0/3611691320 wait complete. 2024-04-24T23:35:22.556 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.553+0000 7fcdad361640 1 Processor -- start 2024-04-24T23:35:22.556 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.553+0000 7fcdad361640 1 -- start start 2024-04-24T23:35:22.556 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.553+0000 7fcdad361640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcda80be360 0x7fcda8154fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:22.557 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.553+0000 7fcdad361640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcda8152b40 0x7fcda81eeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:22.557 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.553+0000 7fcdad361640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcda8155890 0x7fcda81ef0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:22.557 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.553+0000 7fcdad361640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcda8155d40 con 0x7fcda8152b40 2024-04-24T23:35:22.557 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.553+0000 7fcdad361640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcda80cb210 con 0x7fcda80be360 2024-04-24T23:35:22.557 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.553+0000 7fcdad361640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcda80caa00 con 0x7fcda8155890 2024-04-24T23:35:22.557 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.553+0000 7fcda67fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcda8152b40 0x7fcda81eeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:22.557 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.553+0000 7fcda67fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcda8152b40 0x7fcda81eeb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43052/0 (socket says 172.21.15.157:43052) 2024-04-24T23:35:22.557 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.553+0000 7fcda67fc640 1 -- 172.21.15.157:0/490415693 learned_addr learned my addr 172.21.15.157:0/490415693 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:22.557 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.554+0000 7fcda77fe640 1 --2- 172.21.15.157:0/490415693 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcda8155890 0x7fcda81ef0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:22.557 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.554+0000 7fcda6ffd640 1 --2- 172.21.15.157:0/490415693 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcda80be360 0x7fcda8154fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:22.557 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.554+0000 7fcda67fc640 1 -- 172.21.15.157:0/490415693 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcda8155890 msgr2=0x7fcda81ef0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:22.557 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.554+0000 7fcda67fc640 1 --2- 172.21.15.157:0/490415693 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcda8155890 0x7fcda81ef0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:22.557 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.554+0000 7fcda67fc640 1 -- 172.21.15.157:0/490415693 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcda80be360 msgr2=0x7fcda8154fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:22.557 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.554+0000 7fcda67fc640 1 --2- 172.21.15.157:0/490415693 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcda80be360 0x7fcda8154fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:22.558 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.554+0000 7fcda67fc640 1 -- 172.21.15.157:0/490415693 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcda80aced0 con 0x7fcda8152b40 2024-04-24T23:35:22.558 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.554+0000 7fcda67fc640 1 --2- 172.21.15.157:0/490415693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcda8152b40 0x7fcda81eeb60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c035040 tx=0x7fcd9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:22.558 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.554+0000 7fcd8f7fe640 1 -- 172.21.15.157:0/490415693 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd9c041070 con 0x7fcda8152b40 2024-04-24T23:35:22.558 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.554+0000 7fcd8f7fe640 1 -- 172.21.15.157:0/490415693 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd9c03c680 con 0x7fcda8152b40 2024-04-24T23:35:22.558 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.554+0000 7fcdad361640 1 -- 172.21.15.157:0/490415693 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcda81516f0 con 0x7fcda8152b40 2024-04-24T23:35:22.558 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.554+0000 7fcd8f7fe640 1 -- 172.21.15.157:0/490415693 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd9c037040 con 0x7fcda8152b40 2024-04-24T23:35:22.558 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.554+0000 7fcdad361640 1 -- 172.21.15.157:0/490415693 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcda80ad140 con 0x7fcda8152b40 2024-04-24T23:35:22.558 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.555+0000 7fcdad361640 1 -- 172.21.15.157:0/490415693 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcda80c8a90 con 0x7fcda8152b40 2024-04-24T23:35:22.559 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.555+0000 7fcd8f7fe640 1 -- 172.21.15.157:0/490415693 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcd9c03a070 con 0x7fcda8152b40 2024-04-24T23:35:22.559 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.556+0000 7fcd8f7fe640 1 --2- 172.21.15.157:0/490415693 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fcd9406be30 0x7fcd9406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:22.559 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.556+0000 7fcd8f7fe640 1 -- 172.21.15.157:0/490415693 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fcd9c0ab460 con 0x7fcda8152b40 2024-04-24T23:35:22.559 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.556+0000 7fcda6ffd640 1 --2- 172.21.15.157:0/490415693 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fcd9406be30 0x7fcd9406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:22.559 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.557+0000 7fcda6ffd640 1 --2- 172.21.15.157:0/490415693 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fcd9406be30 0x7fcd9406e2f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcda00012e0 tx=0x7fcda0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:22.560 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.560+0000 7fcd8f7fe640 1 -- 172.21.15.157:0/490415693 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcda80c8a90 con 0x7fcda8152b40 2024-04-24T23:35:22.736 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.736+0000 7fcdad361640 1 -- 172.21.15.157:0/490415693 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcda80681c0 con 0x7fcda8152b40 2024-04-24T23:35:22.737 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.737+0000 7fcd8f7fe640 1 -- 172.21.15.157:0/490415693 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18538 (secure 0 0 0) 0x7fcd9c079ae0 con 0x7fcda8152b40 2024-04-24T23:35:22.738 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:35:22.738 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":25,"fsid":"223685fb-5b64-46a5-9857-3791c9c07df2","created":"2024-04-24T23:34:37.504202+0000","modified":"2024-04-24T23:34:58.744500+0000","last_up_change":"2024-04-24T23:34:47.584963+0000","last_in_change":"2024-04-24T23:34:39.632954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:34:47.796655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:34:52.177208+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:34:52.667022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:34:56.395517+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"95d14ac8-fccf-4261-be8e-edafa746f629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6809","nonce":4003822160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6811","nonce":4003822160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6815","nonce":4003822160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6813","nonce":4003822160}]},"public_addr":"172.21.15.157:6809/4003822160","cluster_addr":"172.21.15.157:6811/4003822160","heartbeat_back_addr":"172.21.15.157:6815/4003822160","heartbeat_front_addr":"172.21.15.157:6813/4003822160","state":["exists","up"]},{"osd":1,"uuid":"7be79da9-8eec-4990-9178-71d602e39d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6817","nonce":1557575576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6819","nonce":1557575576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6823","nonce":1557575576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6821","nonce":1557575576}]},"public_addr":"172.21.15.157:6817/1557575576","cluster_addr":"172.21.15.157:6819/1557575576","heartbeat_back_addr":"172.21.15.157:6823/1557575576","heartbeat_front_addr":"172.21.15.157:6821/1557575576","state":["exists","up"]},{"osd":2,"uuid":"0127ed00-b7c3-466d-bfb0-01b0fd315f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6825","nonce":2793948092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6827","nonce":2793948092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6831","nonce":2793948092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6829","nonce":2793948092}]},"public_addr":"172.21.15.157:6825/2793948092","cluster_addr":"172.21.15.157:6827/2793948092","heartbeat_back_addr":"172.21.15.157:6831/2793948092","heartbeat_front_addr":"172.21.15.157:6829/2793948092","state":["exists","up"]},{"osd":3,"uuid":"f2331acc-aa37-433e-868e-3561b6039562","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6801","nonce":422758549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6803","nonce":422758549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6807","nonce":422758549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6805","nonce":422758549}]},"public_addr":"172.21.15.157:6801/422758549","cluster_addr":"172.21.15.157:6803/422758549","heartbeat_back_addr":"172.21.15.157:6807/422758549","heartbeat_front_addr":"172.21.15.157:6805/422758549","state":["exists","up"]},{"osd":4,"uuid":"1998854b-7398-43f5-ab7d-79fa0debcf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6801","nonce":321910099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6803","nonce":321910099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6807","nonce":321910099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6805","nonce":321910099}]},"public_addr":"172.21.15.169:6801/321910099","cluster_addr":"172.21.15.169:6803/321910099","heartbeat_back_addr":"172.21.15.169:6807/321910099","heartbeat_front_addr":"172.21.15.169:6805/321910099","state":["exists","up"]},{"osd":5,"uuid":"c37153ff-9b2b-48b9-a684-2c82b6f39f15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6825","nonce":243321711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6827","nonce":243321711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6831","nonce":243321711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6829","nonce":243321711}]},"public_addr":"172.21.15.169:6825/243321711","cluster_addr":"172.21.15.169:6827/243321711","heartbeat_back_addr":"172.21.15.169:6831/243321711","heartbeat_front_addr":"172.21.15.169:6829/243321711","state":["exists","up"]},{"osd":6,"uuid":"8db79e1c-f36b-49cb-8d8a-a20c2aec57c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6809","nonce":3441420785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6811","nonce":3441420785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6815","nonce":3441420785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6813","nonce":3441420785}]},"public_addr":"172.21.15.169:6809/3441420785","cluster_addr":"172.21.15.169:6811/3441420785","heartbeat_back_addr":"172.21.15.169:6815/3441420785","heartbeat_front_addr":"172.21.15.169:6813/3441420785","state":["exists","up"]},{"osd":7,"uuid":"928586e9-296c-4b8b-9b27-42f9f155dbdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6817","nonce":1663179020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6819","nonce":1663179020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6823","nonce":1663179020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6821","nonce":1663179020}]},"public_addr":"172.21.15.169:6817/1663179020","cluster_addr":"172.21.15.169:6819/1663179020","heartbeat_back_addr":"172.21.15.169:6823/1663179020","heartbeat_front_addr":"172.21.15.169:6821/1663179020","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.848039+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.826489+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.801120+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.727706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.824343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.791915+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.878073+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.810831+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:35:22.740 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.739+0000 7fcdad361640 1 -- 172.21.15.157:0/490415693 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fcd9406be30 msgr2=0x7fcd9406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:22.740 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.739+0000 7fcdad361640 1 --2- 172.21.15.157:0/490415693 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fcd9406be30 0x7fcd9406e2f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcda00012e0 tx=0x7fcda0009040 comp rx=0 tx=0).stop 2024-04-24T23:35:22.740 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.739+0000 7fcdad361640 1 -- 172.21.15.157:0/490415693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcda8152b40 msgr2=0x7fcda81eeb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:22.741 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.739+0000 7fcdad361640 1 --2- 172.21.15.157:0/490415693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcda8152b40 0x7fcda81eeb60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c035040 tx=0x7fcd9c0046c0 comp rx=0 tx=0).stop 2024-04-24T23:35:22.741 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.739+0000 7fcdad361640 1 -- 172.21.15.157:0/490415693 shutdown_connections 2024-04-24T23:35:22.741 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.739+0000 7fcdad361640 1 --2- 172.21.15.157:0/490415693 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fcd9406be30 0x7fcd9406e2f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:22.741 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.739+0000 7fcdad361640 1 --2- 172.21.15.157:0/490415693 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcda8155890 0x7fcda81ef0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:22.741 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.739+0000 7fcdad361640 1 --2- 172.21.15.157:0/490415693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcda8152b40 0x7fcda81eeb60 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:22.741 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.739+0000 7fcdad361640 1 --2- 172.21.15.157:0/490415693 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcda80be360 0x7fcda8154fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:22.741 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.739+0000 7fcdad361640 1 -- 172.21.15.157:0/490415693 >> 172.21.15.157:0/490415693 conn(0x7fcda80672c0 msgr2=0x7fcda80ca010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:22.741 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.739+0000 7fcdad361640 1 -- 172.21.15.157:0/490415693 shutdown_connections 2024-04-24T23:35:22.741 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.739+0000 7fcdad361640 1 -- 172.21.15.157:0/490415693 wait complete. 2024-04-24T23:35:22.751 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:34:47.796655+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:34:52.177208+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:34:52.667022+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T23:34:56.395517+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T23:35:22.752 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:35:22.852 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.852+0000 7f3ccd3b7640 1 Processor -- start 2024-04-24T23:35:22.853 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.852+0000 7f3ccd3b7640 1 -- start start 2024-04-24T23:35:22.853 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.853+0000 7f3ccd3b7640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3cc81567e0 0x7f3cc8155c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:22.853 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.853+0000 7f3ccd3b7640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3cc8156c40 con 0x7f3cc80b03e0 2024-04-24T23:35:22.854 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.853+0000 7f3ccd3b7640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3cc80cb330 con 0x7f3cc80c55d0 2024-04-24T23:35:22.854 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.853+0000 7f3ccd3b7640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3cc80b07c0 con 0x7f3cc81567e0 2024-04-24T23:35:22.854 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.853+0000 7f3cc6ffd640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f3cc80b03e0 0x7f3cc81592f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:47756/0 (socket says 172.21.15.157:47756) 2024-04-24T23:35:22.854 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.853+0000 7f3cc6ffd640 1 -- 172.21.15.157:0/828257683 learned_addr learned my addr 172.21.15.157:0/828257683 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-24T23:35:22.854 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.853+0000 7f3cbffff640 1 --2- 172.21.15.157:0/828257683 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3cc81567e0 0x7f3cc8155c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:22.854 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.853+0000 7f3cc67fc640 1 -- 172.21.15.157:0/828257683 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4255940285 0 0) 0x7f3cc8156c40 con 0x7f3cc80b03e0 2024-04-24T23:35:22.854 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.853+0000 7f3cc67fc640 1 -- 172.21.15.157:0/828257683 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3cb4003520 con 0x7f3cc80b03e0 2024-04-24T23:35:22.856 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.853+0000 7f3cc67fc640 1 -- 172.21.15.157:0/828257683 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 886684243 0 0) 0x7f3cb4003520 con 0x7f3cc80b03e0 2024-04-24T23:35:22.856 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.853+0000 7f3cc67fc640 1 -- 172.21.15.157:0/828257683 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3cc81567e0 msgr2=0x7f3cc8155c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:22.857 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.853+0000 7f3cc67fc640 1 --2- 172.21.15.157:0/828257683 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3cc81567e0 0x7f3cc8155c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:22.857 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.853+0000 7f3cc67fc640 1 -- 172.21.15.157:0/828257683 >> v1:172.21.15.169:6789/0 conn(0x7f3cc80c55d0 legacy=0x7f3cc815fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:22.857 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.853+0000 7f3cc67fc640 1 -- 172.21.15.157:0/828257683 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cc80cb610 con 0x7f3cc80b03e0 2024-04-24T23:35:22.857 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.853+0000 7f3cc67fc640 1 -- 172.21.15.157:0/828257683 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1480525340 0 0) 0x7f3cb8001670 con 0x7f3cc80b03e0 2024-04-24T23:35:22.857 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.853+0000 7f3cc67fc640 1 -- 172.21.15.157:0/828257683 >> v1:172.21.15.157:6789/0 conn(0x7f3cc80b03e0 legacy=0x7f3cc81592f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:22.857 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.853+0000 7f3cc67fc640 1 --2- 172.21.15.157:0/828257683 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3cb40039e0 0x7f3cb4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:22.857 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.854+0000 7f3cc67fc640 1 --2- 172.21.15.157:0/828257683 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3cc81567e0 0x7f3cb4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:22.857 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.854+0000 7f3cc67fc640 1 --2- 172.21.15.157:0/828257683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cb4004d40 0x7f3cb40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:22.857 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.854+0000 7f3cc67fc640 1 -- 172.21.15.157:0/828257683 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb400baf0 con 0x7f3cb4004d40 2024-04-24T23:35:22.857 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.854+0000 7f3cc67fc640 1 -- 172.21.15.157:0/828257683 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb400bca0 con 0x7f3cb40039e0 2024-04-24T23:35:22.857 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.854+0000 7f3cc67fc640 1 -- 172.21.15.157:0/828257683 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3cb400bea0 con 0x7f3cc81567e0 2024-04-24T23:35:22.857 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.854+0000 7f3cc77fe640 1 --2- 172.21.15.157:0/828257683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cb4004d40 0x7f3cb40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:22.857 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.854+0000 7f3cbffff640 1 --2- 172.21.15.157:0/828257683 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3cc81567e0 0x7f3cb4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:22.857 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.854+0000 7f3cc6ffd640 1 --2- 172.21.15.157:0/828257683 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3cb40039e0 0x7f3cb4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:22.858 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.854+0000 7f3cc77fe640 1 -- 172.21.15.157:0/828257683 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3cc81567e0 msgr2=0x7f3cb4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:22.858 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.854+0000 7f3cc77fe640 1 --2- 172.21.15.157:0/828257683 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3cc81567e0 0x7f3cb4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:22.858 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.854+0000 7f3cc77fe640 1 -- 172.21.15.157:0/828257683 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3cb40039e0 msgr2=0x7f3cb4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:22.858 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.854+0000 7f3cc77fe640 1 --2- 172.21.15.157:0/828257683 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3cb40039e0 0x7f3cb4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:22.858 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.854+0000 7f3cc77fe640 1 -- 172.21.15.157:0/828257683 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cb4003520 con 0x7f3cb4004d40 2024-04-24T23:35:22.858 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.855+0000 7f3cc77fe640 1 --2- 172.21.15.157:0/828257683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cb4004d40 0x7f3cb40051c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0001e50 tx=0x7f3cb4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2edb4803c1767e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:22.858 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.855+0000 7f3cc67fc640 1 -- 172.21.15.157:0/828257683 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc0001a70 con 0x7f3cb4004d40 2024-04-24T23:35:22.858 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.855+0000 7f3cc67fc640 1 -- 172.21.15.157:0/828257683 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cc0003970 con 0x7f3cb4004d40 2024-04-24T23:35:22.858 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.855+0000 7f3cc67fc640 1 -- 172.21.15.157:0/828257683 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc0003d60 con 0x7f3cb4004d40 2024-04-24T23:35:22.858 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.855+0000 7f3ccd3b7640 1 -- 172.21.15.157:0/828257683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cb4004d40 msgr2=0x7f3cb40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:22.858 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.855+0000 7f3ccd3b7640 1 --2- 172.21.15.157:0/828257683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cb4004d40 0x7f3cb40051c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0001e50 tx=0x7f3cb4005830 comp rx=0 tx=0).stop 2024-04-24T23:35:22.858 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.855+0000 7f3cc77fe640 1 -- 172.21.15.157:0/828257683 reap_dead start 2024-04-24T23:35:22.858 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.855+0000 7f3ccd3b7640 1 -- 172.21.15.157:0/828257683 shutdown_connections 2024-04-24T23:35:22.859 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.855+0000 7f3ccd3b7640 1 -- 172.21.15.157:0/828257683 >> 172.21.15.157:0/828257683 conn(0x7f3cc8067080 msgr2=0x7f3cc8067490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:22.859 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.855+0000 7f3ccd3b7640 1 -- 172.21.15.157:0/828257683 shutdown_connections 2024-04-24T23:35:22.859 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.855+0000 7f3ccd3b7640 1 -- 172.21.15.157:0/828257683 wait complete. 2024-04-24T23:35:22.861 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.856+0000 7f3ccd3b7640 1 Processor -- start 2024-04-24T23:35:22.861 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.856+0000 7f3ccd3b7640 1 -- start start 2024-04-24T23:35:22.861 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.857+0000 7f3ccd3b7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cb4004d40 0x7f3cc80ab8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:22.861 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.857+0000 7f3ccd3b7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3cc80c4dd0 0x7f3cc814df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:22.861 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.857+0000 7f3ccd3b7640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3cc814e460 0x7f3cc80adb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:22.861 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.857+0000 7f3ccd3b7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc81556e0 con 0x7f3cb4004d40 2024-04-24T23:35:22.861 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.857+0000 7f3ccd3b7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc8155400 con 0x7f3cc80c4dd0 2024-04-24T23:35:22.861 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.857+0000 7f3ccd3b7640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3cc80cafc0 con 0x7f3cc814e460 2024-04-24T23:35:22.861 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.857+0000 7f3cc6ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cb4004d40 0x7f3cc80ab8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:22.862 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.857+0000 7f3cc6ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cb4004d40 0x7f3cc80ab8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43072/0 (socket says 172.21.15.157:43072) 2024-04-24T23:35:22.862 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.857+0000 7f3cc6ffd640 1 -- 172.21.15.157:0/4097277544 learned_addr learned my addr 172.21.15.157:0/4097277544 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:22.862 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.857+0000 7f3cbffff640 1 --2- 172.21.15.157:0/4097277544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3cc80c4dd0 0x7f3cc814df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:22.862 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.857+0000 7f3cc77fe640 1 --2- 172.21.15.157:0/4097277544 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3cc814e460 0x7f3cc80adb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:22.862 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.857+0000 7f3cc6ffd640 1 -- 172.21.15.157:0/4097277544 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3cc814e460 msgr2=0x7f3cc80adb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:22.862 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.857+0000 7f3cc6ffd640 1 --2- 172.21.15.157:0/4097277544 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3cc814e460 0x7f3cc80adb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:22.862 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.857+0000 7f3cc6ffd640 1 -- 172.21.15.157:0/4097277544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3cc80c4dd0 msgr2=0x7f3cc814df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:22.862 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.857+0000 7f3cc6ffd640 1 --2- 172.21.15.157:0/4097277544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3cc80c4dd0 0x7f3cc814df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:22.862 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.857+0000 7f3cc6ffd640 1 -- 172.21.15.157:0/4097277544 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cc81ebda0 con 0x7f3cb4004d40 2024-04-24T23:35:22.862 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.858+0000 7f3cc6ffd640 1 --2- 172.21.15.157:0/4097277544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cb4004d40 0x7f3cc80ab8f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8007340 tx=0x7f3cb8007740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:22.862 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.858+0000 7f3cbf7fe640 1 -- 172.21.15.157:0/4097277544 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb800f940 con 0x7f3cb4004d40 2024-04-24T23:35:22.862 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.858+0000 7f3cbf7fe640 1 -- 172.21.15.157:0/4097277544 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cb8018070 con 0x7f3cb4004d40 2024-04-24T23:35:22.862 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.858+0000 7f3cbf7fe640 1 -- 172.21.15.157:0/4097277544 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb800fce0 con 0x7f3cb4004d40 2024-04-24T23:35:22.863 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.858+0000 7f3ccd3b7640 1 -- 172.21.15.157:0/4097277544 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cc81ec080 con 0x7f3cb4004d40 2024-04-24T23:35:22.863 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.858+0000 7f3ccd3b7640 1 -- 172.21.15.157:0/4097277544 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cc80af580 con 0x7f3cb4004d40 2024-04-24T23:35:22.863 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.859+0000 7f3ccd3b7640 1 -- 172.21.15.157:0/4097277544 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cc8073710 con 0x7f3cb4004d40 2024-04-24T23:35:22.863 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.859+0000 7f3cbf7fe640 1 -- 172.21.15.157:0/4097277544 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3cb801d460 con 0x7f3cb4004d40 2024-04-24T23:35:22.863 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.859+0000 7f3cbf7fe640 1 --2- 172.21.15.157:0/4097277544 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f3ca806be30 0x7f3ca806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:22.863 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.860+0000 7f3cbf7fe640 1 -- 172.21.15.157:0/4097277544 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f3cb808b090 con 0x7f3cb4004d40 2024-04-24T23:35:22.863 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.860+0000 7f3cbffff640 1 --2- 172.21.15.157:0/4097277544 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f3ca806be30 0x7f3ca806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:22.863 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.860+0000 7f3cbffff640 1 --2- 172.21.15.157:0/4097277544 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f3ca806be30 0x7f3ca806e2f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8152370 tx=0x7f3cb0009110 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:22.864 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:22.864+0000 7f3cbf7fe640 1 -- 172.21.15.157:0/4097277544 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3cc8073710 con 0x7f3cb4004d40 2024-04-24T23:35:23.039 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.039+0000 7f3ccd3b7640 1 -- 172.21.15.157:0/4097277544 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3cc8157db0 con 0x7f3cb4004d40 2024-04-24T23:35:23.039 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.039+0000 7f3cbf7fe640 1 -- 172.21.15.157:0/4097277544 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7f3cc8157db0 con 0x7f3cb4004d40 2024-04-24T23:35:23.040 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2024-04-24T23:35:23.042 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.041+0000 7f3ccd3b7640 1 -- 172.21.15.157:0/4097277544 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f3ca806be30 msgr2=0x7f3ca806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:23.042 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.041+0000 7f3ccd3b7640 1 --2- 172.21.15.157:0/4097277544 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f3ca806be30 0x7f3ca806e2f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8152370 tx=0x7f3cb0009110 comp rx=0 tx=0).stop 2024-04-24T23:35:23.042 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.041+0000 7f3ccd3b7640 1 -- 172.21.15.157:0/4097277544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cb4004d40 msgr2=0x7f3cc80ab8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:23.042 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.041+0000 7f3ccd3b7640 1 --2- 172.21.15.157:0/4097277544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cb4004d40 0x7f3cc80ab8f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8007340 tx=0x7f3cb8007740 comp rx=0 tx=0).stop 2024-04-24T23:35:23.042 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.041+0000 7f3ccd3b7640 1 -- 172.21.15.157:0/4097277544 shutdown_connections 2024-04-24T23:35:23.042 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.041+0000 7f3ccd3b7640 1 --2- 172.21.15.157:0/4097277544 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f3ca806be30 0x7f3ca806e2f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:23.042 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.041+0000 7f3ccd3b7640 1 --2- 172.21.15.157:0/4097277544 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3cc814e460 0x7f3cc80adb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:23.043 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.041+0000 7f3ccd3b7640 1 --2- 172.21.15.157:0/4097277544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3cc80c4dd0 0x7f3cc814df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:23.043 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.041+0000 7f3ccd3b7640 1 --2- 172.21.15.157:0/4097277544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cb4004d40 0x7f3cc80ab8f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:23.043 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.041+0000 7f3ccd3b7640 1 -- 172.21.15.157:0/4097277544 >> 172.21.15.157:0/4097277544 conn(0x7f3cc8067080 msgr2=0x7f3cc8154930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:23.043 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.041+0000 7f3ccd3b7640 1 -- 172.21.15.157:0/4097277544 shutdown_connections 2024-04-24T23:35:23.043 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.041+0000 7f3ccd3b7640 1 -- 172.21.15.157:0/4097277544 wait complete. 2024-04-24T23:35:23.053 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:35:23.153 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.153+0000 7f9917741640 1 Processor -- start 2024-04-24T23:35:23.154 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.153+0000 7f9917741640 1 -- start start 2024-04-24T23:35:23.155 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.154+0000 7f9917741640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f99100c13c0 0x7f99100c2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:23.155 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.154+0000 7f9917741640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99101555d0 con 0x7f99100c2090 2024-04-24T23:35:23.155 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.154+0000 7f9917741640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99100c08e0 con 0x7f99100732a0 2024-04-24T23:35:23.156 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.154+0000 7f9917741640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f99100c0690 con 0x7f99100c13c0 2024-04-24T23:35:23.156 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.154+0000 7f9915cde640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f99100c2090 0x7f991015fd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:47766/0 (socket says 172.21.15.157:47766) 2024-04-24T23:35:23.156 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.154+0000 7f9915cde640 1 -- 172.21.15.157:0/3377841722 learned_addr learned my addr 172.21.15.157:0/3377841722 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-24T23:35:23.156 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.154+0000 7f9914cdc640 1 --2- 172.21.15.157:0/3377841722 >> v2:172.21.15.169:3300/0 conn(0x7f99100c13c0 0x7f99100c2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:23.156 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.154+0000 7f9907fff640 1 -- 172.21.15.157:0/3377841722 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 243650715 0 0) 0x7f99101555d0 con 0x7f99100c2090 2024-04-24T23:35:23.156 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.154+0000 7f9907fff640 1 -- 172.21.15.157:0/3377841722 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f98fc003520 con 0x7f99100c2090 2024-04-24T23:35:23.156 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.155+0000 7f9907fff640 1 -- 172.21.15.157:0/3377841722 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2495810706 0 0) 0x7f99100c08e0 con 0x7f99100732a0 2024-04-24T23:35:23.156 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.155+0000 7f9907fff640 1 -- 172.21.15.157:0/3377841722 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f99101555d0 con 0x7f99100732a0 2024-04-24T23:35:23.156 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.155+0000 7f9914cdc640 1 -- 172.21.15.157:0/3377841722 >> v1:172.21.15.169:6789/0 conn(0x7f99100732a0 legacy=0x7f991006b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:23.156 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.155+0000 7f9907fff640 1 -- 172.21.15.157:0/3377841722 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1909291342 0 0) 0x7f98fc003520 con 0x7f99100c2090 2024-04-24T23:35:23.156 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.155+0000 7f9914cdc640 1 -- 172.21.15.157:0/3377841722 >> v1:172.21.15.157:6789/0 conn(0x7f99100c2090 legacy=0x7f991015fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:23.156 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.155+0000 7f9914cdc640 1 -- 172.21.15.157:0/3377841722 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99100c0bc0 con 0x7f99100c13c0 2024-04-24T23:35:23.158 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.155+0000 7f9914cdc640 1 --2- 172.21.15.157:0/3377841722 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99100c13c0 0x7f99100c2770 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f99100c0810 tx=0x7f990002a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=449fdc78a408b152 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:23.158 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.156+0000 7f9907fff640 1 -- 172.21.15.157:0/3377841722 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9900037070 con 0x7f99100c13c0 2024-04-24T23:35:23.158 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.156+0000 7f9907fff640 1 -- 172.21.15.157:0/3377841722 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f990002ac30 con 0x7f99100c13c0 2024-04-24T23:35:23.158 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.156+0000 7f9907fff640 1 -- 172.21.15.157:0/3377841722 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9908001670 con 0x7f99100c13c0 2024-04-24T23:35:23.158 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.156+0000 7f9917741640 1 -- 172.21.15.157:0/3377841722 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99100c13c0 msgr2=0x7f99100c2770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:23.158 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.156+0000 7f9917741640 1 --2- 172.21.15.157:0/3377841722 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99100c13c0 0x7f99100c2770 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f99100c0810 tx=0x7f990002a1a0 comp rx=0 tx=0).stop 2024-04-24T23:35:23.158 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.156+0000 7f9917741640 1 -- 172.21.15.157:0/3377841722 shutdown_connections 2024-04-24T23:35:23.158 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.156+0000 7f9917741640 1 --2- 172.21.15.157:0/3377841722 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99100c13c0 0x7f99100c2770 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:23.158 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.156+0000 7f9917741640 1 -- 172.21.15.157:0/3377841722 >> 172.21.15.157:0/3377841722 conn(0x7f99100c3d60 msgr2=0x7f99100c4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:23.158 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.156+0000 7f9917741640 1 -- 172.21.15.157:0/3377841722 shutdown_connections 2024-04-24T23:35:23.158 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.156+0000 7f9917741640 1 -- 172.21.15.157:0/3377841722 wait complete. 2024-04-24T23:35:23.159 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.157+0000 7f9917741640 1 Processor -- start 2024-04-24T23:35:23.159 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.157+0000 7f9917741640 1 -- start start 2024-04-24T23:35:23.161 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.158+0000 7f9917741640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99100732a0 0x7f99100aa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:23.161 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.158+0000 7f9917741640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99100c13c0 0x7f9910158d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:23.161 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.158+0000 7f9917741640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99100c2090 0x7f9910157200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:23.161 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.158+0000 7f9917741640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f99100711a0 con 0x7f99100732a0 2024-04-24T23:35:23.161 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.158+0000 7f9917741640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f99100c0f30 con 0x7f99100c2090 2024-04-24T23:35:23.161 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.158+0000 7f9917741640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f99100c3a00 con 0x7f99100c13c0 2024-04-24T23:35:23.161 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.158+0000 7f99154dd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99100732a0 0x7f99100aa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:23.161 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.158+0000 7f99154dd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99100732a0 0x7f99100aa0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43086/0 (socket says 172.21.15.157:43086) 2024-04-24T23:35:23.162 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.158+0000 7f99154dd640 1 -- 172.21.15.157:0/3405803202 learned_addr learned my addr 172.21.15.157:0/3405803202 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:23.162 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.158+0000 7f9915cde640 1 --2- 172.21.15.157:0/3405803202 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99100c2090 0x7f9910157200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:23.162 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.158+0000 7f9914cdc640 1 --2- 172.21.15.157:0/3405803202 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99100c13c0 0x7f9910158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:23.162 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.158+0000 7f99154dd640 1 -- 172.21.15.157:0/3405803202 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99100c13c0 msgr2=0x7f9910158d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:23.162 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.158+0000 7f99154dd640 1 --2- 172.21.15.157:0/3405803202 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99100c13c0 0x7f9910158d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:23.162 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.158+0000 7f99154dd640 1 -- 172.21.15.157:0/3405803202 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99100c2090 msgr2=0x7f9910157200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:23.162 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.158+0000 7f99154dd640 1 --2- 172.21.15.157:0/3405803202 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99100c2090 0x7f9910157200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:23.162 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.158+0000 7f99154dd640 1 -- 172.21.15.157:0/3405803202 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99100aa620 con 0x7f99100732a0 2024-04-24T23:35:23.162 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.159+0000 7f99154dd640 1 --2- 172.21.15.157:0/3405803202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99100732a0 0x7f99100aa0e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f990c00c7a0 tx=0x7f990c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:23.162 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.159+0000 7f9905ffb640 1 -- 172.21.15.157:0/3405803202 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990c003b70 con 0x7f99100732a0 2024-04-24T23:35:23.162 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.159+0000 7f9905ffb640 1 -- 172.21.15.157:0/3405803202 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f990c003d10 con 0x7f99100732a0 2024-04-24T23:35:23.162 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.159+0000 7f9917741640 1 -- 172.21.15.157:0/3405803202 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99101ef0d0 con 0x7f99100732a0 2024-04-24T23:35:23.162 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.159+0000 7f9905ffb640 1 -- 172.21.15.157:0/3405803202 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990c00aa90 con 0x7f99100732a0 2024-04-24T23:35:23.163 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.159+0000 7f9917741640 1 -- 172.21.15.157:0/3405803202 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9910156a00 con 0x7f99100732a0 2024-04-24T23:35:23.163 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.160+0000 7f9917741640 1 -- 172.21.15.157:0/3405803202 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99100681c0 con 0x7f99100732a0 2024-04-24T23:35:23.163 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.160+0000 7f9905ffb640 1 -- 172.21.15.157:0/3405803202 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f990c00ac30 con 0x7f99100732a0 2024-04-24T23:35:23.163 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.160+0000 7f9905ffb640 1 --2- 172.21.15.157:0/3405803202 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f98f406be30 0x7f98f406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:23.163 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.161+0000 7f9905ffb640 1 -- 172.21.15.157:0/3405803202 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f990c08aee0 con 0x7f99100732a0 2024-04-24T23:35:23.163 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.161+0000 7f9914cdc640 1 --2- 172.21.15.157:0/3405803202 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f98f406be30 0x7f98f406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:23.163 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.161+0000 7f9914cdc640 1 --2- 172.21.15.157:0/3405803202 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f98f406be30 0x7f98f406e2f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9900034000 tx=0x7f990002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:23.165 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.165+0000 7f9905ffb640 1 -- 172.21.15.157:0/3405803202 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f990c058f90 con 0x7f99100732a0 2024-04-24T23:35:23.336 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.336+0000 7f9917741640 1 -- 172.21.15.157:0/3405803202 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9910152de0 con 0x7f99100732a0 2024-04-24T23:35:23.337 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.337+0000 7f9905ffb640 1 -- 172.21.15.157:0/3405803202 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f990c05d290 con 0x7f99100732a0 2024-04-24T23:35:23.337 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2024-04-24T23:35:23.340 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.338+0000 7f9917741640 1 -- 172.21.15.157:0/3405803202 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f98f406be30 msgr2=0x7f98f406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:23.340 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.338+0000 7f9917741640 1 --2- 172.21.15.157:0/3405803202 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f98f406be30 0x7f98f406e2f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9900034000 tx=0x7f990002e000 comp rx=0 tx=0).stop 2024-04-24T23:35:23.340 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.338+0000 7f9917741640 1 -- 172.21.15.157:0/3405803202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99100732a0 msgr2=0x7f99100aa0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:23.340 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.338+0000 7f9917741640 1 --2- 172.21.15.157:0/3405803202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99100732a0 0x7f99100aa0e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f990c00c7a0 tx=0x7f990c00cc60 comp rx=0 tx=0).stop 2024-04-24T23:35:23.340 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.339+0000 7f9917741640 1 -- 172.21.15.157:0/3405803202 shutdown_connections 2024-04-24T23:35:23.340 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.339+0000 7f9917741640 1 --2- 172.21.15.157:0/3405803202 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f98f406be30 0x7f98f406e2f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:23.340 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.339+0000 7f9917741640 1 --2- 172.21.15.157:0/3405803202 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99100c2090 0x7f9910157200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:23.340 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.339+0000 7f9917741640 1 --2- 172.21.15.157:0/3405803202 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99100c13c0 0x7f9910158d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:23.340 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.339+0000 7f9917741640 1 --2- 172.21.15.157:0/3405803202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99100732a0 0x7f99100aa0e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:23.340 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.339+0000 7f9917741640 1 -- 172.21.15.157:0/3405803202 >> 172.21.15.157:0/3405803202 conn(0x7f99100c3d60 msgr2=0x7f99100c5060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:23.340 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.339+0000 7f9917741640 1 -- 172.21.15.157:0/3405803202 shutdown_connections 2024-04-24T23:35:23.340 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.339+0000 7f9917741640 1 -- 172.21.15.157:0/3405803202 wait complete. 2024-04-24T23:35:23.350 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:35:23.450 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.449+0000 7f9e65361640 1 Processor -- start 2024-04-24T23:35:23.450 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.449+0000 7f9e65361640 1 -- start start 2024-04-24T23:35:23.451 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.450+0000 7f9e65361640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9e600b03e0 0x7f9e601567e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:23.451 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.450+0000 7f9e65361640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9e600c55d0 0x7f9e600c5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:23.451 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.450+0000 7f9e65361640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e600cb330 con 0x7f9e601592f0 2024-04-24T23:35:23.451 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.450+0000 7f9e65361640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f9e600b07c0 con 0x7f9e600b03e0 2024-04-24T23:35:23.451 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.450+0000 7f9e65361640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f9e60156310 con 0x7f9e600c55d0 2024-04-24T23:35:23.451 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.450+0000 7f9e5effd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9e600b03e0 0x7f9e601567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:23.452 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.450+0000 7f9e5effd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9e600b03e0 0x7f9e601567e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43102/0 (socket says 172.21.15.157:43102) 2024-04-24T23:35:23.452 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.450+0000 7f9e5effd640 1 -- 172.21.15.157:0/2571276864 learned_addr learned my addr 172.21.15.157:0/2571276864 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:23.452 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.450+0000 7f9e5f7fe640 1 --2- 172.21.15.157:0/2571276864 >> v2:172.21.15.169:3300/0 conn(0x7f9e600c55d0 0x7f9e600c5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:23.452 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.450+0000 7f9e5effd640 1 -- 172.21.15.157:0/2571276864 >> v2:172.21.15.169:3300/0 conn(0x7f9e600c55d0 msgr2=0x7f9e600c5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:23.452 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.450+0000 7f9e5effd640 1 --2- 172.21.15.157:0/2571276864 >> v2:172.21.15.169:3300/0 conn(0x7f9e600c55d0 0x7f9e600c5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:23.452 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.450+0000 7f9e5effd640 1 -- 172.21.15.157:0/2571276864 >> v1:172.21.15.169:6789/0 conn(0x7f9e601592f0 legacy=0x7f9e60155c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:23.452 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.450+0000 7f9e5effd640 1 -- 172.21.15.157:0/2571276864 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e600cb610 con 0x7f9e600b03e0 2024-04-24T23:35:23.455 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.451+0000 7f9e5effd640 1 --2- 172.21.15.157:0/2571276864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e600b03e0 0x7f9e601567e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9e600c6af0 tx=0x7f9e5002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2f47dc7f70d0c47 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:23.456 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.451+0000 7f9e5dffb640 1 -- 172.21.15.157:0/2571276864 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e50036070 con 0x7f9e600b03e0 2024-04-24T23:35:23.456 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.451+0000 7f9e5dffb640 1 -- 172.21.15.157:0/2571276864 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e50031600 con 0x7f9e600b03e0 2024-04-24T23:35:23.456 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.451+0000 7f9e5dffb640 1 -- 172.21.15.157:0/2571276864 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e500319d0 con 0x7f9e600b03e0 2024-04-24T23:35:23.456 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.452+0000 7f9e65361640 1 -- 172.21.15.157:0/2571276864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e600b03e0 msgr2=0x7f9e601567e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:23.456 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.452+0000 7f9e65361640 1 --2- 172.21.15.157:0/2571276864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e600b03e0 0x7f9e601567e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9e600c6af0 tx=0x7f9e5002a660 comp rx=0 tx=0).stop 2024-04-24T23:35:23.456 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.452+0000 7f9e65361640 1 -- 172.21.15.157:0/2571276864 shutdown_connections 2024-04-24T23:35:23.456 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.452+0000 7f9e65361640 1 --2- 172.21.15.157:0/2571276864 >> v2:172.21.15.169:3300/0 conn(0x7f9e600c55d0 0x7f9e600c5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:23.456 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.452+0000 7f9e65361640 1 --2- 172.21.15.157:0/2571276864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e600b03e0 0x7f9e601567e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:23.456 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.452+0000 7f9e65361640 1 -- 172.21.15.157:0/2571276864 >> 172.21.15.157:0/2571276864 conn(0x7f9e60067080 msgr2=0x7f9e60067490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:23.456 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.452+0000 7f9e65361640 1 -- 172.21.15.157:0/2571276864 shutdown_connections 2024-04-24T23:35:23.457 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.452+0000 7f9e65361640 1 -- 172.21.15.157:0/2571276864 wait complete. 2024-04-24T23:35:23.457 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.453+0000 7f9e65361640 1 Processor -- start 2024-04-24T23:35:23.457 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.453+0000 7f9e65361640 1 -- start start 2024-04-24T23:35:23.457 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.454+0000 7f9e65361640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e600b03e0 0x7f9e600aef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:23.457 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.454+0000 7f9e65361640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e600c55d0 0x7f9e6014d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:23.457 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.454+0000 7f9e65361640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e601592f0 0x7f9e6014b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:23.457 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.454+0000 7f9e65361640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9e601556e0 con 0x7f9e600c55d0 2024-04-24T23:35:23.457 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.454+0000 7f9e65361640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9e60155400 con 0x7f9e601592f0 2024-04-24T23:35:23.457 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.454+0000 7f9e65361640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9e60159780 con 0x7f9e600b03e0 2024-04-24T23:35:23.457 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.454+0000 7f9e5e7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e600c55d0 0x7f9e6014d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:23.457 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.454+0000 7f9e5e7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e600c55d0 0x7f9e6014d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43104/0 (socket says 172.21.15.157:43104) 2024-04-24T23:35:23.457 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.454+0000 7f9e5e7fc640 1 -- 172.21.15.157:0/2117670945 learned_addr learned my addr 172.21.15.157:0/2117670945 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:23.457 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.454+0000 7f9e5f7fe640 1 --2- 172.21.15.157:0/2117670945 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e601592f0 0x7f9e6014b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:23.458 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.454+0000 7f9e5effd640 1 --2- 172.21.15.157:0/2117670945 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e600b03e0 0x7f9e600aef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:23.458 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.454+0000 7f9e5e7fc640 1 -- 172.21.15.157:0/2117670945 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e600b03e0 msgr2=0x7f9e600aef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:23.458 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.454+0000 7f9e5e7fc640 1 --2- 172.21.15.157:0/2117670945 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e600b03e0 0x7f9e600aef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:23.458 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.454+0000 7f9e5e7fc640 1 -- 172.21.15.157:0/2117670945 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e601592f0 msgr2=0x7f9e6014b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:23.458 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.454+0000 7f9e5e7fc640 1 --2- 172.21.15.157:0/2117670945 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e601592f0 0x7f9e6014b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:23.458 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.454+0000 7f9e5e7fc640 1 -- 172.21.15.157:0/2117670945 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e600af6e0 con 0x7f9e600c55d0 2024-04-24T23:35:23.458 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.454+0000 7f9e5e7fc640 1 --2- 172.21.15.157:0/2117670945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e600c55d0 0x7f9e6014d710 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9e54004640 tx=0x7f9e5400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:23.458 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.454+0000 7f9e477fe640 1 -- 172.21.15.157:0/2117670945 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e54011790 con 0x7f9e600c55d0 2024-04-24T23:35:23.458 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.454+0000 7f9e477fe640 1 -- 172.21.15.157:0/2117670945 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e54017070 con 0x7f9e600c55d0 2024-04-24T23:35:23.458 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.454+0000 7f9e477fe640 1 -- 172.21.15.157:0/2117670945 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e54011bb0 con 0x7f9e600c55d0 2024-04-24T23:35:23.458 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.454+0000 7f9e65361640 1 -- 172.21.15.157:0/2117670945 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e600af9c0 con 0x7f9e600c55d0 2024-04-24T23:35:23.458 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.454+0000 7f9e65361640 1 -- 172.21.15.157:0/2117670945 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e600ca190 con 0x7f9e600c55d0 2024-04-24T23:35:23.458 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.455+0000 7f9e65361640 1 -- 172.21.15.157:0/2117670945 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e600681c0 con 0x7f9e600c55d0 2024-04-24T23:35:23.459 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.456+0000 7f9e477fe640 1 -- 172.21.15.157:0/2117670945 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9e54011d50 con 0x7f9e600c55d0 2024-04-24T23:35:23.459 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.456+0000 7f9e477fe640 1 --2- 172.21.15.157:0/2117670945 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f9e4c06be30 0x7f9e4c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:23.459 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.456+0000 7f9e477fe640 1 -- 172.21.15.157:0/2117670945 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f9e5408ace0 con 0x7f9e600c55d0 2024-04-24T23:35:23.460 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.456+0000 7f9e5effd640 1 --2- 172.21.15.157:0/2117670945 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f9e4c06be30 0x7f9e4c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:23.460 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.457+0000 7f9e5effd640 1 --2- 172.21.15.157:0/2117670945 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f9e4c06be30 0x7f9e4c06e2f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9e50033000 tx=0x7f9e50009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:23.461 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.460+0000 7f9e477fe640 1 -- 172.21.15.157:0/2117670945 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9e540593e0 con 0x7f9e600c55d0 2024-04-24T23:35:23.633 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.633+0000 7f9e65361640 1 -- 172.21.15.157:0/2117670945 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9e6006d1d0 con 0x7f9e600c55d0 2024-04-24T23:35:23.634 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.633+0000 7f9e477fe640 1 -- 172.21.15.157:0/2117670945 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f9e54058ce0 con 0x7f9e600c55d0 2024-04-24T23:35:23.634 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 64 2024-04-24T23:35:23.636 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.635+0000 7f9e65361640 1 -- 172.21.15.157:0/2117670945 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f9e4c06be30 msgr2=0x7f9e4c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:23.636 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.635+0000 7f9e65361640 1 --2- 172.21.15.157:0/2117670945 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f9e4c06be30 0x7f9e4c06e2f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9e50033000 tx=0x7f9e50009000 comp rx=0 tx=0).stop 2024-04-24T23:35:23.636 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.635+0000 7f9e65361640 1 -- 172.21.15.157:0/2117670945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e600c55d0 msgr2=0x7f9e6014d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:23.636 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.635+0000 7f9e65361640 1 --2- 172.21.15.157:0/2117670945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e600c55d0 0x7f9e6014d710 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9e54004640 tx=0x7f9e5400c8e0 comp rx=0 tx=0).stop 2024-04-24T23:35:23.636 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.635+0000 7f9e65361640 1 -- 172.21.15.157:0/2117670945 shutdown_connections 2024-04-24T23:35:23.636 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.635+0000 7f9e65361640 1 --2- 172.21.15.157:0/2117670945 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f9e4c06be30 0x7f9e4c06e2f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:23.636 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.635+0000 7f9e65361640 1 --2- 172.21.15.157:0/2117670945 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e601592f0 0x7f9e6014b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:23.636 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.635+0000 7f9e65361640 1 --2- 172.21.15.157:0/2117670945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e600c55d0 0x7f9e6014d710 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:23.636 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.635+0000 7f9e65361640 1 --2- 172.21.15.157:0/2117670945 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e600b03e0 0x7f9e600aef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:23.636 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.635+0000 7f9e65361640 1 -- 172.21.15.157:0/2117670945 >> 172.21.15.157:0/2117670945 conn(0x7f9e60067080 msgr2=0x7f9e60067460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:23.637 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.635+0000 7f9e65361640 1 -- 172.21.15.157:0/2117670945 shutdown_connections 2024-04-24T23:35:23.637 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.635+0000 7f9e65361640 1 -- 172.21.15.157:0/2117670945 wait complete. 2024-04-24T23:35:23.646 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T23:35:23.747 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.746+0000 7f5d92348640 1 Processor -- start 2024-04-24T23:35:23.747 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.746+0000 7f5d92348640 1 -- start start 2024-04-24T23:35:23.748 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.747+0000 7f5d92348640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5d8c152cb0 0x7f5d8c1595b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:23.749 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.747+0000 7f5d92348640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d8c06fed0 0x7f5d8c06b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:23.749 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.747+0000 7f5d92348640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d8c0c1430 con 0x7f5d8c0c2510 2024-04-24T23:35:23.749 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.747+0000 7f5d92348640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f5d8c0c6c80 con 0x7f5d8c152cb0 2024-04-24T23:35:23.749 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.747+0000 7f5d92348640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5d8c06c010 con 0x7f5d8c06fed0 2024-04-24T23:35:23.749 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.747+0000 7f5d908e5640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d8c06fed0 0x7f5d8c06b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:23.749 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.747+0000 7f5d8bfff640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5d8c152cb0 0x7f5d8c1595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:23.749 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.747+0000 7f5d8b7fe640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f5d8c0c2510 0x7f5d8c0c0d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.157:36832/0 (socket says 172.21.15.157:36832) 2024-04-24T23:35:23.749 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.747+0000 7f5d8bfff640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5d8c152cb0 0x7f5d8c1595b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.157:43958/0 (socket says 172.21.15.157:43958) 2024-04-24T23:35:23.749 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.748+0000 7f5d908e5640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d8c06fed0 0x7f5d8c06b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.157:48558/0 (socket says 172.21.15.157:48558) 2024-04-24T23:35:23.749 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.748+0000 7f5d8b7fe640 1 -- 172.21.15.157:0/86347050 learned_addr learned my addr 172.21.15.157:0/86347050 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-24T23:35:23.749 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.748+0000 7f5d908e5640 1 -- 172.21.15.157:0/86347050 >> v2:172.21.15.169:3300/0 conn(0x7f5d8c152cb0 msgr2=0x7f5d8c1595b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:23.749 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.748+0000 7f5d908e5640 1 --2- 172.21.15.157:0/86347050 >> v2:172.21.15.169:3300/0 conn(0x7f5d8c152cb0 0x7f5d8c1595b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:23.749 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.748+0000 7f5d908e5640 1 -- 172.21.15.157:0/86347050 >> v1:172.21.15.169:6789/0 conn(0x7f5d8c0c2510 legacy=0x7f5d8c0c0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:23.750 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.748+0000 7f5d908e5640 1 -- 172.21.15.157:0/86347050 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d8c0a6e90 con 0x7f5d8c06fed0 2024-04-24T23:35:23.750 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.748+0000 7f5d8affd640 1 -- 172.21.15.157:0/86347050 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2121556167 0 0) 0x7f5d8c0c1430 con 0x7f5d8c0c2510 2024-04-24T23:35:23.750 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.748+0000 7f5d8bfff640 1 --2- 172.21.15.157:0/86347050 >> v2:172.21.15.169:3300/0 conn(0x7f5d8c152cb0 0x7f5d8c1595b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:35:23.751 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.749+0000 7f5d908e5640 1 --2- 172.21.15.157:0/86347050 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d8c06fed0 0x7f5d8c06b160 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5d84009d90 tx=0x7f5d8402a530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68c374b84d3b98e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:23.751 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.749+0000 7f5d8affd640 1 -- 172.21.15.157:0/86347050 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d84007740 con 0x7f5d8c06fed0 2024-04-24T23:35:23.751 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.749+0000 7f5d8affd640 1 -- 172.21.15.157:0/86347050 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d840078e0 con 0x7f5d8c06fed0 2024-04-24T23:35:23.751 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.749+0000 7f5d8affd640 1 -- 172.21.15.157:0/86347050 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d84007cb0 con 0x7f5d8c06fed0 2024-04-24T23:35:23.751 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.749+0000 7f5d92348640 1 -- 172.21.15.157:0/86347050 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d8c06fed0 msgr2=0x7f5d8c06b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:23.752 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.749+0000 7f5d92348640 1 --2- 172.21.15.157:0/86347050 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d8c06fed0 0x7f5d8c06b160 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5d84009d90 tx=0x7f5d8402a530 comp rx=0 tx=0).stop 2024-04-24T23:35:23.752 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.749+0000 7f5d92348640 1 -- 172.21.15.157:0/86347050 shutdown_connections 2024-04-24T23:35:23.752 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.749+0000 7f5d92348640 1 --2- 172.21.15.157:0/86347050 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d8c06fed0 0x7f5d8c06b160 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:23.752 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.749+0000 7f5d92348640 1 --2- 172.21.15.157:0/86347050 >> v2:172.21.15.169:3300/0 conn(0x7f5d8c152cb0 0x7f5d8c1595b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:23.752 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.749+0000 7f5d92348640 1 -- 172.21.15.157:0/86347050 >> 172.21.15.157:0/86347050 conn(0x7f5d8c0c43f0 msgr2=0x7f5d8c0c47d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:23.752 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.749+0000 7f5d92348640 1 -- 172.21.15.157:0/86347050 shutdown_connections 2024-04-24T23:35:23.752 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.749+0000 7f5d92348640 1 -- 172.21.15.157:0/86347050 wait complete. 2024-04-24T23:35:23.752 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.750+0000 7f5d92348640 1 Processor -- start 2024-04-24T23:35:23.752 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.751+0000 7f5d92348640 1 -- start start 2024-04-24T23:35:23.755 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.751+0000 7f5d92348640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d8c06fed0 0x7f5d8c155820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:23.755 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.751+0000 7f5d92348640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d8c0c2510 0x7f5d8c0ab610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:23.755 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.751+0000 7f5d92348640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d8c152cb0 0x7f5d8c0abb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:23.755 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.751+0000 7f5d92348640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8c159090 con 0x7f5d8c152cb0 2024-04-24T23:35:23.755 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.751+0000 7f5d92348640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8c158db0 con 0x7f5d8c06fed0 2024-04-24T23:35:23.755 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.751+0000 7f5d92348640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5d8c0a7170 con 0x7f5d8c0c2510 2024-04-24T23:35:23.755 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.751+0000 7f5d908e5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d8c152cb0 0x7f5d8c0abb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:23.755 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.751+0000 7f5d908e5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d8c152cb0 0x7f5d8c0abb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40404/0 (socket says 172.21.15.157:40404) 2024-04-24T23:35:23.755 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.751+0000 7f5d908e5640 1 -- 172.21.15.157:0/3726727551 learned_addr learned my addr 172.21.15.157:0/3726727551 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:23.755 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.752+0000 7f5d8b7fe640 1 --2- 172.21.15.157:0/3726727551 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d8c0c2510 0x7f5d8c0ab610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:23.756 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.752+0000 7f5d8bfff640 1 --2- 172.21.15.157:0/3726727551 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d8c06fed0 0x7f5d8c155820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:23.756 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.752+0000 7f5d908e5640 1 -- 172.21.15.157:0/3726727551 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d8c0c2510 msgr2=0x7f5d8c0ab610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:23.756 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.752+0000 7f5d908e5640 1 --2- 172.21.15.157:0/3726727551 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d8c0c2510 0x7f5d8c0ab610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:23.756 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.752+0000 7f5d908e5640 1 -- 172.21.15.157:0/3726727551 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d8c06fed0 msgr2=0x7f5d8c155820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:23.756 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.752+0000 7f5d908e5640 1 --2- 172.21.15.157:0/3726727551 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d8c06fed0 0x7f5d8c155820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:23.756 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.752+0000 7f5d908e5640 1 -- 172.21.15.157:0/3726727551 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d8c0a92c0 con 0x7f5d8c152cb0 2024-04-24T23:35:23.756 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.752+0000 7f5d8bfff640 1 --2- 172.21.15.157:0/3726727551 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d8c06fed0 0x7f5d8c155820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:35:23.756 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.752+0000 7f5d908e5640 1 --2- 172.21.15.157:0/3726727551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d8c152cb0 0x7f5d8c0abb50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5d84009eb0 tx=0x7f5d84004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:23.756 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.752+0000 7f5d88ff9640 1 -- 172.21.15.157:0/3726727551 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d84040070 con 0x7f5d8c152cb0 2024-04-24T23:35:23.756 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.752+0000 7f5d88ff9640 1 -- 172.21.15.157:0/3726727551 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d84031c40 con 0x7f5d8c152cb0 2024-04-24T23:35:23.756 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.752+0000 7f5d92348640 1 -- 172.21.15.157:0/3726727551 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d8c156330 con 0x7f5d8c152cb0 2024-04-24T23:35:23.756 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.752+0000 7f5d88ff9640 1 -- 172.21.15.157:0/3726727551 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d84009450 con 0x7f5d8c152cb0 2024-04-24T23:35:23.756 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.752+0000 7f5d92348640 1 -- 172.21.15.157:0/3726727551 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d8c0acd20 con 0x7f5d8c152cb0 2024-04-24T23:35:23.756 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.753+0000 7f5d92348640 1 -- 172.21.15.157:0/3726727551 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d8c0681c0 con 0x7f5d8c152cb0 2024-04-24T23:35:23.757 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.753+0000 7f5d88ff9640 1 -- 172.21.15.157:0/3726727551 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5d84039040 con 0x7f5d8c152cb0 2024-04-24T23:35:23.757 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.754+0000 7f5d88ff9640 1 --2- 172.21.15.157:0/3726727551 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f5d7806be60 0x7f5d7806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:23.757 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.754+0000 7f5d88ff9640 1 -- 172.21.15.157:0/3726727551 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f5d840ac2a0 con 0x7f5d8c152cb0 2024-04-24T23:35:23.757 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.754+0000 7f5d8bfff640 1 --2- 172.21.15.157:0/3726727551 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f5d7806be60 0x7f5d7806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:23.757 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.755+0000 7f5d8bfff640 1 --2- 172.21.15.157:0/3726727551 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f5d7806be60 0x7f5d7806e320 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c14fb50 tx=0x7f5d7c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:23.758 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.758+0000 7f5d88ff9640 1 -- 172.21.15.157:0/3726727551 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5d8407a9a0 con 0x7f5d8c152cb0 2024-04-24T23:35:23.930 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.930+0000 7f5d92348640 1 -- 172.21.15.157:0/3726727551 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f5d8c06bdf0 con 0x7f5d8c152cb0 2024-04-24T23:35:23.931 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.930+0000 7f5d88ff9640 1 -- 172.21.15.157:0/3726727551 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7f5d8407a2a0 con 0x7f5d8c152cb0 2024-04-24T23:35:23.931 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2024-04-24T23:35:23.933 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.932+0000 7f5d92348640 1 -- 172.21.15.157:0/3726727551 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f5d7806be60 msgr2=0x7f5d7806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:23.933 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.932+0000 7f5d92348640 1 --2- 172.21.15.157:0/3726727551 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f5d7806be60 0x7f5d7806e320 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c14fb50 tx=0x7f5d7c008040 comp rx=0 tx=0).stop 2024-04-24T23:35:23.934 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.932+0000 7f5d92348640 1 -- 172.21.15.157:0/3726727551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d8c152cb0 msgr2=0x7f5d8c0abb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:23.934 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.932+0000 7f5d92348640 1 --2- 172.21.15.157:0/3726727551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d8c152cb0 0x7f5d8c0abb50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5d84009eb0 tx=0x7f5d84004a40 comp rx=0 tx=0).stop 2024-04-24T23:35:23.934 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.933+0000 7f5d92348640 1 -- 172.21.15.157:0/3726727551 shutdown_connections 2024-04-24T23:35:23.934 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.933+0000 7f5d92348640 1 --2- 172.21.15.157:0/3726727551 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f5d7806be60 0x7f5d7806e320 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:23.934 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.933+0000 7f5d92348640 1 --2- 172.21.15.157:0/3726727551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d8c152cb0 0x7f5d8c0abb50 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:23.934 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.933+0000 7f5d92348640 1 --2- 172.21.15.157:0/3726727551 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d8c0c2510 0x7f5d8c0ab610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:23.934 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.933+0000 7f5d92348640 1 --2- 172.21.15.157:0/3726727551 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d8c06fed0 0x7f5d8c155820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:23.934 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.933+0000 7f5d92348640 1 -- 172.21.15.157:0/3726727551 >> 172.21.15.157:0/3726727551 conn(0x7f5d8c0c43f0 msgr2=0x7f5d8c06b160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:23.934 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.933+0000 7f5d92348640 1 -- 172.21.15.157:0/3726727551 shutdown_connections 2024-04-24T23:35:23.934 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:23.933+0000 7f5d92348640 1 -- 172.21.15.157:0/3726727551 wait complete. 2024-04-24T23:35:23.944 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:35:24.041 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.040+0000 7f68a504c640 1 Processor -- start 2024-04-24T23:35:24.041 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.040+0000 7f68a504c640 1 -- start start 2024-04-24T23:35:24.042 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.041+0000 7f68a504c640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f68a0154e60 0x7f68a00c5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:24.042 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.041+0000 7f68a504c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f68a00c5640 0x7f68a0151ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:24.042 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.041+0000 7f68a504c640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68a014bd30 con 0x7f68a014b950 2024-04-24T23:35:24.043 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.041+0000 7f68a504c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f68a006c480 con 0x7f68a00c5640 2024-04-24T23:35:24.043 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.041+0000 7f68a504c640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f68a00c3240 con 0x7f68a0154e60 2024-04-24T23:35:24.043 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.041+0000 7f689f59e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f68a00c5640 0x7f68a0151ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:24.043 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.041+0000 7f689f59e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f68a00c5640 0x7f68a0151ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40418/0 (socket says 172.21.15.157:40418) 2024-04-24T23:35:24.043 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.041+0000 7f689f59e640 1 -- 172.21.15.157:0/1024956178 learned_addr learned my addr 172.21.15.157:0/1024956178 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:24.043 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.041+0000 7f6897fff640 1 --2- 172.21.15.157:0/1024956178 >> v2:172.21.15.169:3300/0 conn(0x7f68a0154e60 0x7f68a00c5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:24.043 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.042+0000 7f689f59e640 1 -- 172.21.15.157:0/1024956178 >> v2:172.21.15.169:3300/0 conn(0x7f68a0154e60 msgr2=0x7f68a00c5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:24.043 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.042+0000 7f689f59e640 1 --2- 172.21.15.157:0/1024956178 >> v2:172.21.15.169:3300/0 conn(0x7f68a0154e60 0x7f68a00c5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:24.043 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.042+0000 7f689f59e640 1 -- 172.21.15.157:0/1024956178 >> v1:172.21.15.169:6789/0 conn(0x7f68a014b950 legacy=0x7f68a00c2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:24.043 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.042+0000 7f689f59e640 1 -- 172.21.15.157:0/1024956178 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68a01552f0 con 0x7f68a00c5640 2024-04-24T23:35:24.045 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.042+0000 7f689f59e640 1 --2- 172.21.15.157:0/1024956178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68a00c5640 0x7f68a0151ca0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f68a0067ef0 tx=0x7f689802c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98d32de83d8db44 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:24.045 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.042+0000 7f689e59c640 1 -- 172.21.15.157:0/1024956178 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6898035610 con 0x7f68a00c5640 2024-04-24T23:35:24.045 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.043+0000 7f689e59c640 1 -- 172.21.15.157:0/1024956178 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68980357b0 con 0x7f68a00c5640 2024-04-24T23:35:24.045 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.043+0000 7f689e59c640 1 -- 172.21.15.157:0/1024956178 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6898035b80 con 0x7f68a00c5640 2024-04-24T23:35:24.045 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.043+0000 7f68a504c640 1 -- 172.21.15.157:0/1024956178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68a00c5640 msgr2=0x7f68a0151ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:24.045 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.043+0000 7f68a504c640 1 --2- 172.21.15.157:0/1024956178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68a00c5640 0x7f68a0151ca0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f68a0067ef0 tx=0x7f689802c8b0 comp rx=0 tx=0).stop 2024-04-24T23:35:24.045 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.043+0000 7f68a504c640 1 -- 172.21.15.157:0/1024956178 shutdown_connections 2024-04-24T23:35:24.045 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.043+0000 7f68a504c640 1 --2- 172.21.15.157:0/1024956178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68a00c5640 0x7f68a0151ca0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:24.045 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.043+0000 7f68a504c640 1 --2- 172.21.15.157:0/1024956178 >> v2:172.21.15.169:3300/0 conn(0x7f68a0154e60 0x7f68a00c5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:24.045 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.043+0000 7f68a504c640 1 -- 172.21.15.157:0/1024956178 >> 172.21.15.157:0/1024956178 conn(0x7f68a00a8720 msgr2=0x7f68a00a8b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:24.046 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.043+0000 7f68a504c640 1 -- 172.21.15.157:0/1024956178 shutdown_connections 2024-04-24T23:35:24.046 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.043+0000 7f68a504c640 1 -- 172.21.15.157:0/1024956178 wait complete. 2024-04-24T23:35:24.046 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.044+0000 7f68a504c640 1 Processor -- start 2024-04-24T23:35:24.046 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.044+0000 7f68a504c640 1 -- start start 2024-04-24T23:35:24.048 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.045+0000 7f68a504c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68a00c5640 0x7f68a00a7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:24.048 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.045+0000 7f68a504c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68a014b950 0x7f68a0147dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:24.048 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.045+0000 7f68a504c640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68a00ac820 0x7f68a0148310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:24.048 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.045+0000 7f68a504c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f68a0155b90 con 0x7f68a00c5640 2024-04-24T23:35:24.048 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.045+0000 7f68a504c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f68a01558b0 con 0x7f68a014b950 2024-04-24T23:35:24.048 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.045+0000 7f68a504c640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f68a01521e0 con 0x7f68a00ac820 2024-04-24T23:35:24.049 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.045+0000 7f689ed9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68a00c5640 0x7f68a00a7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:24.049 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.045+0000 7f689ed9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68a00c5640 0x7f68a00a7d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40420/0 (socket says 172.21.15.157:40420) 2024-04-24T23:35:24.049 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.045+0000 7f689ed9d640 1 -- 172.21.15.157:0/3344548169 learned_addr learned my addr 172.21.15.157:0/3344548169 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:24.049 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.045+0000 7f6897fff640 1 --2- 172.21.15.157:0/3344548169 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68a014b950 0x7f68a0147dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:24.049 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.045+0000 7f689f59e640 1 --2- 172.21.15.157:0/3344548169 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68a00ac820 0x7f68a0148310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:24.049 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.045+0000 7f689ed9d640 1 -- 172.21.15.157:0/3344548169 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68a00ac820 msgr2=0x7f68a0148310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:24.049 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.045+0000 7f689ed9d640 1 --2- 172.21.15.157:0/3344548169 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68a00ac820 0x7f68a0148310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:24.049 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.045+0000 7f689ed9d640 1 -- 172.21.15.157:0/3344548169 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68a014b950 msgr2=0x7f68a0147dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:24.049 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.045+0000 7f689ed9d640 1 --2- 172.21.15.157:0/3344548169 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68a014b950 0x7f68a0147dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:24.049 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.045+0000 7f689ed9d640 1 -- 172.21.15.157:0/3344548169 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68a01eaa90 con 0x7f68a00c5640 2024-04-24T23:35:24.049 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.046+0000 7f689ed9d640 1 --2- 172.21.15.157:0/3344548169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68a00c5640 0x7f68a00a7d60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f689000c430 tx=0x7f689000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:24.049 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.046+0000 7f68977fe640 1 -- 172.21.15.157:0/3344548169 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6890017070 con 0x7f68a00c5640 2024-04-24T23:35:24.049 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.046+0000 7f68977fe640 1 -- 172.21.15.157:0/3344548169 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68900052d0 con 0x7f68a00c5640 2024-04-24T23:35:24.049 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.046+0000 7f68a504c640 1 -- 172.21.15.157:0/3344548169 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68a00aa760 con 0x7f68a00c5640 2024-04-24T23:35:24.050 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.046+0000 7f68977fe640 1 -- 172.21.15.157:0/3344548169 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6890003850 con 0x7f68a00c5640 2024-04-24T23:35:24.050 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.046+0000 7f68a504c640 1 -- 172.21.15.157:0/3344548169 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68a00a82a0 con 0x7f68a00c5640 2024-04-24T23:35:24.050 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.047+0000 7f68a504c640 1 -- 172.21.15.157:0/3344548169 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68a014f220 con 0x7f68a00c5640 2024-04-24T23:35:24.050 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.047+0000 7f68977fe640 1 -- 172.21.15.157:0/3344548169 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f689001a070 con 0x7f68a00c5640 2024-04-24T23:35:24.050 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.047+0000 7f68977fe640 1 --2- 172.21.15.157:0/3344548169 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f688006be60 0x7f688006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:24.050 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.048+0000 7f68977fe640 1 -- 172.21.15.157:0/3344548169 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f689008a480 con 0x7f68a00c5640 2024-04-24T23:35:24.050 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.048+0000 7f6897fff640 1 --2- 172.21.15.157:0/3344548169 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f688006be60 0x7f688006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:24.050 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.048+0000 7f6897fff640 1 --2- 172.21.15.157:0/3344548169 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f688006be60 0x7f688006e320 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f68a01e8520 tx=0x7f6888008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:24.052 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.052+0000 7f68977fe640 1 -- 172.21.15.157:0/3344548169 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6890058b80 con 0x7f68a00c5640 2024-04-24T23:35:24.262 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.261+0000 7f68a504c640 1 -- 172.21.15.157:0/3344548169 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f68a00c6620 con 0x7f68a00c5640 2024-04-24T23:35:24.262 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.262+0000 7f68977fe640 1 -- 172.21.15.157:0/3344548169 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5189 (secure 0 0 0) 0x7f6890058480 con 0x7f68a00c5640 2024-04-24T23:35:24.263 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:35:24.263 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1276438765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1276438765},{"type":"v1","addr":"172.21.15.169:6837","nonce":1276438765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1398599086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":1398599086},{"type":"v1","addr":"172.21.15.157:6835","nonce":1398599086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4258,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3795840339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3795840339},{"type":"v1","addr":"172.21.15.157:6834","nonce":3795840339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:34:53.701085+0000","modified":"2024-04-24T23:35:15.789166+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6836/3782746080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6835","nonce":3782746080},{"type":"v1","addr":"172.21.15.169:6836","nonce":3782746080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4270,"qdb_cluster":[4270]},"id":1}]} 2024-04-24T23:35:24.263 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 9 2024-04-24T23:35:24.265 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.264+0000 7f68a504c640 1 -- 172.21.15.157:0/3344548169 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f688006be60 msgr2=0x7f688006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:24.265 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.264+0000 7f68a504c640 1 --2- 172.21.15.157:0/3344548169 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f688006be60 0x7f688006e320 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f68a01e8520 tx=0x7f6888008040 comp rx=0 tx=0).stop 2024-04-24T23:35:24.265 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.264+0000 7f68a504c640 1 -- 172.21.15.157:0/3344548169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68a00c5640 msgr2=0x7f68a00a7d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:24.265 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.264+0000 7f68a504c640 1 --2- 172.21.15.157:0/3344548169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68a00c5640 0x7f68a00a7d60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f689000c430 tx=0x7f689000c8f0 comp rx=0 tx=0).stop 2024-04-24T23:35:24.265 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.264+0000 7f68a504c640 1 -- 172.21.15.157:0/3344548169 shutdown_connections 2024-04-24T23:35:24.265 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.264+0000 7f68a504c640 1 --2- 172.21.15.157:0/3344548169 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f688006be60 0x7f688006e320 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:24.265 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.264+0000 7f68a504c640 1 --2- 172.21.15.157:0/3344548169 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68a00ac820 0x7f68a0148310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:24.265 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.264+0000 7f68a504c640 1 --2- 172.21.15.157:0/3344548169 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68a014b950 0x7f68a0147dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:24.266 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.264+0000 7f68a504c640 1 --2- 172.21.15.157:0/3344548169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68a00c5640 0x7f68a00a7d60 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:24.266 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.264+0000 7f68a504c640 1 -- 172.21.15.157:0/3344548169 >> 172.21.15.157:0/3344548169 conn(0x7f68a00a8720 msgr2=0x7f68a0153650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:24.266 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.264+0000 7f68a504c640 1 -- 172.21.15.157:0/3344548169 shutdown_connections 2024-04-24T23:35:24.266 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.264+0000 7f68a504c640 1 -- 172.21.15.157:0/3344548169 wait complete. 2024-04-24T23:35:24.276 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:35:24.372 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.372+0000 7f64229e7640 1 Processor -- start 2024-04-24T23:35:24.372 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.372+0000 7f64229e7640 1 -- start start 2024-04-24T23:35:24.373 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.372+0000 7f64229e7640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f641c0be360 0x7f641c155350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:24.373 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.372+0000 7f64229e7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f641c155890 0x7f641c15fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:24.374 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.372+0000 7f64229e7640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f641c0c8bb0 con 0x7f641c152b40 2024-04-24T23:35:24.374 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.372+0000 7f64229e7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f641c0c8e90 con 0x7f641c155890 2024-04-24T23:35:24.374 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.372+0000 7f64229e7640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f641c159480 con 0x7f641c0be360 2024-04-24T23:35:24.374 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.372+0000 7f6420f84640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f641c155890 0x7f641c15fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:24.374 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.372+0000 7f641bfff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f641c152b40 0x7f641c0c5630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49086/0 (socket says 172.21.15.157:49086) 2024-04-24T23:35:24.374 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.373+0000 7f641bfff640 1 -- 172.21.15.157:0/731169553 learned_addr learned my addr 172.21.15.157:0/731169553 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-24T23:35:24.374 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.373+0000 7f6420f84640 1 --2- 172.21.15.157:0/731169553 >> v2:172.21.15.157:3300/0 conn(0x7f641c155890 0x7f641c15fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40428/0 (socket says 172.21.15.157:40428) 2024-04-24T23:35:24.374 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.373+0000 7f641b7fe640 1 --2- 172.21.15.157:0/731169553 >> v2:172.21.15.169:3300/0 conn(0x7f641c0be360 0x7f641c155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:24.374 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.373+0000 7f6420f84640 1 -- 172.21.15.157:0/731169553 >> v2:172.21.15.169:3300/0 conn(0x7f641c0be360 msgr2=0x7f641c155350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:24.374 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.373+0000 7f6420f84640 1 --2- 172.21.15.157:0/731169553 >> v2:172.21.15.169:3300/0 conn(0x7f641c0be360 0x7f641c155350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:24.374 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.373+0000 7f6420f84640 1 -- 172.21.15.157:0/731169553 >> v1:172.21.15.157:6789/0 conn(0x7f641c152b40 legacy=0x7f641c0c5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:24.374 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.373+0000 7f6420f84640 1 -- 172.21.15.157:0/731169553 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f641c159760 con 0x7f641c155890 2024-04-24T23:35:24.375 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.373+0000 7f641affd640 1 -- 172.21.15.157:0/731169553 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2237153028 0 0) 0x7f641c0c8bb0 con 0x7f641c152b40 2024-04-24T23:35:24.376 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.373+0000 7f6420f84640 1 --2- 172.21.15.157:0/731169553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f641c155890 0x7f641c15fcd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f641c06d120 tx=0x7f641402a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdcbc661c36ea044 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:24.376 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.374+0000 7f641affd640 1 -- 172.21.15.157:0/731169553 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6414007740 con 0x7f641c155890 2024-04-24T23:35:24.376 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.374+0000 7f641affd640 1 -- 172.21.15.157:0/731169553 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64140078e0 con 0x7f641c155890 2024-04-24T23:35:24.376 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.374+0000 7f641affd640 1 -- 172.21.15.157:0/731169553 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6414007cb0 con 0x7f641c155890 2024-04-24T23:35:24.376 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.374+0000 7f64229e7640 1 -- 172.21.15.157:0/731169553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f641c155890 msgr2=0x7f641c15fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:24.376 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.374+0000 7f64229e7640 1 --2- 172.21.15.157:0/731169553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f641c155890 0x7f641c15fcd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f641c06d120 tx=0x7f641402a560 comp rx=0 tx=0).stop 2024-04-24T23:35:24.376 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.374+0000 7f64229e7640 1 -- 172.21.15.157:0/731169553 shutdown_connections 2024-04-24T23:35:24.376 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.374+0000 7f64229e7640 1 --2- 172.21.15.157:0/731169553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f641c155890 0x7f641c15fcd0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:24.376 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.374+0000 7f64229e7640 1 --2- 172.21.15.157:0/731169553 >> v2:172.21.15.169:3300/0 conn(0x7f641c0be360 0x7f641c155350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:24.377 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.374+0000 7f64229e7640 1 -- 172.21.15.157:0/731169553 >> 172.21.15.157:0/731169553 conn(0x7f641c0672c0 msgr2=0x7f641c0676d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:24.377 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.374+0000 7f64229e7640 1 -- 172.21.15.157:0/731169553 shutdown_connections 2024-04-24T23:35:24.377 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.374+0000 7f64229e7640 1 -- 172.21.15.157:0/731169553 wait complete. 2024-04-24T23:35:24.377 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.375+0000 7f64229e7640 1 Processor -- start 2024-04-24T23:35:24.377 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.375+0000 7f64229e7640 1 -- start start 2024-04-24T23:35:24.379 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.376+0000 7f64229e7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f641c0be360 0x7f641c1ee9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:24.379 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.376+0000 7f64229e7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f641c152b40 0x7f641c1497a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:24.379 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.376+0000 7f64229e7640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f641c155890 0x7f641c149ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:24.380 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.376+0000 7f64229e7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f641c0caa00 con 0x7f641c0be360 2024-04-24T23:35:24.380 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.376+0000 7f64229e7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f641c0cb210 con 0x7f641c152b40 2024-04-24T23:35:24.380 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.376+0000 7f64229e7640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f641c155d00 con 0x7f641c155890 2024-04-24T23:35:24.380 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.376+0000 7f641bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f641c0be360 0x7f641c1ee9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:24.380 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.376+0000 7f641bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f641c0be360 0x7f641c1ee9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40430/0 (socket says 172.21.15.157:40430) 2024-04-24T23:35:24.380 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.376+0000 7f641bfff640 1 -- 172.21.15.157:0/79017767 learned_addr learned my addr 172.21.15.157:0/79017767 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:24.380 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.376+0000 7f641b7fe640 1 --2- 172.21.15.157:0/79017767 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f641c152b40 0x7f641c1497a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:24.380 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.376+0000 7f6420f84640 1 --2- 172.21.15.157:0/79017767 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f641c155890 0x7f641c149ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:24.380 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.376+0000 7f641bfff640 1 -- 172.21.15.157:0/79017767 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f641c155890 msgr2=0x7f641c149ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:24.380 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.376+0000 7f641bfff640 1 --2- 172.21.15.157:0/79017767 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f641c155890 0x7f641c149ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:24.380 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.376+0000 7f641bfff640 1 -- 172.21.15.157:0/79017767 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f641c152b40 msgr2=0x7f641c1497a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:24.380 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.376+0000 7f641bfff640 1 --2- 172.21.15.157:0/79017767 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f641c152b40 0x7f641c1497a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:24.380 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.376+0000 7f641bfff640 1 -- 172.21.15.157:0/79017767 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f641c153150 con 0x7f641c0be360 2024-04-24T23:35:24.381 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.377+0000 7f6420f84640 1 --2- 172.21.15.157:0/79017767 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f641c155890 0x7f641c149ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:35:24.381 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.377+0000 7f641bfff640 1 --2- 172.21.15.157:0/79017767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f641c0be360 0x7f641c1ee9a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f640400c4d0 tx=0x7f640400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:24.381 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.377+0000 7f6418ff9640 1 -- 172.21.15.157:0/79017767 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6404018070 con 0x7f641c0be360 2024-04-24T23:35:24.381 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.377+0000 7f6418ff9640 1 -- 172.21.15.157:0/79017767 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6404007500 con 0x7f641c0be360 2024-04-24T23:35:24.381 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.377+0000 7f64229e7640 1 -- 172.21.15.157:0/79017767 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f641c14b650 con 0x7f641c0be360 2024-04-24T23:35:24.381 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.377+0000 7f6418ff9640 1 -- 172.21.15.157:0/79017767 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6404004a20 con 0x7f641c0be360 2024-04-24T23:35:24.381 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.377+0000 7f64229e7640 1 -- 172.21.15.157:0/79017767 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f641c1ef5c0 con 0x7f641c0be360 2024-04-24T23:35:24.381 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.378+0000 7f64229e7640 1 -- 172.21.15.157:0/79017767 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f641c148d30 con 0x7f641c0be360 2024-04-24T23:35:24.381 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.378+0000 7f6418ff9640 1 -- 172.21.15.157:0/79017767 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f640401b070 con 0x7f641c0be360 2024-04-24T23:35:24.381 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.379+0000 7f6418ff9640 1 --2- 172.21.15.157:0/79017767 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f63fc06be30 0x7f63fc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:24.381 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.379+0000 7f6418ff9640 1 -- 172.21.15.157:0/79017767 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f640408a3b0 con 0x7f641c0be360 2024-04-24T23:35:24.381 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.379+0000 7f641b7fe640 1 --2- 172.21.15.157:0/79017767 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f63fc06be30 0x7f63fc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:24.382 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.379+0000 7f641b7fe640 1 --2- 172.21.15.157:0/79017767 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f63fc06be30 0x7f63fc06e2f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f641c153c90 tx=0x7f640c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:24.383 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.383+0000 7f6418ff9640 1 -- 172.21.15.157:0/79017767 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6404058460 con 0x7f641c0be360 2024-04-24T23:35:24.557 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.557+0000 7f64229e7640 1 -- 172.21.15.157:0/79017767 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f641c159170 con 0x7f641c0be360 2024-04-24T23:35:24.559 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.558+0000 7f6418ff9640 1 -- 172.21.15.157:0/79017767 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18538 (secure 0 0 0) 0x7f641c159170 con 0x7f641c0be360 2024-04-24T23:35:24.559 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:35:24.559 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":25,"fsid":"223685fb-5b64-46a5-9857-3791c9c07df2","created":"2024-04-24T23:34:37.504202+0000","modified":"2024-04-24T23:34:58.744500+0000","last_up_change":"2024-04-24T23:34:47.584963+0000","last_in_change":"2024-04-24T23:34:39.632954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:34:47.796655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:34:52.177208+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:34:52.667022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:34:56.395517+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"95d14ac8-fccf-4261-be8e-edafa746f629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6809","nonce":4003822160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6811","nonce":4003822160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6815","nonce":4003822160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6813","nonce":4003822160}]},"public_addr":"172.21.15.157:6809/4003822160","cluster_addr":"172.21.15.157:6811/4003822160","heartbeat_back_addr":"172.21.15.157:6815/4003822160","heartbeat_front_addr":"172.21.15.157:6813/4003822160","state":["exists","up"]},{"osd":1,"uuid":"7be79da9-8eec-4990-9178-71d602e39d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6817","nonce":1557575576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6819","nonce":1557575576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6823","nonce":1557575576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6821","nonce":1557575576}]},"public_addr":"172.21.15.157:6817/1557575576","cluster_addr":"172.21.15.157:6819/1557575576","heartbeat_back_addr":"172.21.15.157:6823/1557575576","heartbeat_front_addr":"172.21.15.157:6821/1557575576","state":["exists","up"]},{"osd":2,"uuid":"0127ed00-b7c3-466d-bfb0-01b0fd315f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6825","nonce":2793948092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6827","nonce":2793948092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6831","nonce":2793948092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6829","nonce":2793948092}]},"public_addr":"172.21.15.157:6825/2793948092","cluster_addr":"172.21.15.157:6827/2793948092","heartbeat_back_addr":"172.21.15.157:6831/2793948092","heartbeat_front_addr":"172.21.15.157:6829/2793948092","state":["exists","up"]},{"osd":3,"uuid":"f2331acc-aa37-433e-868e-3561b6039562","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6801","nonce":422758549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6803","nonce":422758549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6807","nonce":422758549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6805","nonce":422758549}]},"public_addr":"172.21.15.157:6801/422758549","cluster_addr":"172.21.15.157:6803/422758549","heartbeat_back_addr":"172.21.15.157:6807/422758549","heartbeat_front_addr":"172.21.15.157:6805/422758549","state":["exists","up"]},{"osd":4,"uuid":"1998854b-7398-43f5-ab7d-79fa0debcf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6801","nonce":321910099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6803","nonce":321910099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6807","nonce":321910099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6805","nonce":321910099}]},"public_addr":"172.21.15.169:6801/321910099","cluster_addr":"172.21.15.169:6803/321910099","heartbeat_back_addr":"172.21.15.169:6807/321910099","heartbeat_front_addr":"172.21.15.169:6805/321910099","state":["exists","up"]},{"osd":5,"uuid":"c37153ff-9b2b-48b9-a684-2c82b6f39f15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6825","nonce":243321711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6827","nonce":243321711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6831","nonce":243321711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6829","nonce":243321711}]},"public_addr":"172.21.15.169:6825/243321711","cluster_addr":"172.21.15.169:6827/243321711","heartbeat_back_addr":"172.21.15.169:6831/243321711","heartbeat_front_addr":"172.21.15.169:6829/243321711","state":["exists","up"]},{"osd":6,"uuid":"8db79e1c-f36b-49cb-8d8a-a20c2aec57c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6809","nonce":3441420785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6811","nonce":3441420785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6815","nonce":3441420785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6813","nonce":3441420785}]},"public_addr":"172.21.15.169:6809/3441420785","cluster_addr":"172.21.15.169:6811/3441420785","heartbeat_back_addr":"172.21.15.169:6815/3441420785","heartbeat_front_addr":"172.21.15.169:6813/3441420785","state":["exists","up"]},{"osd":7,"uuid":"928586e9-296c-4b8b-9b27-42f9f155dbdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6817","nonce":1663179020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6819","nonce":1663179020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6823","nonce":1663179020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6821","nonce":1663179020}]},"public_addr":"172.21.15.169:6817/1663179020","cluster_addr":"172.21.15.169:6819/1663179020","heartbeat_back_addr":"172.21.15.169:6823/1663179020","heartbeat_front_addr":"172.21.15.169:6821/1663179020","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.848039+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.826489+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.801120+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.727706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.824343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.791915+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.878073+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.810831+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:35:24.561 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.560+0000 7f64229e7640 1 -- 172.21.15.157:0/79017767 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f63fc06be30 msgr2=0x7f63fc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:24.561 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.560+0000 7f64229e7640 1 --2- 172.21.15.157:0/79017767 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f63fc06be30 0x7f63fc06e2f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f641c153c90 tx=0x7f640c008040 comp rx=0 tx=0).stop 2024-04-24T23:35:24.561 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.560+0000 7f64229e7640 1 -- 172.21.15.157:0/79017767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f641c0be360 msgr2=0x7f641c1ee9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:24.561 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.560+0000 7f64229e7640 1 --2- 172.21.15.157:0/79017767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f641c0be360 0x7f641c1ee9a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f640400c4d0 tx=0x7f640400c990 comp rx=0 tx=0).stop 2024-04-24T23:35:24.561 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.560+0000 7f64229e7640 1 -- 172.21.15.157:0/79017767 shutdown_connections 2024-04-24T23:35:24.561 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.560+0000 7f64229e7640 1 --2- 172.21.15.157:0/79017767 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f63fc06be30 0x7f63fc06e2f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:24.561 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.560+0000 7f64229e7640 1 --2- 172.21.15.157:0/79017767 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f641c155890 0x7f641c149ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:24.562 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.560+0000 7f64229e7640 1 --2- 172.21.15.157:0/79017767 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f641c152b40 0x7f641c1497a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:24.562 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.560+0000 7f64229e7640 1 --2- 172.21.15.157:0/79017767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f641c0be360 0x7f641c1ee9a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:24.562 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.560+0000 7f64229e7640 1 -- 172.21.15.157:0/79017767 >> 172.21.15.157:0/79017767 conn(0x7f641c0672c0 msgr2=0x7f641c14c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:24.562 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.560+0000 7f64229e7640 1 -- 172.21.15.157:0/79017767 shutdown_connections 2024-04-24T23:35:24.562 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.560+0000 7f64229e7640 1 -- 172.21.15.157:0/79017767 wait complete. 2024-04-24T23:35:24.573 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:34:47.796655+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:34:52.177208+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:34:52.667022+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T23:34:56.395517+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T23:35:24.573 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:35:24.673 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.672+0000 7fe843ee7640 1 Processor -- start 2024-04-24T23:35:24.673 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.673+0000 7fe843ee7640 1 -- start start 2024-04-24T23:35:24.674 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.673+0000 7fe843ee7640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe83c150720 0x7fe83c150b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:24.674 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.673+0000 7fe843ee7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe83c153330 0x7fe83c1537c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:24.674 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.673+0000 7fe843ee7640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe83c06e240 con 0x7fe83c072c50 2024-04-24T23:35:24.674 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.673+0000 7fe843ee7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fe83c156a50 con 0x7fe83c153330 2024-04-24T23:35:24.675 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.673+0000 7fe843ee7640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe83c156cf0 con 0x7fe83c150720 2024-04-24T23:35:24.675 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.673+0000 7fe842484640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe83c153330 0x7fe83c1537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:24.675 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.673+0000 7fe841c83640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fe83c072c50 0x7fe83c06d830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49102/0 (socket says 172.21.15.157:49102) 2024-04-24T23:35:24.675 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.673+0000 7fe841c83640 1 -- 172.21.15.157:0/2217336275 learned_addr learned my addr 172.21.15.157:0/2217336275 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-24T23:35:24.675 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.674+0000 7fe841482640 1 --2- 172.21.15.157:0/2217336275 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe83c150720 0x7fe83c150b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:24.675 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.674+0000 7fe842484640 1 -- 172.21.15.157:0/2217336275 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe83c150720 msgr2=0x7fe83c150b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:24.675 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.674+0000 7fe842484640 1 --2- 172.21.15.157:0/2217336275 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe83c150720 0x7fe83c150b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:24.675 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.674+0000 7fe842484640 1 -- 172.21.15.157:0/2217336275 >> v1:172.21.15.157:6789/0 conn(0x7fe83c072c50 legacy=0x7fe83c06d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:24.675 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.674+0000 7fe842484640 1 -- 172.21.15.157:0/2217336275 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe83c156fd0 con 0x7fe83c153330 2024-04-24T23:35:24.677 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.674+0000 7fe842484640 1 --2- 172.21.15.157:0/2217336275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe83c153330 0x7fe83c1537c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe8340097a0 tx=0x7fe83402a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38122a8c8106eeed server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:24.677 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.674+0000 7fe840c81640 1 -- 172.21.15.157:0/2217336275 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe834037070 con 0x7fe83c153330 2024-04-24T23:35:24.677 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.674+0000 7fe840c81640 1 -- 172.21.15.157:0/2217336275 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe83402adb0 con 0x7fe83c153330 2024-04-24T23:35:24.677 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.675+0000 7fe840c81640 1 -- 172.21.15.157:0/2217336275 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe834032660 con 0x7fe83c153330 2024-04-24T23:35:24.677 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.675+0000 7fe843ee7640 1 -- 172.21.15.157:0/2217336275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe83c153330 msgr2=0x7fe83c1537c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:24.677 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.675+0000 7fe843ee7640 1 --2- 172.21.15.157:0/2217336275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe83c153330 0x7fe83c1537c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe8340097a0 tx=0x7fe83402a060 comp rx=0 tx=0).stop 2024-04-24T23:35:24.677 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.675+0000 7fe843ee7640 1 -- 172.21.15.157:0/2217336275 shutdown_connections 2024-04-24T23:35:24.677 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.675+0000 7fe843ee7640 1 --2- 172.21.15.157:0/2217336275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe83c153330 0x7fe83c1537c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:24.677 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.675+0000 7fe843ee7640 1 --2- 172.21.15.157:0/2217336275 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe83c150720 0x7fe83c150b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:24.677 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.675+0000 7fe843ee7640 1 -- 172.21.15.157:0/2217336275 >> 172.21.15.157:0/2217336275 conn(0x7fe83c0c68e0 msgr2=0x7fe83c0c6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:24.677 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.675+0000 7fe843ee7640 1 -- 172.21.15.157:0/2217336275 shutdown_connections 2024-04-24T23:35:24.677 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.675+0000 7fe843ee7640 1 -- 172.21.15.157:0/2217336275 wait complete. 2024-04-24T23:35:24.677 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.676+0000 7fe843ee7640 1 Processor -- start 2024-04-24T23:35:24.677 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.676+0000 7fe843ee7640 1 -- start start 2024-04-24T23:35:24.679 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.677+0000 7fe843ee7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe83c072c50 0x7fe83c0707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:24.679 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.677+0000 7fe843ee7640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe83c150720 0x7fe83c0aba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:24.679 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.677+0000 7fe843ee7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe83c153330 0x7fe83c0bff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:24.679 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.677+0000 7fe843ee7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe83c1577d0 con 0x7fe83c072c50 2024-04-24T23:35:24.679 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.677+0000 7fe843ee7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe83c06df40 con 0x7fe83c153330 2024-04-24T23:35:24.679 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.677+0000 7fe843ee7640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe83c153d00 con 0x7fe83c150720 2024-04-24T23:35:24.679 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.677+0000 7fe841c83640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe83c072c50 0x7fe83c0707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:24.680 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.677+0000 7fe841c83640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe83c072c50 0x7fe83c0707d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40452/0 (socket says 172.21.15.157:40452) 2024-04-24T23:35:24.680 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.677+0000 7fe841c83640 1 -- 172.21.15.157:0/422780571 learned_addr learned my addr 172.21.15.157:0/422780571 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:24.680 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.677+0000 7fe841482640 1 --2- 172.21.15.157:0/422780571 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe83c150720 0x7fe83c0aba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:24.680 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.677+0000 7fe842484640 1 --2- 172.21.15.157:0/422780571 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe83c153330 0x7fe83c0bff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:24.680 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.677+0000 7fe841c83640 1 -- 172.21.15.157:0/422780571 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe83c150720 msgr2=0x7fe83c0aba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:24.680 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.677+0000 7fe841c83640 1 --2- 172.21.15.157:0/422780571 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe83c150720 0x7fe83c0aba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:24.680 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.677+0000 7fe841c83640 1 -- 172.21.15.157:0/422780571 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe83c153330 msgr2=0x7fe83c0bff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:24.680 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.677+0000 7fe841c83640 1 --2- 172.21.15.157:0/422780571 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe83c153330 0x7fe83c0bff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:24.680 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.677+0000 7fe841c83640 1 -- 172.21.15.157:0/422780571 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe83c0aa9e0 con 0x7fe83c072c50 2024-04-24T23:35:24.680 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.678+0000 7fe841c83640 1 --2- 172.21.15.157:0/422780571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe83c072c50 0x7fe83c0707d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe83800ca70 tx=0x7fe83800cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:24.680 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.678+0000 7fe8327fc640 1 -- 172.21.15.157:0/422780571 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe838017070 con 0x7fe83c072c50 2024-04-24T23:35:24.680 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.678+0000 7fe8327fc640 1 -- 172.21.15.157:0/422780571 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe83800abd0 con 0x7fe83c072c50 2024-04-24T23:35:24.680 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.678+0000 7fe8327fc640 1 -- 172.21.15.157:0/422780571 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe838012400 con 0x7fe83c072c50 2024-04-24T23:35:24.680 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.678+0000 7fe843ee7640 1 -- 172.21.15.157:0/422780571 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe83c0aacc0 con 0x7fe83c072c50 2024-04-24T23:35:24.681 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.678+0000 7fe843ee7640 1 -- 172.21.15.157:0/422780571 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe83c0ab280 con 0x7fe83c072c50 2024-04-24T23:35:24.681 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.679+0000 7fe843ee7640 1 -- 172.21.15.157:0/422780571 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe83c0681c0 con 0x7fe83c072c50 2024-04-24T23:35:24.681 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.679+0000 7fe8327fc640 1 -- 172.21.15.157:0/422780571 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe83801a070 con 0x7fe83c072c50 2024-04-24T23:35:24.681 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.679+0000 7fe8327fc640 1 --2- 172.21.15.157:0/422780571 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fe82006be30 0x7fe82006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:24.682 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.680+0000 7fe8327fc640 1 -- 172.21.15.157:0/422780571 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe83808ade0 con 0x7fe83c072c50 2024-04-24T23:35:24.682 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.680+0000 7fe841482640 1 --2- 172.21.15.157:0/422780571 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fe82006be30 0x7fe82006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:24.682 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.680+0000 7fe841482640 1 --2- 172.21.15.157:0/422780571 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fe82006be30 0x7fe82006e2f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe83c1ebf50 tx=0x7fe82c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:24.684 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.684+0000 7fe8327fc640 1 -- 172.21.15.157:0/422780571 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe838058e00 con 0x7fe83c072c50 2024-04-24T23:35:24.859 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.858+0000 7fe843ee7640 1 -- 172.21.15.157:0/422780571 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe83c0abfc0 con 0x7fe83c072c50 2024-04-24T23:35:24.859 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.858+0000 7fe8327fc640 1 -- 172.21.15.157:0/422780571 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7fe83805d100 con 0x7fe83c072c50 2024-04-24T23:35:24.859 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2024-04-24T23:35:24.861 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.860+0000 7fe843ee7640 1 -- 172.21.15.157:0/422780571 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fe82006be30 msgr2=0x7fe82006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:24.861 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.860+0000 7fe843ee7640 1 --2- 172.21.15.157:0/422780571 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fe82006be30 0x7fe82006e2f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe83c1ebf50 tx=0x7fe82c008040 comp rx=0 tx=0).stop 2024-04-24T23:35:24.862 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.860+0000 7fe843ee7640 1 -- 172.21.15.157:0/422780571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe83c072c50 msgr2=0x7fe83c0707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:24.862 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.860+0000 7fe843ee7640 1 --2- 172.21.15.157:0/422780571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe83c072c50 0x7fe83c0707d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe83800ca70 tx=0x7fe83800cf30 comp rx=0 tx=0).stop 2024-04-24T23:35:24.862 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.860+0000 7fe843ee7640 1 -- 172.21.15.157:0/422780571 shutdown_connections 2024-04-24T23:35:24.862 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.860+0000 7fe843ee7640 1 --2- 172.21.15.157:0/422780571 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fe82006be30 0x7fe82006e2f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:24.862 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.860+0000 7fe843ee7640 1 --2- 172.21.15.157:0/422780571 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe83c153330 0x7fe83c0bff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:24.862 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.860+0000 7fe843ee7640 1 --2- 172.21.15.157:0/422780571 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe83c150720 0x7fe83c0aba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:24.862 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.860+0000 7fe843ee7640 1 --2- 172.21.15.157:0/422780571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe83c072c50 0x7fe83c0707d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:24.862 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.860+0000 7fe843ee7640 1 -- 172.21.15.157:0/422780571 >> 172.21.15.157:0/422780571 conn(0x7fe83c0c68e0 msgr2=0x7fe83c073030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:24.862 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.860+0000 7fe843ee7640 1 -- 172.21.15.157:0/422780571 shutdown_connections 2024-04-24T23:35:24.862 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.860+0000 7fe843ee7640 1 -- 172.21.15.157:0/422780571 wait complete. 2024-04-24T23:35:24.872 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:35:24.972 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.972+0000 7f137c6ee640 1 Processor -- start 2024-04-24T23:35:24.972 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.972+0000 7f137c6ee640 1 -- start start 2024-04-24T23:35:24.974 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.973+0000 7f137c6ee640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f13740a6090 0x7f13740c12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:24.974 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.973+0000 7f137c6ee640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f13740c1820 0x7f137406df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:24.974 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.973+0000 7f137c6ee640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f137406e480 con 0x7f13740c5740 2024-04-24T23:35:24.975 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.973+0000 7f137c6ee640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f13740a6470 con 0x7f13740c1820 2024-04-24T23:35:24.975 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.973+0000 7f137c6ee640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f13740c52b0 con 0x7f13740a6090 2024-04-24T23:35:24.975 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.973+0000 7f137ac8b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f13740c1820 0x7f137406df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:24.975 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.973+0000 7f137ac8b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f13740c1820 0x7f137406df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40466/0 (socket says 172.21.15.157:40466) 2024-04-24T23:35:24.975 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.973+0000 7f1379c89640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f13740c5740 0x7f137406d830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49118/0 (socket says 172.21.15.157:49118) 2024-04-24T23:35:24.975 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.973+0000 7f137ac8b640 1 -- 172.21.15.157:0/2873053767 learned_addr learned my addr 172.21.15.157:0/2873053767 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:24.975 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.973+0000 7f137a48a640 1 --2- 172.21.15.157:0/2873053767 >> v2:172.21.15.169:3300/0 conn(0x7f13740a6090 0x7f13740c12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:24.975 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.973+0000 7f137ac8b640 1 -- 172.21.15.157:0/2873053767 >> v2:172.21.15.169:3300/0 conn(0x7f13740a6090 msgr2=0x7f13740c12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:24.975 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.973+0000 7f137ac8b640 1 --2- 172.21.15.157:0/2873053767 >> v2:172.21.15.169:3300/0 conn(0x7f13740a6090 0x7f13740c12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:24.975 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.973+0000 7f137ac8b640 1 -- 172.21.15.157:0/2873053767 >> v1:172.21.15.157:6789/0 conn(0x7f13740c5740 legacy=0x7f137406d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:24.975 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.973+0000 7f137ac8b640 1 -- 172.21.15.157:0/2873053767 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f137406e830 con 0x7f13740c1820 2024-04-24T23:35:24.975 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.973+0000 7f1379488640 1 -- 172.21.15.157:0/2873053767 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3982975834 0 0) 0x7f137406e480 con 0x7f13740c5740 2024-04-24T23:35:24.975 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.974+0000 7f137ac8b640 1 --2- 172.21.15.157:0/2873053767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13740c1820 0x7f137406df40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f13740c33b0 tx=0x7f136c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cad19b83e32ee0b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:24.975 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.974+0000 7f1379488640 1 -- 172.21.15.157:0/2873053767 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136c036070 con 0x7f13740c1820 2024-04-24T23:35:24.976 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.974+0000 7f1379488640 1 -- 172.21.15.157:0/2873053767 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f136c031600 con 0x7f13740c1820 2024-04-24T23:35:24.976 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.974+0000 7f1379488640 1 -- 172.21.15.157:0/2873053767 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136c0319d0 con 0x7f13740c1820 2024-04-24T23:35:24.978 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.974+0000 7f137c6ee640 1 -- 172.21.15.157:0/2873053767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13740c1820 msgr2=0x7f137406df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:24.978 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.974+0000 7f137c6ee640 1 --2- 172.21.15.157:0/2873053767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13740c1820 0x7f137406df40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f13740c33b0 tx=0x7f136c02a6a0 comp rx=0 tx=0).stop 2024-04-24T23:35:24.978 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.974+0000 7f137c6ee640 1 -- 172.21.15.157:0/2873053767 shutdown_connections 2024-04-24T23:35:24.979 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.974+0000 7f137c6ee640 1 --2- 172.21.15.157:0/2873053767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13740c1820 0x7f137406df40 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:24.979 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.974+0000 7f137c6ee640 1 --2- 172.21.15.157:0/2873053767 >> v2:172.21.15.169:3300/0 conn(0x7f13740a6090 0x7f13740c12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:24.979 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.974+0000 7f137c6ee640 1 -- 172.21.15.157:0/2873053767 >> 172.21.15.157:0/2873053767 conn(0x7f13740c68e0 msgr2=0x7f13740c6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:24.979 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.975+0000 7f137c6ee640 1 -- 172.21.15.157:0/2873053767 shutdown_connections 2024-04-24T23:35:24.979 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.975+0000 7f137c6ee640 1 -- 172.21.15.157:0/2873053767 wait complete. 2024-04-24T23:35:24.979 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.976+0000 7f137c6ee640 1 Processor -- start 2024-04-24T23:35:24.979 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.976+0000 7f137c6ee640 1 -- start start 2024-04-24T23:35:24.979 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.976+0000 7f137c6ee640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13740a6090 0x7f1374154580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:24.979 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.976+0000 7f137c6ee640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13740c1820 0x7f13741dd480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:24.979 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.976+0000 7f137c6ee640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13740c5740 0x7f13741dd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:24.979 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.976+0000 7f137c6ee640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f13740c4f40 con 0x7f13740c5740 2024-04-24T23:35:24.980 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.976+0000 7f137c6ee640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f13740c4c60 con 0x7f13740c1820 2024-04-24T23:35:24.980 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.976+0000 7f137c6ee640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f13740c65a0 con 0x7f13740a6090 2024-04-24T23:35:24.980 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.976+0000 7f137ac8b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13740c5740 0x7f13741dd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:24.980 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.976+0000 7f137ac8b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13740c5740 0x7f13741dd9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40478/0 (socket says 172.21.15.157:40478) 2024-04-24T23:35:24.980 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.976+0000 7f137ac8b640 1 -- 172.21.15.157:0/446684690 learned_addr learned my addr 172.21.15.157:0/446684690 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:24.980 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.977+0000 7f1379c89640 1 --2- 172.21.15.157:0/446684690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13740c1820 0x7f13741dd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:24.980 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.977+0000 7f137ac8b640 1 -- 172.21.15.157:0/446684690 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13740a6090 msgr2=0x7f1374154580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:24.980 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.977+0000 7f137a48a640 1 --2- 172.21.15.157:0/446684690 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13740a6090 0x7f1374154580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:24.980 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.977+0000 7f137ac8b640 1 --2- 172.21.15.157:0/446684690 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13740a6090 0x7f1374154580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:24.980 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.977+0000 7f137ac8b640 1 -- 172.21.15.157:0/446684690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13740c1820 msgr2=0x7f13741dd480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:24.980 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.977+0000 7f137ac8b640 1 --2- 172.21.15.157:0/446684690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13740c1820 0x7f13741dd480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:24.980 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.977+0000 7f137ac8b640 1 -- 172.21.15.157:0/446684690 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13741ddf00 con 0x7f13740c5740 2024-04-24T23:35:24.980 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.977+0000 7f137ac8b640 1 --2- 172.21.15.157:0/446684690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13740c5740 0x7f13741dd9c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f136c034040 tx=0x7f136c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:24.980 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.977+0000 7f136affd640 1 -- 172.21.15.157:0/446684690 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136c036050 con 0x7f13740c5740 2024-04-24T23:35:24.980 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.977+0000 7f136affd640 1 -- 172.21.15.157:0/446684690 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f136c042070 con 0x7f13740c5740 2024-04-24T23:35:24.981 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.977+0000 7f136affd640 1 -- 172.21.15.157:0/446684690 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136c009510 con 0x7f13740c5740 2024-04-24T23:35:24.981 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.977+0000 7f137c6ee640 1 -- 172.21.15.157:0/446684690 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13741dcfa0 con 0x7f13740c5740 2024-04-24T23:35:24.981 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.977+0000 7f137c6ee640 1 -- 172.21.15.157:0/446684690 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1374146fc0 con 0x7f13740c5740 2024-04-24T23:35:24.981 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.978+0000 7f137c6ee640 1 -- 172.21.15.157:0/446684690 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13740c4250 con 0x7f13740c5740 2024-04-24T23:35:24.981 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.978+0000 7f136affd640 1 -- 172.21.15.157:0/446684690 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f136c007bd0 con 0x7f13740c5740 2024-04-24T23:35:24.982 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.979+0000 7f136affd640 1 --2- 172.21.15.157:0/446684690 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f135806be30 0x7f135806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:24.982 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.979+0000 7f136affd640 1 -- 172.21.15.157:0/446684690 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f136c0ac080 con 0x7f13740c5740 2024-04-24T23:35:24.982 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.979+0000 7f137a48a640 1 --2- 172.21.15.157:0/446684690 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f135806be30 0x7f135806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:24.982 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.980+0000 7f137a48a640 1 --2- 172.21.15.157:0/446684690 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f135806be30 0x7f135806e2f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1370006fd0 tx=0x7f1370008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:24.983 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:24.983+0000 7f136affd640 1 -- 172.21.15.157:0/446684690 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f136c07a130 con 0x7f13740c5740 2024-04-24T23:35:25.156 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.156+0000 7f137c6ee640 1 -- 172.21.15.157:0/446684690 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f13740c1150 con 0x7f13740c5740 2024-04-24T23:35:25.157 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.156+0000 7f136affd640 1 -- 172.21.15.157:0/446684690 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f136c07e430 con 0x7f13740c5740 2024-04-24T23:35:25.157 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2024-04-24T23:35:25.159 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.158+0000 7f137c6ee640 1 -- 172.21.15.157:0/446684690 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f135806be30 msgr2=0x7f135806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:25.159 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.158+0000 7f137c6ee640 1 --2- 172.21.15.157:0/446684690 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f135806be30 0x7f135806e2f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1370006fd0 tx=0x7f1370008040 comp rx=0 tx=0).stop 2024-04-24T23:35:25.159 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.158+0000 7f137c6ee640 1 -- 172.21.15.157:0/446684690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13740c5740 msgr2=0x7f13741dd9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:25.159 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.158+0000 7f137c6ee640 1 --2- 172.21.15.157:0/446684690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13740c5740 0x7f13741dd9c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f136c034040 tx=0x7f136c004a40 comp rx=0 tx=0).stop 2024-04-24T23:35:25.159 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.158+0000 7f137c6ee640 1 -- 172.21.15.157:0/446684690 shutdown_connections 2024-04-24T23:35:25.159 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.158+0000 7f137c6ee640 1 --2- 172.21.15.157:0/446684690 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f135806be30 0x7f135806e2f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:25.160 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.158+0000 7f137c6ee640 1 --2- 172.21.15.157:0/446684690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13740c5740 0x7f13741dd9c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:25.160 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.158+0000 7f137c6ee640 1 --2- 172.21.15.157:0/446684690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13740c1820 0x7f13741dd480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:25.160 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.158+0000 7f137c6ee640 1 --2- 172.21.15.157:0/446684690 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13740a6090 0x7f1374154580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:25.160 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.158+0000 7f137c6ee640 1 -- 172.21.15.157:0/446684690 >> 172.21.15.157:0/446684690 conn(0x7f13740c68e0 msgr2=0x7f1374072860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:25.160 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.158+0000 7f137c6ee640 1 -- 172.21.15.157:0/446684690 shutdown_connections 2024-04-24T23:35:25.160 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.158+0000 7f137c6ee640 1 -- 172.21.15.157:0/446684690 wait complete. 2024-04-24T23:35:25.170 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:35:25.269 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.268+0000 7f8760b12640 1 Processor -- start 2024-04-24T23:35:25.269 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.269+0000 7f8760b12640 1 -- start start 2024-04-24T23:35:25.270 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.269+0000 7f8760b12640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f875c0a5c50 0x7f875c156390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:25.270 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.269+0000 7f8760b12640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f875c152cc0 0x7f875c0c3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:25.271 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.269+0000 7f8760b12640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f875c1530f0 con 0x7f875c0c3370 2024-04-24T23:35:25.271 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.269+0000 7f8760b12640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f875c155f00 con 0x7f875c152cc0 2024-04-24T23:35:25.271 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.269+0000 7f8760b12640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f875c0736b0 con 0x7f875c0a5c50 2024-04-24T23:35:25.271 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.269+0000 7f875ad9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f875c152cc0 0x7f875c0c3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:25.271 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.269+0000 7f875ad9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f875c152cc0 0x7f875c0c3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40492/0 (socket says 172.21.15.157:40492) 2024-04-24T23:35:25.271 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.269+0000 7f875ad9d640 1 -- 172.21.15.157:0/2992278337 learned_addr learned my addr 172.21.15.157:0/2992278337 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:25.271 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.270+0000 7f875ad9d640 1 -- 172.21.15.157:0/2992278337 >> v2:172.21.15.169:3300/0 conn(0x7f875c0a5c50 msgr2=0x7f875c156390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:25.271 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.270+0000 7f8753fff640 1 --2- 172.21.15.157:0/2992278337 >> v2:172.21.15.169:3300/0 conn(0x7f875c0a5c50 0x7f875c156390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:25.272 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.270+0000 7f875ad9d640 1 --2- 172.21.15.157:0/2992278337 >> v2:172.21.15.169:3300/0 conn(0x7f875c0a5c50 0x7f875c156390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:25.272 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.270+0000 7f875ad9d640 1 -- 172.21.15.157:0/2992278337 >> v1:172.21.15.169:6789/0 conn(0x7f875c0c3370 legacy=0x7f875c0c12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:25.272 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.270+0000 7f875ad9d640 1 -- 172.21.15.157:0/2992278337 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f875c0a60e0 con 0x7f875c152cc0 2024-04-24T23:35:25.272 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.270+0000 7f8753fff640 1 --2- 172.21.15.157:0/2992278337 >> v2:172.21.15.169:3300/0 conn(0x7f875c0a5c50 0x7f875c156390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:35:25.276 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.270+0000 7f875ad9d640 1 --2- 172.21.15.157:0/2992278337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f875c152cc0 0x7f875c0c3750 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f875c1557e0 tx=0x7f875402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abc57547b21c897 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:25.276 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.271+0000 7f8759d9b640 1 -- 172.21.15.157:0/2992278337 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8754036070 con 0x7f875c152cc0 2024-04-24T23:35:25.276 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.271+0000 7f8759d9b640 1 -- 172.21.15.157:0/2992278337 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8754031600 con 0x7f875c152cc0 2024-04-24T23:35:25.276 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.271+0000 7f8759d9b640 1 -- 172.21.15.157:0/2992278337 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87540319d0 con 0x7f875c152cc0 2024-04-24T23:35:25.276 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.271+0000 7f8760b12640 1 -- 172.21.15.157:0/2992278337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f875c152cc0 msgr2=0x7f875c0c3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:25.276 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.271+0000 7f8760b12640 1 --2- 172.21.15.157:0/2992278337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f875c152cc0 0x7f875c0c3750 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f875c1557e0 tx=0x7f875402a6f0 comp rx=0 tx=0).stop 2024-04-24T23:35:25.276 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.271+0000 7f8760b12640 1 -- 172.21.15.157:0/2992278337 shutdown_connections 2024-04-24T23:35:25.277 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.271+0000 7f8760b12640 1 --2- 172.21.15.157:0/2992278337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f875c152cc0 0x7f875c0c3750 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:25.277 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.271+0000 7f8760b12640 1 --2- 172.21.15.157:0/2992278337 >> v2:172.21.15.169:3300/0 conn(0x7f875c0a5c50 0x7f875c156390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:25.277 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.271+0000 7f8760b12640 1 -- 172.21.15.157:0/2992278337 >> 172.21.15.157:0/2992278337 conn(0x7f875c0c68e0 msgr2=0x7f875c0c6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:25.277 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.271+0000 7f8760b12640 1 -- 172.21.15.157:0/2992278337 shutdown_connections 2024-04-24T23:35:25.277 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.271+0000 7f8760b12640 1 -- 172.21.15.157:0/2992278337 wait complete. 2024-04-24T23:35:25.277 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.272+0000 7f8760b12640 1 Processor -- start 2024-04-24T23:35:25.277 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.272+0000 7f8760b12640 1 -- start start 2024-04-24T23:35:25.277 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.273+0000 7f8760b12640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f875c0c3370 0x7f875c149280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:25.277 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.273+0000 7f8760b12640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f875c152cc0 0x7f875c145900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:25.277 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.273+0000 7f8760b12640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f875c145e40 0x7f875c1deac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:25.277 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.273+0000 7f8760b12640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f875c155b90 con 0x7f875c145e40 2024-04-24T23:35:25.277 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.273+0000 7f8760b12640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f875c1558b0 con 0x7f875c0c3370 2024-04-24T23:35:25.277 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.273+0000 7f8760b12640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f875c0c19f0 con 0x7f875c152cc0 2024-04-24T23:35:25.278 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.273+0000 7f875ad9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f875c145e40 0x7f875c1deac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:25.278 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.273+0000 7f875ad9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f875c145e40 0x7f875c1deac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40502/0 (socket says 172.21.15.157:40502) 2024-04-24T23:35:25.278 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.273+0000 7f875ad9d640 1 -- 172.21.15.157:0/2094154064 learned_addr learned my addr 172.21.15.157:0/2094154064 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:25.278 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.273+0000 7f875a59c640 1 --2- 172.21.15.157:0/2094154064 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f875c0c3370 0x7f875c149280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:25.278 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.273+0000 7f8753fff640 1 --2- 172.21.15.157:0/2094154064 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f875c152cc0 0x7f875c145900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:25.278 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.273+0000 7f875ad9d640 1 -- 172.21.15.157:0/2094154064 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f875c152cc0 msgr2=0x7f875c145900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:25.278 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.273+0000 7f875ad9d640 1 --2- 172.21.15.157:0/2094154064 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f875c152cc0 0x7f875c145900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:25.278 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.273+0000 7f875ad9d640 1 -- 172.21.15.157:0/2094154064 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f875c0c3370 msgr2=0x7f875c149280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:25.278 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.273+0000 7f875ad9d640 1 --2- 172.21.15.157:0/2094154064 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f875c0c3370 0x7f875c149280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:25.279 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.273+0000 7f875ad9d640 1 -- 172.21.15.157:0/2094154064 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f875c1e7cd0 con 0x7f875c145e40 2024-04-24T23:35:25.279 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.274+0000 7f875ad9d640 1 --2- 172.21.15.157:0/2094154064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f875c145e40 0x7f875c1deac0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8754034040 tx=0x7f8754004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:25.279 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.274+0000 7f8752ffd640 1 -- 172.21.15.157:0/2094154064 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8754036050 con 0x7f875c145e40 2024-04-24T23:35:25.279 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.274+0000 7f8752ffd640 1 -- 172.21.15.157:0/2094154064 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8754042070 con 0x7f875c145e40 2024-04-24T23:35:25.279 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.274+0000 7f8760b12640 1 -- 172.21.15.157:0/2094154064 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f875c1e7fb0 con 0x7f875c145e40 2024-04-24T23:35:25.279 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.274+0000 7f8752ffd640 1 -- 172.21.15.157:0/2094154064 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8754009510 con 0x7f875c145e40 2024-04-24T23:35:25.279 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.274+0000 7f8760b12640 1 -- 172.21.15.157:0/2094154064 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f875c1e8570 con 0x7f875c145e40 2024-04-24T23:35:25.279 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.275+0000 7f8760b12640 1 -- 172.21.15.157:0/2094154064 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f875c0681c0 con 0x7f875c145e40 2024-04-24T23:35:25.279 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.275+0000 7f8752ffd640 1 -- 172.21.15.157:0/2094154064 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8754007bf0 con 0x7f875c145e40 2024-04-24T23:35:25.279 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.275+0000 7f8752ffd640 1 --2- 172.21.15.157:0/2094154064 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f873c06be30 0x7f873c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:25.279 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.276+0000 7f8752ffd640 1 -- 172.21.15.157:0/2094154064 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f87540abf60 con 0x7f875c145e40 2024-04-24T23:35:25.279 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.276+0000 7f875a59c640 1 --2- 172.21.15.157:0/2094154064 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f873c06be30 0x7f873c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:25.280 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.276+0000 7f875a59c640 1 --2- 172.21.15.157:0/2094154064 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f873c06be30 0x7f873c06e2f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f875c1519f0 tx=0x7f874c007e30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:25.280 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.280+0000 7f8752ffd640 1 -- 172.21.15.157:0/2094154064 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f875c1530f0 con 0x7f875c145e40 2024-04-24T23:35:25.456 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.455+0000 7f8760b12640 1 -- 172.21.15.157:0/2094154064 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f875c0c4240 con 0x7f875c145e40 2024-04-24T23:35:25.456 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.455+0000 7f8752ffd640 1 -- 172.21.15.157:0/2094154064 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f875407a130 con 0x7f875c145e40 2024-04-24T23:35:25.456 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 64 2024-04-24T23:35:25.458 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.458+0000 7f8760b12640 1 -- 172.21.15.157:0/2094154064 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f873c06be30 msgr2=0x7f873c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:25.458 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.458+0000 7f8760b12640 1 --2- 172.21.15.157:0/2094154064 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f873c06be30 0x7f873c06e2f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f875c1519f0 tx=0x7f874c007e30 comp rx=0 tx=0).stop 2024-04-24T23:35:25.459 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.458+0000 7f8760b12640 1 -- 172.21.15.157:0/2094154064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f875c145e40 msgr2=0x7f875c1deac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:25.459 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.458+0000 7f8760b12640 1 --2- 172.21.15.157:0/2094154064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f875c145e40 0x7f875c1deac0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8754034040 tx=0x7f8754004a40 comp rx=0 tx=0).stop 2024-04-24T23:35:25.459 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.458+0000 7f8760b12640 1 -- 172.21.15.157:0/2094154064 shutdown_connections 2024-04-24T23:35:25.459 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.458+0000 7f8760b12640 1 --2- 172.21.15.157:0/2094154064 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f873c06be30 0x7f873c06e2f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:25.459 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.458+0000 7f8760b12640 1 --2- 172.21.15.157:0/2094154064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f875c145e40 0x7f875c1deac0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:25.459 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.458+0000 7f8760b12640 1 --2- 172.21.15.157:0/2094154064 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f875c152cc0 0x7f875c145900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:25.459 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.458+0000 7f8760b12640 1 --2- 172.21.15.157:0/2094154064 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f875c0c3370 0x7f875c149280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:25.459 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.458+0000 7f8760b12640 1 -- 172.21.15.157:0/2094154064 >> 172.21.15.157:0/2094154064 conn(0x7f875c0c68e0 msgr2=0x7f875c154920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:25.459 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.458+0000 7f8760b12640 1 -- 172.21.15.157:0/2094154064 shutdown_connections 2024-04-24T23:35:25.459 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.458+0000 7f8760b12640 1 -- 172.21.15.157:0/2094154064 wait complete. 2024-04-24T23:35:25.469 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T23:35:25.570 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.569+0000 7f33ea6da640 1 Processor -- start 2024-04-24T23:35:25.570 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.569+0000 7f33ea6da640 1 -- start start 2024-04-24T23:35:25.573 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.570+0000 7f33ea6da640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f33e40b03e0 0x7f33e41567e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:25.573 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.570+0000 7f33ea6da640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33e41592f0 0x7f33e4155c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:25.573 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.570+0000 7f33ea6da640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f33e40c55d0 0x7f33e40c5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:25.573 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.570+0000 7f33ea6da640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f33e40b07c0 con 0x7f33e40c55d0 2024-04-24T23:35:25.573 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.570+0000 7f33ea6da640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f33e40cb330 con 0x7f33e40b03e0 2024-04-24T23:35:25.573 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.570+0000 7f33ea6da640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f33e40cb610 con 0x7f33e41592f0 2024-04-24T23:35:25.573 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.570+0000 7f33e8c77640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f33e40c55d0 0x7f33e40c5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:25.573 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.570+0000 7f33e8c77640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f33e40c55d0 0x7f33e40c5a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40504/0 (socket says 172.21.15.157:40504) 2024-04-24T23:35:25.573 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.570+0000 7f33e8c77640 1 -- 172.21.15.157:0/1995428306 learned_addr learned my addr 172.21.15.157:0/1995428306 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:25.574 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.570+0000 7f33e3fff640 1 --2- 172.21.15.157:0/1995428306 >> v2:172.21.15.169:3300/0 conn(0x7f33e40b03e0 0x7f33e41567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:25.574 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.570+0000 7f33e37fe640 1 --2- 172.21.15.157:0/1995428306 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33e41592f0 0x7f33e4155c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:25.574 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.571+0000 7f33e8c77640 1 -- 172.21.15.157:0/1995428306 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33e41592f0 msgr2=0x7f33e4155c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:25.574 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.571+0000 7f33e8c77640 1 --2- 172.21.15.157:0/1995428306 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33e41592f0 0x7f33e4155c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:25.574 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.571+0000 7f33e8c77640 1 -- 172.21.15.157:0/1995428306 >> v2:172.21.15.169:3300/0 conn(0x7f33e40b03e0 msgr2=0x7f33e41567e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:25.574 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.571+0000 7f33e8c77640 1 --2- 172.21.15.157:0/1995428306 >> v2:172.21.15.169:3300/0 conn(0x7f33e40b03e0 0x7f33e41567e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:25.574 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.571+0000 7f33e8c77640 1 -- 172.21.15.157:0/1995428306 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33e40cb8f0 con 0x7f33e40c55d0 2024-04-24T23:35:25.574 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.571+0000 7f33e37fe640 1 --2- 172.21.15.157:0/1995428306 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33e41592f0 0x7f33e4155c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:35:25.574 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.571+0000 7f33e8c77640 1 --2- 172.21.15.157:0/1995428306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33e40c55d0 0x7f33e40c5a40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f33e40c8750 tx=0x7f33dc02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aebf2939555d178f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:25.574 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.571+0000 7f33e2ffd640 1 -- 172.21.15.157:0/1995428306 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33dc036070 con 0x7f33e40c55d0 2024-04-24T23:35:25.574 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.571+0000 7f33e2ffd640 1 -- 172.21.15.157:0/1995428306 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33dc031600 con 0x7f33e40c55d0 2024-04-24T23:35:25.574 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.572+0000 7f33e2ffd640 1 -- 172.21.15.157:0/1995428306 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33dc0319d0 con 0x7f33e40c55d0 2024-04-24T23:35:25.574 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.572+0000 7f33ea6da640 1 -- 172.21.15.157:0/1995428306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33e40c55d0 msgr2=0x7f33e40c5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:25.575 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.572+0000 7f33ea6da640 1 --2- 172.21.15.157:0/1995428306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33e40c55d0 0x7f33e40c5a40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f33e40c8750 tx=0x7f33dc02a610 comp rx=0 tx=0).stop 2024-04-24T23:35:25.575 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.572+0000 7f33ea6da640 1 -- 172.21.15.157:0/1995428306 shutdown_connections 2024-04-24T23:35:25.575 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.572+0000 7f33ea6da640 1 --2- 172.21.15.157:0/1995428306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33e40c55d0 0x7f33e40c5a40 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:25.575 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.572+0000 7f33ea6da640 1 --2- 172.21.15.157:0/1995428306 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33e41592f0 0x7f33e4155c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:25.575 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.572+0000 7f33ea6da640 1 --2- 172.21.15.157:0/1995428306 >> v2:172.21.15.169:3300/0 conn(0x7f33e40b03e0 0x7f33e41567e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:25.575 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.572+0000 7f33ea6da640 1 -- 172.21.15.157:0/1995428306 >> 172.21.15.157:0/1995428306 conn(0x7f33e4067080 msgr2=0x7f33e4067490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:25.575 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.572+0000 7f33ea6da640 1 -- 172.21.15.157:0/1995428306 shutdown_connections 2024-04-24T23:35:25.575 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.572+0000 7f33ea6da640 1 -- 172.21.15.157:0/1995428306 wait complete. 2024-04-24T23:35:25.578 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.573+0000 7f33ea6da640 1 Processor -- start 2024-04-24T23:35:25.578 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.573+0000 7f33ea6da640 1 -- start start 2024-04-24T23:35:25.578 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.574+0000 7f33ea6da640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33e40b03e0 0x7f33e414d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:25.578 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.574+0000 7f33ea6da640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33e40c55d0 0x7f33e40adc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:25.578 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.574+0000 7f33ea6da640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33e41592f0 0x7f33e40aa5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:25.578 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.574+0000 7f33ea6da640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f33e40cafc0 con 0x7f33e41592f0 2024-04-24T23:35:25.578 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.574+0000 7f33ea6da640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f33e41556e0 con 0x7f33e40c55d0 2024-04-24T23:35:25.578 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.574+0000 7f33ea6da640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f33e4159750 con 0x7f33e40b03e0 2024-04-24T23:35:25.578 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.574+0000 7f33e8c77640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33e41592f0 0x7f33e40aa5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:25.578 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.574+0000 7f33e8c77640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33e41592f0 0x7f33e40aa5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40514/0 (socket says 172.21.15.157:40514) 2024-04-24T23:35:25.579 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.574+0000 7f33e8c77640 1 -- 172.21.15.157:0/1681441262 learned_addr learned my addr 172.21.15.157:0/1681441262 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:25.579 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.574+0000 7f33e3fff640 1 --2- 172.21.15.157:0/1681441262 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33e40b03e0 0x7f33e414d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:25.579 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.574+0000 7f33e37fe640 1 --2- 172.21.15.157:0/1681441262 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33e40c55d0 0x7f33e40adc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:25.579 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.574+0000 7f33e8c77640 1 -- 172.21.15.157:0/1681441262 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33e40b03e0 msgr2=0x7f33e414d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:25.579 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.574+0000 7f33e8c77640 1 --2- 172.21.15.157:0/1681441262 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33e40b03e0 0x7f33e414d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:25.579 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.574+0000 7f33e8c77640 1 -- 172.21.15.157:0/1681441262 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33e40c55d0 msgr2=0x7f33e40adc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:25.579 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.574+0000 7f33e8c77640 1 --2- 172.21.15.157:0/1681441262 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33e40c55d0 0x7f33e40adc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:25.579 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.574+0000 7f33e8c77640 1 -- 172.21.15.157:0/1681441262 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33e41ea800 con 0x7f33e41592f0 2024-04-24T23:35:25.579 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.575+0000 7f33e8c77640 1 --2- 172.21.15.157:0/1681441262 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33e41592f0 0x7f33e40aa5f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f33dc009f40 tx=0x7f33dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:25.579 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.575+0000 7f33e0ff9640 1 -- 172.21.15.157:0/1681441262 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33dc036050 con 0x7f33e41592f0 2024-04-24T23:35:25.579 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.575+0000 7f33e0ff9640 1 -- 172.21.15.157:0/1681441262 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33dc042070 con 0x7f33e41592f0 2024-04-24T23:35:25.579 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.575+0000 7f33ea6da640 1 -- 172.21.15.157:0/1681441262 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33e41edf90 con 0x7f33e41592f0 2024-04-24T23:35:25.579 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.575+0000 7f33e0ff9640 1 -- 172.21.15.157:0/1681441262 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33dc009510 con 0x7f33e41592f0 2024-04-24T23:35:25.580 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.575+0000 7f33ea6da640 1 -- 172.21.15.157:0/1681441262 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33e41eb080 con 0x7f33e41592f0 2024-04-24T23:35:25.580 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.576+0000 7f33ea6da640 1 -- 172.21.15.157:0/1681441262 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33e40681c0 con 0x7f33e41592f0 2024-04-24T23:35:25.580 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.576+0000 7f33e0ff9640 1 -- 172.21.15.157:0/1681441262 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f33dc007bf0 con 0x7f33e41592f0 2024-04-24T23:35:25.580 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.577+0000 7f33e0ff9640 1 --2- 172.21.15.157:0/1681441262 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f33d006be30 0x7f33d006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:25.580 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.577+0000 7f33e0ff9640 1 -- 172.21.15.157:0/1681441262 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f33dc0abe00 con 0x7f33e41592f0 2024-04-24T23:35:25.580 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.577+0000 7f33e3fff640 1 --2- 172.21.15.157:0/1681441262 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f33d006be30 0x7f33d006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:25.580 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.577+0000 7f33e3fff640 1 --2- 172.21.15.157:0/1681441262 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f33d006be30 0x7f33d006e2f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f33d4009b40 tx=0x7f33d4009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:25.581 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.581+0000 7f33e0ff9640 1 -- 172.21.15.157:0/1681441262 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f33dc079eb0 con 0x7f33e41592f0 2024-04-24T23:35:25.754 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.753+0000 7f33ea6da640 1 -- 172.21.15.157:0/1681441262 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f33e406d1d0 con 0x7f33e41592f0 2024-04-24T23:35:25.754 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.754+0000 7f33e0ff9640 1 -- 172.21.15.157:0/1681441262 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7f33dc07e1b0 con 0x7f33e41592f0 2024-04-24T23:35:25.754 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2024-04-24T23:35:25.756 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.756+0000 7f33ea6da640 1 -- 172.21.15.157:0/1681441262 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f33d006be30 msgr2=0x7f33d006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:25.757 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.756+0000 7f33ea6da640 1 --2- 172.21.15.157:0/1681441262 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f33d006be30 0x7f33d006e2f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f33d4009b40 tx=0x7f33d4009190 comp rx=0 tx=0).stop 2024-04-24T23:35:25.757 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.756+0000 7f33ea6da640 1 -- 172.21.15.157:0/1681441262 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33e41592f0 msgr2=0x7f33e40aa5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:25.757 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.756+0000 7f33ea6da640 1 --2- 172.21.15.157:0/1681441262 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33e41592f0 0x7f33e40aa5f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f33dc009f40 tx=0x7f33dc004a40 comp rx=0 tx=0).stop 2024-04-24T23:35:25.757 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.756+0000 7f33ea6da640 1 -- 172.21.15.157:0/1681441262 shutdown_connections 2024-04-24T23:35:25.757 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.756+0000 7f33ea6da640 1 --2- 172.21.15.157:0/1681441262 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f33d006be30 0x7f33d006e2f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:25.757 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.756+0000 7f33ea6da640 1 --2- 172.21.15.157:0/1681441262 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33e41592f0 0x7f33e40aa5f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:25.757 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.756+0000 7f33ea6da640 1 --2- 172.21.15.157:0/1681441262 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33e40c55d0 0x7f33e40adc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:25.757 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.756+0000 7f33ea6da640 1 --2- 172.21.15.157:0/1681441262 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33e40b03e0 0x7f33e414d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:25.757 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.756+0000 7f33ea6da640 1 -- 172.21.15.157:0/1681441262 >> 172.21.15.157:0/1681441262 conn(0x7f33e4067080 msgr2=0x7f33e40c63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:25.757 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.756+0000 7f33ea6da640 1 -- 172.21.15.157:0/1681441262 shutdown_connections 2024-04-24T23:35:25.757 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.756+0000 7f33ea6da640 1 -- 172.21.15.157:0/1681441262 wait complete. 2024-04-24T23:35:25.767 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:35:25.866 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.865+0000 7fb20194f640 1 Processor -- start 2024-04-24T23:35:25.866 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.865+0000 7fb20194f640 1 -- start start 2024-04-24T23:35:25.869 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.866+0000 7fb20194f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb1fc073130 0x7fb1fc0c2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:25.869 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.866+0000 7fb20194f640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb1fc0c4b20 0x7fb1fc06ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:25.869 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.866+0000 7fb20194f640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1fc06c400 con 0x7fb1fc14b720 2024-04-24T23:35:25.869 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.866+0000 7fb20194f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fb1fc073510 con 0x7fb1fc073130 2024-04-24T23:35:25.870 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.866+0000 7fb20194f640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fb1fc06c6a0 con 0x7fb1fc0c4b20 2024-04-24T23:35:25.870 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.866+0000 7fb1faffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb1fc073130 0x7fb1fc0c2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:25.870 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.866+0000 7fb1faffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb1fc073130 0x7fb1fc0c2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40520/0 (socket says 172.21.15.157:40520) 2024-04-24T23:35:25.870 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.866+0000 7fb1faffd640 1 -- 172.21.15.157:0/2985911605 learned_addr learned my addr 172.21.15.157:0/2985911605 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:25.870 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.866+0000 7fb1fa7fc640 1 --2- 172.21.15.157:0/2985911605 >> v2:172.21.15.169:3300/0 conn(0x7fb1fc0c4b20 0x7fb1fc06ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:25.870 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.866+0000 7fb1faffd640 1 -- 172.21.15.157:0/2985911605 >> v2:172.21.15.169:3300/0 conn(0x7fb1fc0c4b20 msgr2=0x7fb1fc06ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:25.870 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.866+0000 7fb1faffd640 1 --2- 172.21.15.157:0/2985911605 >> v2:172.21.15.169:3300/0 conn(0x7fb1fc0c4b20 0x7fb1fc06ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:25.870 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.866+0000 7fb1faffd640 1 -- 172.21.15.157:0/2985911605 >> v1:172.21.15.169:6789/0 conn(0x7fb1fc14b720 legacy=0x7fb1fc14bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:25.870 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.866+0000 7fb1faffd640 1 -- 172.21.15.157:0/2985911605 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1fc0c1eb0 con 0x7fb1fc073130 2024-04-24T23:35:25.870 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.867+0000 7fb1faffd640 1 --2- 172.21.15.157:0/2985911605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1fc073130 0x7fb1fc0c2990 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc067ef0 tx=0x7fb1ec02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50fd27f0f49f3856 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:25.870 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.867+0000 7fb1f9ffb640 1 -- 172.21.15.157:0/2985911605 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec037070 con 0x7fb1fc073130 2024-04-24T23:35:25.871 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.867+0000 7fb1f9ffb640 1 -- 172.21.15.157:0/2985911605 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1ec032440 con 0x7fb1fc073130 2024-04-24T23:35:25.871 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.867+0000 7fb1f9ffb640 1 -- 172.21.15.157:0/2985911605 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec032810 con 0x7fb1fc073130 2024-04-24T23:35:25.871 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.868+0000 7fb20194f640 1 -- 172.21.15.157:0/2985911605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1fc073130 msgr2=0x7fb1fc0c2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:25.871 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.868+0000 7fb20194f640 1 --2- 172.21.15.157:0/2985911605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1fc073130 0x7fb1fc0c2990 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc067ef0 tx=0x7fb1ec02a1a0 comp rx=0 tx=0).stop 2024-04-24T23:35:25.871 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.868+0000 7fb20194f640 1 -- 172.21.15.157:0/2985911605 shutdown_connections 2024-04-24T23:35:25.871 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.868+0000 7fb20194f640 1 --2- 172.21.15.157:0/2985911605 >> v2:172.21.15.169:3300/0 conn(0x7fb1fc0c4b20 0x7fb1fc06ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:25.871 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.868+0000 7fb20194f640 1 --2- 172.21.15.157:0/2985911605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1fc073130 0x7fb1fc0c2990 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:25.871 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.868+0000 7fb20194f640 1 -- 172.21.15.157:0/2985911605 >> 172.21.15.157:0/2985911605 conn(0x7fb1fc0a8720 msgr2=0x7fb1fc0a8b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:25.871 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.868+0000 7fb20194f640 1 -- 172.21.15.157:0/2985911605 shutdown_connections 2024-04-24T23:35:25.871 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.868+0000 7fb20194f640 1 -- 172.21.15.157:0/2985911605 wait complete. 2024-04-24T23:35:25.871 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.869+0000 7fb20194f640 1 Processor -- start 2024-04-24T23:35:25.871 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.869+0000 7fb20194f640 1 -- start start 2024-04-24T23:35:25.874 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.869+0000 7fb20194f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1fc073130 0x7fb1fc1522a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:25.874 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.869+0000 7fb20194f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1fc0c4b20 0x7fb1fc1ed2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:25.874 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.870+0000 7fb20194f640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1fc14b720 0x7fb1fc1ed830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:25.874 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.870+0000 7fb20194f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb1fc06d1f0 con 0x7fb1fc073130 2024-04-24T23:35:25.874 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.870+0000 7fb20194f640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb1fc0c2470 con 0x7fb1fc0c4b20 2024-04-24T23:35:25.874 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.870+0000 7fb20194f640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb1fc06d370 con 0x7fb1fc14b720 2024-04-24T23:35:25.875 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.870+0000 7fb1faffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1fc073130 0x7fb1fc1522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:25.875 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.870+0000 7fb1faffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1fc073130 0x7fb1fc1522a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40526/0 (socket says 172.21.15.157:40526) 2024-04-24T23:35:25.875 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.870+0000 7fb1faffd640 1 -- 172.21.15.157:0/1639473367 learned_addr learned my addr 172.21.15.157:0/1639473367 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:25.875 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.870+0000 7fb1faffd640 1 -- 172.21.15.157:0/1639473367 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1fc14b720 msgr2=0x7fb1fc1ed830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:25.875 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.870+0000 7fb1faffd640 1 --2- 172.21.15.157:0/1639473367 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1fc14b720 0x7fb1fc1ed830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:25.875 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.870+0000 7fb1fa7fc640 1 --2- 172.21.15.157:0/1639473367 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1fc0c4b20 0x7fb1fc1ed2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:25.875 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.870+0000 7fb1faffd640 1 -- 172.21.15.157:0/1639473367 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1fc0c4b20 msgr2=0x7fb1fc1ed2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:25.875 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.870+0000 7fb1faffd640 1 --2- 172.21.15.157:0/1639473367 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1fc0c4b20 0x7fb1fc1ed2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:25.875 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.870+0000 7fb1faffd640 1 -- 172.21.15.157:0/1639473367 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1fc0ab790 con 0x7fb1fc073130 2024-04-24T23:35:25.875 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.870+0000 7fb1faffd640 1 --2- 172.21.15.157:0/1639473367 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1fc073130 0x7fb1fc1522a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec000c00 tx=0x7fb1ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:25.875 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.870+0000 7fb1e3fff640 1 -- 172.21.15.157:0/1639473367 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec041070 con 0x7fb1fc073130 2024-04-24T23:35:25.875 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.870+0000 7fb1e3fff640 1 -- 172.21.15.157:0/1639473367 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1ec03c870 con 0x7fb1fc073130 2024-04-24T23:35:25.876 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.870+0000 7fb20194f640 1 -- 172.21.15.157:0/1639473367 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1fc14c9b0 con 0x7fb1fc073130 2024-04-24T23:35:25.876 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.870+0000 7fb1e3fff640 1 -- 172.21.15.157:0/1639473367 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec037040 con 0x7fb1fc073130 2024-04-24T23:35:25.876 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.870+0000 7fb20194f640 1 -- 172.21.15.157:0/1639473367 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1fc0aa380 con 0x7fb1fc073130 2024-04-24T23:35:25.876 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.871+0000 7fb20194f640 1 -- 172.21.15.157:0/1639473367 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1fc15d760 con 0x7fb1fc073130 2024-04-24T23:35:25.876 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.872+0000 7fb1e3fff640 1 -- 172.21.15.157:0/1639473367 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb1ec044070 con 0x7fb1fc073130 2024-04-24T23:35:25.876 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.872+0000 7fb1e3fff640 1 --2- 172.21.15.157:0/1639473367 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fb1e806be60 0x7fb1e806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:25.876 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.872+0000 7fb1e3fff640 1 -- 172.21.15.157:0/1639473367 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fb1ec0abe60 con 0x7fb1fc073130 2024-04-24T23:35:25.876 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.872+0000 7fb1fa7fc640 1 --2- 172.21.15.157:0/1639473367 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fb1e806be60 0x7fb1e806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:25.876 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.873+0000 7fb1fa7fc640 1 --2- 172.21.15.157:0/1639473367 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fb1e806be60 0x7fb1e806e320 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc06ce80 tx=0x7fb1f0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:25.877 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:25.877+0000 7fb1e3fff640 1 -- 172.21.15.157:0/1639473367 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb1ec07a5f0 con 0x7fb1fc073130 2024-04-24T23:35:26.086 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.085+0000 7fb20194f640 1 -- 172.21.15.157:0/1639473367 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb1fc06dc30 con 0x7fb1fc073130 2024-04-24T23:35:26.087 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.086+0000 7fb1e3fff640 1 -- 172.21.15.157:0/1639473367 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5189 (secure 0 0 0) 0x7fb1ec079ef0 con 0x7fb1fc073130 2024-04-24T23:35:26.087 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:35:26.087 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1276438765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1276438765},{"type":"v1","addr":"172.21.15.169:6837","nonce":1276438765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1398599086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":1398599086},{"type":"v1","addr":"172.21.15.157:6835","nonce":1398599086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4258,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3795840339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3795840339},{"type":"v1","addr":"172.21.15.157:6834","nonce":3795840339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:34:53.701085+0000","modified":"2024-04-24T23:35:15.789166+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6836/3782746080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6835","nonce":3782746080},{"type":"v1","addr":"172.21.15.169:6836","nonce":3782746080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4270,"qdb_cluster":[4270]},"id":1}]} 2024-04-24T23:35:26.087 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 9 2024-04-24T23:35:26.089 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.088+0000 7fb20194f640 1 -- 172.21.15.157:0/1639473367 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fb1e806be60 msgr2=0x7fb1e806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:26.089 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.088+0000 7fb20194f640 1 --2- 172.21.15.157:0/1639473367 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fb1e806be60 0x7fb1e806e320 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc06ce80 tx=0x7fb1f0008040 comp rx=0 tx=0).stop 2024-04-24T23:35:26.089 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.088+0000 7fb20194f640 1 -- 172.21.15.157:0/1639473367 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1fc073130 msgr2=0x7fb1fc1522a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:26.089 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.088+0000 7fb20194f640 1 --2- 172.21.15.157:0/1639473367 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1fc073130 0x7fb1fc1522a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec000c00 tx=0x7fb1ec0046c0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.089 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.088+0000 7fb20194f640 1 -- 172.21.15.157:0/1639473367 shutdown_connections 2024-04-24T23:35:26.089 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.088+0000 7fb20194f640 1 --2- 172.21.15.157:0/1639473367 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fb1e806be60 0x7fb1e806e320 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.089 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.088+0000 7fb20194f640 1 --2- 172.21.15.157:0/1639473367 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1fc14b720 0x7fb1fc1ed830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.089 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.088+0000 7fb20194f640 1 --2- 172.21.15.157:0/1639473367 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1fc0c4b20 0x7fb1fc1ed2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.089 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.088+0000 7fb20194f640 1 --2- 172.21.15.157:0/1639473367 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1fc073130 0x7fb1fc1522a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.090 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.088+0000 7fb20194f640 1 -- 172.21.15.157:0/1639473367 >> 172.21.15.157:0/1639473367 conn(0x7fb1fc0a8720 msgr2=0x7fb1fc1553f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:26.090 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.088+0000 7fb20194f640 1 -- 172.21.15.157:0/1639473367 shutdown_connections 2024-04-24T23:35:26.090 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.088+0000 7fb20194f640 1 -- 172.21.15.157:0/1639473367 wait complete. 2024-04-24T23:35:26.100 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:35:26.197 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.196+0000 7fa424bc1640 1 Processor -- start 2024-04-24T23:35:26.197 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.196+0000 7fa424bc1640 1 -- start start 2024-04-24T23:35:26.198 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.197+0000 7fa424bc1640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa420155a60 0x7fa420160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:26.198 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.197+0000 7fa424bc1640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4200c8bb0 con 0x7fa4200be360 2024-04-24T23:35:26.198 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.197+0000 7fa424bc1640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa420159480 con 0x7fa420152b40 2024-04-24T23:35:26.198 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.197+0000 7fa424bc1640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fa4200c8e90 con 0x7fa420155a60 2024-04-24T23:35:26.199 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.197+0000 7fa41ed9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa420155a60 0x7fa420160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:26.199 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.197+0000 7fa41dd9b640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fa4200be360 0x7fa420155350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49130/0 (socket says 172.21.15.157:49130) 2024-04-24T23:35:26.199 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.197+0000 7fa41dd9b640 1 -- 172.21.15.157:0/2068973194 learned_addr learned my addr 172.21.15.157:0/2068973194 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-24T23:35:26.199 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.198+0000 7fa41ed9d640 1 -- 172.21.15.157:0/2068973194 >> v1:172.21.15.169:6789/0 conn(0x7fa420152b40 legacy=0x7fa4200c5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:26.199 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.198+0000 7fa41ed9d640 1 -- 172.21.15.157:0/2068973194 >> v1:172.21.15.157:6789/0 conn(0x7fa4200be360 legacy=0x7fa420155350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:26.199 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.198+0000 7fa41ed9d640 1 -- 172.21.15.157:0/2068973194 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa420159760 con 0x7fa420155a60 2024-04-24T23:35:26.199 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.198+0000 7fa41d59a640 1 -- 172.21.15.157:0/2068973194 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3672339369 0 0) 0x7fa4200c8bb0 con 0x7fa4200be360 2024-04-24T23:35:26.200 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.198+0000 7fa41ed9d640 1 --2- 172.21.15.157:0/2068973194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa420155a60 0x7fa420160ce0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa418009d90 tx=0x7fa41802a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd80a7f19d243187 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:26.200 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.199+0000 7fa41d59a640 1 -- 172.21.15.157:0/2068973194 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa418031600 con 0x7fa420155a60 2024-04-24T23:35:26.201 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.199+0000 7fa41d59a640 1 -- 172.21.15.157:0/2068973194 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4180317a0 con 0x7fa420155a60 2024-04-24T23:35:26.201 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.199+0000 7fa41d59a640 1 -- 172.21.15.157:0/2068973194 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa418031b70 con 0x7fa420155a60 2024-04-24T23:35:26.201 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.199+0000 7fa424bc1640 1 -- 172.21.15.157:0/2068973194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa420155a60 msgr2=0x7fa420160ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:26.201 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.199+0000 7fa424bc1640 1 --2- 172.21.15.157:0/2068973194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa420155a60 0x7fa420160ce0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa418009d90 tx=0x7fa41802a5d0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.201 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.199+0000 7fa424bc1640 1 -- 172.21.15.157:0/2068973194 shutdown_connections 2024-04-24T23:35:26.201 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.199+0000 7fa424bc1640 1 --2- 172.21.15.157:0/2068973194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa420155a60 0x7fa420160ce0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.201 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.199+0000 7fa424bc1640 1 -- 172.21.15.157:0/2068973194 >> 172.21.15.157:0/2068973194 conn(0x7fa4200672c0 msgr2=0x7fa4200676d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:26.201 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.199+0000 7fa424bc1640 1 -- 172.21.15.157:0/2068973194 shutdown_connections 2024-04-24T23:35:26.201 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.199+0000 7fa424bc1640 1 -- 172.21.15.157:0/2068973194 wait complete. 2024-04-24T23:35:26.203 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.200+0000 7fa424bc1640 1 Processor -- start 2024-04-24T23:35:26.203 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.200+0000 7fa424bc1640 1 -- start start 2024-04-24T23:35:26.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.201+0000 7fa424bc1640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4200be360 0x7fa4200aa960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:26.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.201+0000 7fa424bc1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa420152b40 0x7fa4201541e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:26.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.201+0000 7fa424bc1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa420155a60 0x7fa4201eb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:26.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.201+0000 7fa424bc1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa4200caa00 con 0x7fa420155a60 2024-04-24T23:35:26.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.201+0000 7fa424bc1640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa4200cb210 con 0x7fa420152b40 2024-04-24T23:35:26.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.201+0000 7fa424bc1640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa4200cba40 con 0x7fa4200be360 2024-04-24T23:35:26.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.201+0000 7fa41ed9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa420155a60 0x7fa4201eb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:26.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.201+0000 7fa41ed9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa420155a60 0x7fa4201eb2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40544/0 (socket says 172.21.15.157:40544) 2024-04-24T23:35:26.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.201+0000 7fa41ed9d640 1 -- 172.21.15.157:0/1140694253 learned_addr learned my addr 172.21.15.157:0/1140694253 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:26.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.201+0000 7fa41dd9b640 1 --2- 172.21.15.157:0/1140694253 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa420152b40 0x7fa4201541e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:26.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.201+0000 7fa41e59c640 1 --2- 172.21.15.157:0/1140694253 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4200be360 0x7fa4200aa960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:26.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.201+0000 7fa41ed9d640 1 -- 172.21.15.157:0/1140694253 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4200be360 msgr2=0x7fa4200aa960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:26.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.201+0000 7fa41ed9d640 1 --2- 172.21.15.157:0/1140694253 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4200be360 0x7fa4200aa960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.202+0000 7fa41ed9d640 1 -- 172.21.15.157:0/1140694253 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa420152b40 msgr2=0x7fa4201541e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:26.206 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.202+0000 7fa41ed9d640 1 --2- 172.21.15.157:0/1140694253 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa420152b40 0x7fa4201541e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.206 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.202+0000 7fa41ed9d640 1 -- 172.21.15.157:0/1140694253 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4200aaea0 con 0x7fa420155a60 2024-04-24T23:35:26.206 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.202+0000 7fa41ed9d640 1 --2- 172.21.15.157:0/1140694253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa420155a60 0x7fa4201eb2c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa418009eb0 tx=0x7fa418004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:26.206 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.202+0000 7fa40effd640 1 -- 172.21.15.157:0/1140694253 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa41803f070 con 0x7fa420155a60 2024-04-24T23:35:26.206 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.202+0000 7fa40effd640 1 -- 172.21.15.157:0/1140694253 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa418030b60 con 0x7fa420155a60 2024-04-24T23:35:26.207 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.202+0000 7fa424bc1640 1 -- 172.21.15.157:0/1140694253 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4200ad350 con 0x7fa420155a60 2024-04-24T23:35:26.207 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.202+0000 7fa40effd640 1 -- 172.21.15.157:0/1140694253 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa418043410 con 0x7fa420155a60 2024-04-24T23:35:26.207 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.202+0000 7fa424bc1640 1 -- 172.21.15.157:0/1140694253 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa42014fa60 con 0x7fa420155a60 2024-04-24T23:35:26.207 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.203+0000 7fa424bc1640 1 -- 172.21.15.157:0/1140694253 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa42006ccc0 con 0x7fa420155a60 2024-04-24T23:35:26.208 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.203+0000 7fa40effd640 1 -- 172.21.15.157:0/1140694253 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa41803b410 con 0x7fa420155a60 2024-04-24T23:35:26.208 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.204+0000 7fa40effd640 1 --2- 172.21.15.157:0/1140694253 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fa40006be30 0x7fa40006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:26.208 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.204+0000 7fa40effd640 1 -- 172.21.15.157:0/1140694253 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa4180ac430 con 0x7fa420155a60 2024-04-24T23:35:26.208 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.204+0000 7fa41e59c640 1 --2- 172.21.15.157:0/1140694253 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fa40006be30 0x7fa40006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:26.208 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.204+0000 7fa41e59c640 1 --2- 172.21.15.157:0/1140694253 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fa40006be30 0x7fa40006e2f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa4201512f0 tx=0x7fa410009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:26.208 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.208+0000 7fa40effd640 1 -- 172.21.15.157:0/1140694253 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa41807abf0 con 0x7fa420155a60 2024-04-24T23:35:26.390 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.389+0000 7fa424bc1640 1 -- 172.21.15.157:0/1140694253 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa4200681c0 con 0x7fa420155a60 2024-04-24T23:35:26.391 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.391+0000 7fa40effd640 1 -- 172.21.15.157:0/1140694253 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18538 (secure 0 0 0) 0x7fa41807a4f0 con 0x7fa420155a60 2024-04-24T23:35:26.391 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:35:26.392 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":25,"fsid":"223685fb-5b64-46a5-9857-3791c9c07df2","created":"2024-04-24T23:34:37.504202+0000","modified":"2024-04-24T23:34:58.744500+0000","last_up_change":"2024-04-24T23:34:47.584963+0000","last_in_change":"2024-04-24T23:34:39.632954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:34:47.796655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:34:52.177208+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:34:52.667022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:34:56.395517+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"95d14ac8-fccf-4261-be8e-edafa746f629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6809","nonce":4003822160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6811","nonce":4003822160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6815","nonce":4003822160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6813","nonce":4003822160}]},"public_addr":"172.21.15.157:6809/4003822160","cluster_addr":"172.21.15.157:6811/4003822160","heartbeat_back_addr":"172.21.15.157:6815/4003822160","heartbeat_front_addr":"172.21.15.157:6813/4003822160","state":["exists","up"]},{"osd":1,"uuid":"7be79da9-8eec-4990-9178-71d602e39d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6817","nonce":1557575576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6819","nonce":1557575576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6823","nonce":1557575576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6821","nonce":1557575576}]},"public_addr":"172.21.15.157:6817/1557575576","cluster_addr":"172.21.15.157:6819/1557575576","heartbeat_back_addr":"172.21.15.157:6823/1557575576","heartbeat_front_addr":"172.21.15.157:6821/1557575576","state":["exists","up"]},{"osd":2,"uuid":"0127ed00-b7c3-466d-bfb0-01b0fd315f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6825","nonce":2793948092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6827","nonce":2793948092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6831","nonce":2793948092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6829","nonce":2793948092}]},"public_addr":"172.21.15.157:6825/2793948092","cluster_addr":"172.21.15.157:6827/2793948092","heartbeat_back_addr":"172.21.15.157:6831/2793948092","heartbeat_front_addr":"172.21.15.157:6829/2793948092","state":["exists","up"]},{"osd":3,"uuid":"f2331acc-aa37-433e-868e-3561b6039562","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6801","nonce":422758549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6803","nonce":422758549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6807","nonce":422758549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6805","nonce":422758549}]},"public_addr":"172.21.15.157:6801/422758549","cluster_addr":"172.21.15.157:6803/422758549","heartbeat_back_addr":"172.21.15.157:6807/422758549","heartbeat_front_addr":"172.21.15.157:6805/422758549","state":["exists","up"]},{"osd":4,"uuid":"1998854b-7398-43f5-ab7d-79fa0debcf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6801","nonce":321910099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6803","nonce":321910099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6807","nonce":321910099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6805","nonce":321910099}]},"public_addr":"172.21.15.169:6801/321910099","cluster_addr":"172.21.15.169:6803/321910099","heartbeat_back_addr":"172.21.15.169:6807/321910099","heartbeat_front_addr":"172.21.15.169:6805/321910099","state":["exists","up"]},{"osd":5,"uuid":"c37153ff-9b2b-48b9-a684-2c82b6f39f15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6825","nonce":243321711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6827","nonce":243321711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6831","nonce":243321711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6829","nonce":243321711}]},"public_addr":"172.21.15.169:6825/243321711","cluster_addr":"172.21.15.169:6827/243321711","heartbeat_back_addr":"172.21.15.169:6831/243321711","heartbeat_front_addr":"172.21.15.169:6829/243321711","state":["exists","up"]},{"osd":6,"uuid":"8db79e1c-f36b-49cb-8d8a-a20c2aec57c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6809","nonce":3441420785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6811","nonce":3441420785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6815","nonce":3441420785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6813","nonce":3441420785}]},"public_addr":"172.21.15.169:6809/3441420785","cluster_addr":"172.21.15.169:6811/3441420785","heartbeat_back_addr":"172.21.15.169:6815/3441420785","heartbeat_front_addr":"172.21.15.169:6813/3441420785","state":["exists","up"]},{"osd":7,"uuid":"928586e9-296c-4b8b-9b27-42f9f155dbdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6817","nonce":1663179020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6819","nonce":1663179020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6823","nonce":1663179020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6821","nonce":1663179020}]},"public_addr":"172.21.15.169:6817/1663179020","cluster_addr":"172.21.15.169:6819/1663179020","heartbeat_back_addr":"172.21.15.169:6823/1663179020","heartbeat_front_addr":"172.21.15.169:6821/1663179020","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.848039+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.826489+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.801120+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.727706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.824343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.791915+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.878073+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.810831+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:35:26.394 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.393+0000 7fa424bc1640 1 -- 172.21.15.157:0/1140694253 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fa40006be30 msgr2=0x7fa40006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:26.394 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.393+0000 7fa424bc1640 1 --2- 172.21.15.157:0/1140694253 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fa40006be30 0x7fa40006e2f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa4201512f0 tx=0x7fa410009040 comp rx=0 tx=0).stop 2024-04-24T23:35:26.394 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.393+0000 7fa424bc1640 1 -- 172.21.15.157:0/1140694253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa420155a60 msgr2=0x7fa4201eb2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:26.394 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.393+0000 7fa424bc1640 1 --2- 172.21.15.157:0/1140694253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa420155a60 0x7fa4201eb2c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa418009eb0 tx=0x7fa418004a40 comp rx=0 tx=0).stop 2024-04-24T23:35:26.394 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.393+0000 7fa424bc1640 1 -- 172.21.15.157:0/1140694253 shutdown_connections 2024-04-24T23:35:26.394 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.393+0000 7fa424bc1640 1 --2- 172.21.15.157:0/1140694253 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fa40006be30 0x7fa40006e2f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.394 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.393+0000 7fa424bc1640 1 --2- 172.21.15.157:0/1140694253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa420155a60 0x7fa4201eb2c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.394 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.393+0000 7fa424bc1640 1 --2- 172.21.15.157:0/1140694253 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa420152b40 0x7fa4201541e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.395 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.393+0000 7fa424bc1640 1 --2- 172.21.15.157:0/1140694253 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4200be360 0x7fa4200aa960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.395 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.393+0000 7fa424bc1640 1 -- 172.21.15.157:0/1140694253 >> 172.21.15.157:0/1140694253 conn(0x7fa4200672c0 msgr2=0x7fa420150360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:26.395 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.393+0000 7fa424bc1640 1 -- 172.21.15.157:0/1140694253 shutdown_connections 2024-04-24T23:35:26.395 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.393+0000 7fa424bc1640 1 -- 172.21.15.157:0/1140694253 wait complete. 2024-04-24T23:35:26.405 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:35:26.502 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.502+0000 7fcd4a1ed640 1 Processor -- start 2024-04-24T23:35:26.503 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.502+0000 7fcd4a1ed640 1 -- start start 2024-04-24T23:35:26.505 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.502+0000 7fcd4a1ed640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcd44152b40 0x7fcd440c5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:26.505 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.502+0000 7fcd4a1ed640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcd440be360 0x7fcd44155350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:26.505 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.502+0000 7fcd4a1ed640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd440c5b70 0x7fcd44155890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:26.505 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.502+0000 7fcd4a1ed640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fcd440c8e90 con 0x7fcd44152b40 2024-04-24T23:35:26.505 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.502+0000 7fcd4a1ed640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fcd44155dd0 con 0x7fcd440be360 2024-04-24T23:35:26.505 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.502+0000 7fcd4a1ed640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcd440c8bb0 con 0x7fcd440c5b70 2024-04-24T23:35:26.505 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.503+0000 7fcd437fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcd44152b40 0x7fcd440c5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:26.505 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.503+0000 7fcd437fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcd44152b40 0x7fcd440c5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40548/0 (socket says 172.21.15.157:40548) 2024-04-24T23:35:26.505 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.503+0000 7fcd437fe640 1 -- 172.21.15.157:0/1808168767 learned_addr learned my addr 172.21.15.157:0/1808168767 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:26.506 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.503+0000 7fcd43fff640 1 --2- 172.21.15.157:0/1808168767 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd440c5b70 0x7fcd44155890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:26.506 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.503+0000 7fcd42ffd640 1 --2- 172.21.15.157:0/1808168767 >> v2:172.21.15.169:3300/0 conn(0x7fcd440be360 0x7fcd44155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:26.506 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.503+0000 7fcd437fe640 1 -- 172.21.15.157:0/1808168767 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd440c5b70 msgr2=0x7fcd44155890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:26.506 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.503+0000 7fcd437fe640 1 --2- 172.21.15.157:0/1808168767 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd440c5b70 0x7fcd44155890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.506 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.503+0000 7fcd437fe640 1 -- 172.21.15.157:0/1808168767 >> v2:172.21.15.169:3300/0 conn(0x7fcd440be360 msgr2=0x7fcd44155350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:26.506 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.503+0000 7fcd437fe640 1 --2- 172.21.15.157:0/1808168767 >> v2:172.21.15.169:3300/0 conn(0x7fcd440be360 0x7fcd44155350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.506 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.503+0000 7fcd437fe640 1 -- 172.21.15.157:0/1808168767 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd44159480 con 0x7fcd44152b40 2024-04-24T23:35:26.506 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.504+0000 7fcd437fe640 1 --2- 172.21.15.157:0/1808168767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd44152b40 0x7fcd440c5630 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcd4406d120 tx=0x7fcd3402a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88002c6966c92a0b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:26.506 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.504+0000 7fcd427fc640 1 -- 172.21.15.157:0/1808168767 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd34037070 con 0x7fcd44152b40 2024-04-24T23:35:26.507 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.504+0000 7fcd427fc640 1 -- 172.21.15.157:0/1808168767 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd3402ae00 con 0x7fcd44152b40 2024-04-24T23:35:26.507 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.504+0000 7fcd427fc640 1 -- 172.21.15.157:0/1808168767 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd34032660 con 0x7fcd44152b40 2024-04-24T23:35:26.507 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.504+0000 7fcd4a1ed640 1 -- 172.21.15.157:0/1808168767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd44152b40 msgr2=0x7fcd440c5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:26.507 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.504+0000 7fcd4a1ed640 1 --2- 172.21.15.157:0/1808168767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd44152b40 0x7fcd440c5630 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcd4406d120 tx=0x7fcd3402a0b0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.507 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.504+0000 7fcd4a1ed640 1 -- 172.21.15.157:0/1808168767 shutdown_connections 2024-04-24T23:35:26.507 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.504+0000 7fcd4a1ed640 1 --2- 172.21.15.157:0/1808168767 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd440c5b70 0x7fcd44155890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.507 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.504+0000 7fcd4a1ed640 1 --2- 172.21.15.157:0/1808168767 >> v2:172.21.15.169:3300/0 conn(0x7fcd440be360 0x7fcd44155350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.507 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.504+0000 7fcd4a1ed640 1 --2- 172.21.15.157:0/1808168767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd44152b40 0x7fcd440c5630 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.507 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.504+0000 7fcd4a1ed640 1 -- 172.21.15.157:0/1808168767 >> 172.21.15.157:0/1808168767 conn(0x7fcd440672c0 msgr2=0x7fcd440676d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:26.507 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.504+0000 7fcd4a1ed640 1 -- 172.21.15.157:0/1808168767 shutdown_connections 2024-04-24T23:35:26.507 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.504+0000 7fcd4a1ed640 1 -- 172.21.15.157:0/1808168767 wait complete. 2024-04-24T23:35:26.510 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.505+0000 7fcd4a1ed640 1 Processor -- start 2024-04-24T23:35:26.510 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.506+0000 7fcd4a1ed640 1 -- start start 2024-04-24T23:35:26.510 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.506+0000 7fcd4a1ed640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd440be360 0x7fcd441ec0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:26.510 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.506+0000 7fcd4a1ed640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd440c5b70 0x7fcd441ee460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:26.510 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.506+0000 7fcd4a1ed640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd44152b40 0x7fcd4414bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:26.510 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.506+0000 7fcd4a1ed640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcd440cb210 con 0x7fcd44152b40 2024-04-24T23:35:26.510 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.506+0000 7fcd4a1ed640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcd440cb520 con 0x7fcd440c5b70 2024-04-24T23:35:26.511 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.506+0000 7fcd4a1ed640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcd440caa00 con 0x7fcd440be360 2024-04-24T23:35:26.511 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.506+0000 7fcd43fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd44152b40 0x7fcd4414bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:26.511 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.506+0000 7fcd43fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd44152b40 0x7fcd4414bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40552/0 (socket says 172.21.15.157:40552) 2024-04-24T23:35:26.511 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.506+0000 7fcd43fff640 1 -- 172.21.15.157:0/3087976554 learned_addr learned my addr 172.21.15.157:0/3087976554 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:26.511 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.506+0000 7fcd437fe640 1 --2- 172.21.15.157:0/3087976554 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd440be360 0x7fcd441ec0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:26.511 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.506+0000 7fcd42ffd640 1 --2- 172.21.15.157:0/3087976554 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd440c5b70 0x7fcd441ee460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:26.511 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.507+0000 7fcd43fff640 1 -- 172.21.15.157:0/3087976554 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd440be360 msgr2=0x7fcd441ec0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:26.511 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.507+0000 7fcd43fff640 1 --2- 172.21.15.157:0/3087976554 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd440be360 0x7fcd441ec0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.511 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.507+0000 7fcd43fff640 1 -- 172.21.15.157:0/3087976554 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd440c5b70 msgr2=0x7fcd441ee460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:26.511 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.507+0000 7fcd43fff640 1 --2- 172.21.15.157:0/3087976554 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd440c5b70 0x7fcd441ee460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.511 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.507+0000 7fcd43fff640 1 -- 172.21.15.157:0/3087976554 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd440ad5e0 con 0x7fcd44152b40 2024-04-24T23:35:26.511 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.507+0000 7fcd43fff640 1 --2- 172.21.15.157:0/3087976554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd44152b40 0x7fcd4414bd20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c007e90 tx=0x7fcd3c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:26.511 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.507+0000 7fcd2bfff640 1 -- 172.21.15.157:0/3087976554 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c014420 con 0x7fcd44152b40 2024-04-24T23:35:26.511 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.507+0000 7fcd2bfff640 1 -- 172.21.15.157:0/3087976554 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd3c019070 con 0x7fcd44152b40 2024-04-24T23:35:26.512 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.507+0000 7fcd4a1ed640 1 -- 172.21.15.157:0/3087976554 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd440ab270 con 0x7fcd44152b40 2024-04-24T23:35:26.512 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.507+0000 7fcd2bfff640 1 -- 172.21.15.157:0/3087976554 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c014800 con 0x7fcd44152b40 2024-04-24T23:35:26.512 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.507+0000 7fcd4a1ed640 1 -- 172.21.15.157:0/3087976554 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd440ab830 con 0x7fcd44152b40 2024-04-24T23:35:26.512 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.508+0000 7fcd4a1ed640 1 -- 172.21.15.157:0/3087976554 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd44069260 con 0x7fcd44152b40 2024-04-24T23:35:26.512 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.508+0000 7fcd2bfff640 1 -- 172.21.15.157:0/3087976554 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcd3c0149a0 con 0x7fcd44152b40 2024-04-24T23:35:26.512 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.509+0000 7fcd2bfff640 1 --2- 172.21.15.157:0/3087976554 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fcd3006be30 0x7fcd3006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:26.512 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.509+0000 7fcd2bfff640 1 -- 172.21.15.157:0/3087976554 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fcd3c08b830 con 0x7fcd44152b40 2024-04-24T23:35:26.512 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.509+0000 7fcd437fe640 1 --2- 172.21.15.157:0/3087976554 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fcd3006be30 0x7fcd3006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:26.512 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.510+0000 7fcd437fe640 1 --2- 172.21.15.157:0/3087976554 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fcd3006be30 0x7fcd3006e2f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fcd34034000 tx=0x7fcd3402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:26.514 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.513+0000 7fcd2bfff640 1 -- 172.21.15.157:0/3087976554 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcd44069260 con 0x7fcd44152b40 2024-04-24T23:35:26.723 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.722+0000 7fcd4a1ed640 1 -- 172.21.15.157:0/3087976554 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcd440681c0 con 0x7fcd44152b40 2024-04-24T23:35:26.723 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.722+0000 7fcd2bfff640 1 -- 172.21.15.157:0/3087976554 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5189 (secure 0 0 0) 0x7fcd3c059850 con 0x7fcd44152b40 2024-04-24T23:35:26.723 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:35:26.723 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1276438765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1276438765},{"type":"v1","addr":"172.21.15.169:6837","nonce":1276438765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1398599086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":1398599086},{"type":"v1","addr":"172.21.15.157:6835","nonce":1398599086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4258,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3795840339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3795840339},{"type":"v1","addr":"172.21.15.157:6834","nonce":3795840339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:34:53.701085+0000","modified":"2024-04-24T23:35:15.789166+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6836/3782746080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6835","nonce":3782746080},{"type":"v1","addr":"172.21.15.169:6836","nonce":3782746080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4270,"qdb_cluster":[4270]},"id":1}]} 2024-04-24T23:35:26.723 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 9 2024-04-24T23:35:26.725 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.724+0000 7fcd4a1ed640 1 -- 172.21.15.157:0/3087976554 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fcd3006be30 msgr2=0x7fcd3006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:26.725 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.724+0000 7fcd4a1ed640 1 --2- 172.21.15.157:0/3087976554 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fcd3006be30 0x7fcd3006e2f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fcd34034000 tx=0x7fcd3402e000 comp rx=0 tx=0).stop 2024-04-24T23:35:26.725 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.724+0000 7fcd4a1ed640 1 -- 172.21.15.157:0/3087976554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd44152b40 msgr2=0x7fcd4414bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:26.725 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.724+0000 7fcd4a1ed640 1 --2- 172.21.15.157:0/3087976554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd44152b40 0x7fcd4414bd20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c007e90 tx=0x7fcd3c00e4b0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.725 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.724+0000 7fcd4a1ed640 1 -- 172.21.15.157:0/3087976554 shutdown_connections 2024-04-24T23:35:26.725 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.724+0000 7fcd4a1ed640 1 --2- 172.21.15.157:0/3087976554 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fcd3006be30 0x7fcd3006e2f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.725 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.724+0000 7fcd4a1ed640 1 --2- 172.21.15.157:0/3087976554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd44152b40 0x7fcd4414bd20 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.726 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.724+0000 7fcd4a1ed640 1 --2- 172.21.15.157:0/3087976554 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd440c5b70 0x7fcd441ee460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.726 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.724+0000 7fcd4a1ed640 1 --2- 172.21.15.157:0/3087976554 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd440be360 0x7fcd441ec0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.726 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.724+0000 7fcd4a1ed640 1 -- 172.21.15.157:0/3087976554 >> 172.21.15.157:0/3087976554 conn(0x7fcd440672c0 msgr2=0x7fcd440c9af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:26.726 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.724+0000 7fcd4a1ed640 1 -- 172.21.15.157:0/3087976554 shutdown_connections 2024-04-24T23:35:26.726 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:26.724+0000 7fcd4a1ed640 1 -- 172.21.15.157:0/3087976554 wait complete. 2024-04-24T23:35:27.138 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T23:34:53.701085+0000', 'modified': '2024-04-24T23:35:15.789166+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4270}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4270': {'gid': 4270, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.169:6836/3782746080', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6835', 'nonce': 3782746080}, {'type': 'v1', 'addr': '172.21.15.169:6836', 'nonce': 3782746080}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4270, 'qdb_cluster': [4270]} 2024-04-24T23:35:27.138 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-24T23:35:27.138 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-24T23:35:27.237 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.236+0000 7f2e8f012640 1 Processor -- start 2024-04-24T23:35:27.237 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.236+0000 7f2e8f012640 1 -- start start 2024-04-24T23:35:27.238 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.237+0000 7f2e8f012640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e88072bc0 0x7f2e880c12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.238 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.237+0000 7f2e8f012640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2e88073110 0x7f2e88155790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.239 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.237+0000 7f2e8f012640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e88156240 con 0x7f2e8814f960 2024-04-24T23:35:27.239 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.237+0000 7f2e8f012640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f2e8806ed30 con 0x7f2e88073110 2024-04-24T23:35:27.239 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.237+0000 7f2e8f012640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2e88072470 con 0x7f2e88072bc0 2024-04-24T23:35:27.239 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.238+0000 7f2e87fff640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e88072bc0 0x7f2e880c12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.239 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.238+0000 7f2e8d5af640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2e88073110 0x7f2e88155790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.239 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.238+0000 7f2e87fff640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e88072bc0 0x7f2e880c12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.157:48678/0 (socket says 172.21.15.157:48678) 2024-04-24T23:35:27.239 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.238+0000 7f2e87fff640 1 -- 172.21.15.157:0/2400342458 learned_addr learned my addr 172.21.15.157:0/2400342458 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:27.239 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.238+0000 7f2e8cdae640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f2e8814f960 0x7f2e880a5ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.157:36876/0 (socket says 172.21.15.157:36876) 2024-04-24T23:35:27.239 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.238+0000 7f2e8d5af640 1 -- 172.21.15.157:0/2400342458 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e88072bc0 msgr2=0x7f2e880c12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:27.239 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.238+0000 7f2e8d5af640 1 --2- 172.21.15.157:0/2400342458 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e88072bc0 0x7f2e880c12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.239 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.238+0000 7f2e8d5af640 1 -- 172.21.15.157:0/2400342458 >> v1:172.21.15.169:6789/0 conn(0x7f2e8814f960 legacy=0x7f2e880a5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:27.239 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.238+0000 7f2e8d5af640 1 -- 172.21.15.157:0/2400342458 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e880a63f0 con 0x7f2e88073110 2024-04-24T23:35:27.239 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.238+0000 7f2e87fff640 1 --2- 172.21.15.157:0/2400342458 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e88072bc0 0x7f2e880c12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:35:27.241 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.239+0000 7f2e8d5af640 1 --2- 172.21.15.157:0/2400342458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e88073110 0x7f2e88155790 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2e80009d90 tx=0x7f2e8002a530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f99f112c8df4ab server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:27.241 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.239+0000 7f2e877fe640 1 -- 172.21.15.157:0/2400342458 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e80007740 con 0x7f2e88073110 2024-04-24T23:35:27.241 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.239+0000 7f2e877fe640 1 -- 172.21.15.157:0/2400342458 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e800078e0 con 0x7f2e88073110 2024-04-24T23:35:27.241 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.239+0000 7f2e877fe640 1 -- 172.21.15.157:0/2400342458 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e80007cb0 con 0x7f2e88073110 2024-04-24T23:35:27.241 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.239+0000 7f2e8f012640 1 -- 172.21.15.157:0/2400342458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e88073110 msgr2=0x7f2e88155790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:27.241 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.239+0000 7f2e8f012640 1 --2- 172.21.15.157:0/2400342458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e88073110 0x7f2e88155790 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2e80009d90 tx=0x7f2e8002a530 comp rx=0 tx=0).stop 2024-04-24T23:35:27.241 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.239+0000 7f2e8f012640 1 -- 172.21.15.157:0/2400342458 shutdown_connections 2024-04-24T23:35:27.241 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.239+0000 7f2e8f012640 1 --2- 172.21.15.157:0/2400342458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e88073110 0x7f2e88155790 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.242 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.239+0000 7f2e8f012640 1 --2- 172.21.15.157:0/2400342458 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e88072bc0 0x7f2e880c12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.242 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.239+0000 7f2e8f012640 1 -- 172.21.15.157:0/2400342458 >> 172.21.15.157:0/2400342458 conn(0x7f2e880a8690 msgr2=0x7f2e880a8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:27.242 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.239+0000 7f2e8f012640 1 -- 172.21.15.157:0/2400342458 shutdown_connections 2024-04-24T23:35:27.242 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.239+0000 7f2e8f012640 1 -- 172.21.15.157:0/2400342458 wait complete. 2024-04-24T23:35:27.242 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.240+0000 7f2e8f012640 1 Processor -- start 2024-04-24T23:35:27.242 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.241+0000 7f2e8f012640 1 -- start start 2024-04-24T23:35:27.244 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.241+0000 7f2e8f012640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e88072bc0 0x7f2e880c2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.244 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.241+0000 7f2e8f012640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e88073110 0x7f2e88152190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.244 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.241+0000 7f2e8f012640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e8814f960 0x7f2e881526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.245 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.241+0000 7f2e8f012640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2e880c4dd0 con 0x7f2e88072bc0 2024-04-24T23:35:27.245 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.241+0000 7f2e8f012640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2e880726c0 con 0x7f2e88073110 2024-04-24T23:35:27.245 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.241+0000 7f2e8f012640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2e880c4a60 con 0x7f2e8814f960 2024-04-24T23:35:27.245 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.241+0000 7f2e8cdae640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e88072bc0 0x7f2e880c2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.245 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.242+0000 7f2e8cdae640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e88072bc0 0x7f2e880c2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40564/0 (socket says 172.21.15.157:40564) 2024-04-24T23:35:27.245 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.242+0000 7f2e8cdae640 1 -- 172.21.15.157:0/2241649672 learned_addr learned my addr 172.21.15.157:0/2241649672 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:27.245 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.242+0000 7f2e8d5af640 1 --2- 172.21.15.157:0/2241649672 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e8814f960 0x7f2e881526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.245 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.242+0000 7f2e87fff640 1 --2- 172.21.15.157:0/2241649672 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e88073110 0x7f2e88152190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.245 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.242+0000 7f2e8cdae640 1 -- 172.21.15.157:0/2241649672 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e8814f960 msgr2=0x7f2e881526d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:27.245 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.242+0000 7f2e8cdae640 1 --2- 172.21.15.157:0/2241649672 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e8814f960 0x7f2e881526d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.246 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.242+0000 7f2e8cdae640 1 -- 172.21.15.157:0/2241649672 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e88073110 msgr2=0x7f2e88152190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:27.246 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.242+0000 7f2e8cdae640 1 --2- 172.21.15.157:0/2241649672 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e88073110 0x7f2e88152190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.246 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.242+0000 7f2e8cdae640 1 -- 172.21.15.157:0/2241649672 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e881e9100 con 0x7f2e88072bc0 2024-04-24T23:35:27.246 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.242+0000 7f2e8cdae640 1 --2- 172.21.15.157:0/2241649672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e88072bc0 0x7f2e880c2880 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2e7800c400 tx=0x7f2e7800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:27.246 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.242+0000 7f2e857fa640 1 -- 172.21.15.157:0/2241649672 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e780074e0 con 0x7f2e88072bc0 2024-04-24T23:35:27.246 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.242+0000 7f2e857fa640 1 -- 172.21.15.157:0/2241649672 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e78007680 con 0x7f2e88072bc0 2024-04-24T23:35:27.246 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.242+0000 7f2e8f012640 1 -- 172.21.15.157:0/2241649672 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e881e93e0 con 0x7f2e88072bc0 2024-04-24T23:35:27.246 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.242+0000 7f2e857fa640 1 -- 172.21.15.157:0/2241649672 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e78004a00 con 0x7f2e88072bc0 2024-04-24T23:35:27.246 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.242+0000 7f2e8f012640 1 -- 172.21.15.157:0/2241649672 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e881e99a0 con 0x7f2e88072bc0 2024-04-24T23:35:27.246 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.244+0000 7f2e857fa640 1 -- 172.21.15.157:0/2241649672 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2e7801a020 con 0x7f2e88072bc0 2024-04-24T23:35:27.253 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.244+0000 7f2e857fa640 1 --2- 172.21.15.157:0/2241649672 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f2e7406be30 0x7f2e7406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.253 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.244+0000 7f2e857fa640 1 -- 172.21.15.157:0/2241649672 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f2e7808a7e0 con 0x7f2e88072bc0 2024-04-24T23:35:27.253 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.245+0000 7f2e87fff640 1 --2- 172.21.15.157:0/2241649672 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f2e7406be30 0x7f2e7406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.253 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.245+0000 7f2e87fff640 1 --2- 172.21.15.157:0/2241649672 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f2e7406be30 0x7f2e7406e2f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2e880c3150 tx=0x7f2e7c009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:27.254 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.245+0000 7f2e91df7740 1 Processor -- start 2024-04-24T23:35:27.254 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.245+0000 7f2e91df7740 1 -- start start 2024-04-24T23:35:27.254 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.246+0000 7f2e91df7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573fdc98ad0 0x5573fdc95940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.254 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.246+0000 7f2e91df7740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5573fdc5b8e0 0x5573fdc95e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.254 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.246+0000 7f2e91df7740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5573fdc7c510 0x5573fdc96bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.254 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.246+0000 7f2e91df7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5573fdbbdf00 con 0x5573fdc98ad0 2024-04-24T23:35:27.254 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.246+0000 7f2e91df7740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5573fdb66750 con 0x5573fdc7c510 2024-04-24T23:35:27.254 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.246+0000 7f2e91df7740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5573fdbb9ec0 con 0x5573fdc5b8e0 2024-04-24T23:35:27.254 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.246+0000 7f2e8cdae640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573fdc98ad0 0x5573fdc95940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.254 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.246+0000 7f2e8cdae640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573fdc98ad0 0x5573fdc95940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40574/0 (socket says 172.21.15.157:40574) 2024-04-24T23:35:27.254 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.246+0000 7f2e8cdae640 1 -- 172.21.15.157:0/1608090270 learned_addr learned my addr 172.21.15.157:0/1608090270 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:27.254 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.246+0000 7f2e87fff640 1 --2- 172.21.15.157:0/1608090270 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5573fdc5b8e0 0x5573fdc95e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.255 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.246+0000 7f2e8d5af640 1 --2- 172.21.15.157:0/1608090270 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5573fdc7c510 0x5573fdc96bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.255 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.246+0000 7f2e8cdae640 1 -- 172.21.15.157:0/1608090270 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5573fdc5b8e0 msgr2=0x5573fdc95e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:27.255 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.246+0000 7f2e8cdae640 1 --2- 172.21.15.157:0/1608090270 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5573fdc5b8e0 0x5573fdc95e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.255 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.246+0000 7f2e8cdae640 1 -- 172.21.15.157:0/1608090270 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5573fdc7c510 msgr2=0x5573fdc96bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:27.255 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.246+0000 7f2e8cdae640 1 --2- 172.21.15.157:0/1608090270 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5573fdc7c510 0x5573fdc96bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.255 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.246+0000 7f2e8cdae640 1 -- 172.21.15.157:0/1608090270 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573fd8d22d0 con 0x5573fdc98ad0 2024-04-24T23:35:27.255 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.246+0000 7f2e8d5af640 1 --2- 172.21.15.157:0/1608090270 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5573fdc7c510 0x5573fdc96bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:35:27.255 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.247+0000 7f2e8cdae640 1 --2- 172.21.15.157:0/1608090270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573fdc98ad0 0x5573fdc95940 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2e7800cfd0 tx=0x7f2e7805ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15518e2ae55370f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:27.255 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.247+0000 7f2e6e7fc640 1 -- 172.21.15.157:0/1608090270 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e78056b90 con 0x5573fdc98ad0 2024-04-24T23:35:27.255 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.247+0000 7f2e6e7fc640 1 -- 172.21.15.157:0/1608090270 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e78054f00 con 0x5573fdc98ad0 2024-04-24T23:35:27.255 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.247+0000 7f2e6e7fc640 1 -- 172.21.15.157:0/1608090270 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e78058d30 con 0x5573fdc98ad0 2024-04-24T23:35:27.255 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.247+0000 7f2e91df7740 1 -- 172.21.15.157:0/1608090270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573fdc98ad0 msgr2=0x5573fdc95940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:27.255 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.247+0000 7f2e91df7740 1 --2- 172.21.15.157:0/1608090270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573fdc98ad0 0x5573fdc95940 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2e7800cfd0 tx=0x7f2e7805ccd0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.255 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.247+0000 7f2e91df7740 1 -- 172.21.15.157:0/1608090270 shutdown_connections 2024-04-24T23:35:27.256 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.247+0000 7f2e91df7740 1 --2- 172.21.15.157:0/1608090270 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5573fdc7c510 0x5573fdc96bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.256 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.247+0000 7f2e91df7740 1 --2- 172.21.15.157:0/1608090270 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5573fdc5b8e0 0x5573fdc95e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.256 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.247+0000 7f2e91df7740 1 --2- 172.21.15.157:0/1608090270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573fdc98ad0 0x5573fdc95940 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.256 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.247+0000 7f2e91df7740 1 -- 172.21.15.157:0/1608090270 >> 172.21.15.157:0/1608090270 conn(0x5573fdc57640 msgr2=0x5573fdc7be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:27.256 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.247+0000 7f2e91df7740 1 -- 172.21.15.157:0/1608090270 shutdown_connections 2024-04-24T23:35:27.256 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.247+0000 7f2e91df7740 1 -- 172.21.15.157:0/1608090270 wait complete. 2024-04-24T23:35:27.256 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.248+0000 7f2e91df7740 1 -- start start 2024-04-24T23:35:27.256 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.248+0000 7f2e91df7740 1 Processor -- start 2024-04-24T23:35:27.256 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.249+0000 7f2e91df7740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5573fdc7c510 0x5573fdc58e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.256 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.249+0000 7f2e91df7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573fdc98ad0 0x5573fdc93290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.256 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.249+0000 7f2e91df7740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5573fdc771e0 0x5573fdc937d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.256 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.249+0000 7f2e91df7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5573fdc5bdc0 con 0x5573fdc98ad0 2024-04-24T23:35:27.256 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.249+0000 7f2e91df7740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5573fdb79cf0 con 0x5573fdc771e0 2024-04-24T23:35:27.257 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.249+0000 7f2e91df7740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5573fdc954b0 con 0x5573fdc7c510 2024-04-24T23:35:27.257 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.249+0000 7f2e91df7740 10 client.0 _ll_register_callbacks cb 0x5573fdb796e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:35:27.257 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.249+0000 7f2e87fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573fdc98ad0 0x5573fdc93290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.257 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.249+0000 7f2e87fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573fdc98ad0 0x5573fdc93290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40582/0 (socket says 172.21.15.157:40582) 2024-04-24T23:35:27.257 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.249+0000 7f2e87fff640 1 -- 172.21.15.157:0/3777807680 learned_addr learned my addr 172.21.15.157:0/3777807680 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:27.257 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.249+0000 7f2e8d5af640 1 --2- 172.21.15.157:0/3777807680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5573fdc771e0 0x5573fdc937d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.257 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.249+0000 7f2e8cdae640 1 --2- 172.21.15.157:0/3777807680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5573fdc7c510 0x5573fdc58e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.257 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.249+0000 7f2e87fff640 1 -- 172.21.15.157:0/3777807680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5573fdc7c510 msgr2=0x5573fdc58e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:27.257 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.249+0000 7f2e87fff640 1 --2- 172.21.15.157:0/3777807680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5573fdc7c510 0x5573fdc58e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.257 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.249+0000 7f2e87fff640 1 -- 172.21.15.157:0/3777807680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5573fdc771e0 msgr2=0x5573fdc937d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:27.257 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.249+0000 7f2e87fff640 1 --2- 172.21.15.157:0/3777807680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5573fdc771e0 0x5573fdc937d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.257 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.249+0000 7f2e87fff640 1 -- 172.21.15.157:0/3777807680 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573fd99c6b0 con 0x5573fdc98ad0 2024-04-24T23:35:27.257 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.249+0000 7f2e87fff640 1 --2- 172.21.15.157:0/3777807680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573fdc98ad0 0x5573fdc93290 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c0149f0 tx=0x7f2e7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26951b3cf40ac182 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:27.257 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.249+0000 7f2e677fe640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-04-24T23:35:27.258 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.249+0000 7f2e677fe640 1 -- 172.21.15.157:0/3777807680 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c027030 con 0x5573fdc98ad0 2024-04-24T23:35:27.258 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.249+0000 7f2e677fe640 1 -- 172.21.15.157:0/3777807680 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f2e7c00f040 con 0x5573fdc98ad0 2024-04-24T23:35:27.258 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.249+0000 7f2e91df7740 1 -- 172.21.15.157:0/3777807680 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e78054f00 con 0x5573fdc98ad0 2024-04-24T23:35:27.258 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.250+0000 7f2e677fe640 1 -- 172.21.15.157:0/3777807680 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f2e7c033050 con 0x5573fdc98ad0 2024-04-24T23:35:27.258 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.250+0000 7f2e91df7740 10 client.4803 fetch_fsmap learned FSMap version 9 2024-04-24T23:35:27.258 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.250+0000 7f2e91df7740 1 -- 172.21.15.157:0/3777807680 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x5573fdc5a8b0 con 0x5573fdc98ad0 2024-04-24T23:35:27.258 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.250+0000 7f2e677fe640 1 -- 172.21.15.157:0/3777807680 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e78054f00 con 0x5573fdc98ad0 2024-04-24T23:35:27.258 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.250+0000 7f2e677fe640 1 -- 172.21.15.157:0/3777807680 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7f2e7c036020 con 0x5573fdc98ad0 2024-04-24T23:35:27.258 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.250+0000 7f2e91df7740 10 client.4803 fetch_fsmap finished waiting for FSMap version 9 2024-04-24T23:35:27.258 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.250+0000 7f2e91df7740 10 client.4803 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T23:35:27.258 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.250+0000 7f2e91df7740 20 client.4803 populate_metadata read hostname 'smithi157' 2024-04-24T23:35:27.258 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.250+0000 7f2e91df7740 1 --2- 172.21.15.157:0/3777807680 >> [v2:172.21.15.169:6835/3782746080,v1:172.21.15.169:6836/3782746080] conn(0x5573fdc93d10 0x5573fdcb5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.258 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.250+0000 7f2e91df7740 4 client.4803 mds_command: new command op to 4270 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:35:27.258 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.250+0000 7f2e91df7740 1 -- 172.21.15.157:0/3777807680 --> [v2:172.21.15.169:6835/3782746080,v1:172.21.15.169:6836/3782746080] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573fdbcb440 con 0x5573fdc93d10 2024-04-24T23:35:27.259 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.250+0000 7f2e8cdae640 1 --2- 172.21.15.157:0/3777807680 >> [v2:172.21.15.169:6835/3782746080,v1:172.21.15.169:6836/3782746080] conn(0x5573fdc93d10 0x5573fdcb5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.259 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.250+0000 7f2e677fe640 10 client.4803 ms_handle_connect on v2:172.21.15.169:6835/3782746080 2024-04-24T23:35:27.259 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.250+0000 7f2e8cdae640 1 --2- 172.21.15.157:0/3777807680 >> [v2:172.21.15.169:6835/3782746080,v1:172.21.15.169:6836/3782746080] conn(0x5573fdc93d10 0x5573fdcb5bd0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6781850b32c495e server_cookie=365831dad7023559 in_seq=0 out_seq=0 2024-04-24T23:35:27.259 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.251+0000 7f2e677fe640 1 -- 172.21.15.157:0/3777807680 <== mds.0 v2:172.21.15.169:6835/3782746080 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5573fdbcb440 con 0x5573fdc93d10 2024-04-24T23:35:27.259 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.251+0000 7f2e677fe640 10 client.4803 handle_command_reply: tid=0 2024-04-24T23:35:27.259 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.251+0000 7f2e91df7740 1 client.4803 shutdown 2024-04-24T23:35:27.259 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.251+0000 7f2e91df7740 1 -- 172.21.15.157:0/3777807680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573fdc98ad0 msgr2=0x5573fdc93290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:27.259 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.251+0000 7f2e91df7740 1 --2- 172.21.15.157:0/3777807680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573fdc98ad0 0x5573fdc93290 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c0149f0 tx=0x7f2e7c0046c0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.259 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.251+0000 7f2e91df7740 1 -- 172.21.15.157:0/3777807680 shutdown_connections 2024-04-24T23:35:27.259 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.251+0000 7f2e91df7740 1 --2- 172.21.15.157:0/3777807680 >> [v2:172.21.15.169:6835/3782746080,v1:172.21.15.169:6836/3782746080] conn(0x5573fdc93d10 0x5573fdcb5bd0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.259 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.251+0000 7f2e91df7740 1 --2- 172.21.15.157:0/3777807680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5573fdc771e0 0x5573fdc937d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.259 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.251+0000 7f2e91df7740 1 --2- 172.21.15.157:0/3777807680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573fdc98ad0 0x5573fdc93290 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.259 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.251+0000 7f2e677fe640 0 client.4803 ms_handle_reset on v2:172.21.15.169:6835/3782746080 2024-04-24T23:35:27.259 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.251+0000 7f2e91df7740 1 --2- 172.21.15.157:0/3777807680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5573fdc7c510 0x5573fdc58e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.259 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.251+0000 7f2e91df7740 1 -- 172.21.15.157:0/3777807680 >> 172.21.15.157:0/3777807680 conn(0x5573fdc5b8e0 msgr2=0x5573fdc5bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:27.260 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.251+0000 7f2e91df7740 1 -- 172.21.15.157:0/3777807680 shutdown_connections 2024-04-24T23:35:27.260 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.251+0000 7f2e91df7740 1 -- 172.21.15.157:0/3777807680 wait complete. 2024-04-24T23:35:27.260 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.252+0000 7f2e91df7740 20 client.4803 trim_cache size 0 max 16384 2024-04-24T23:35:27.270 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.269+0000 7f2e91df7740 1 Processor -- start 2024-04-24T23:35:27.270 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.269+0000 7f2e91df7740 1 -- start start 2024-04-24T23:35:27.273 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.270+0000 7f2e91df7740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5573fdc771e0 0x5573fdc56550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.273 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.270+0000 7f2e91df7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573fdc7c510 0x5573fdc98090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.273 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.270+0000 7f2e91df7740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5573fdc98ad0 0x5573fdcbac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.273 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.270+0000 7f2e91df7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2e881f5f30 con 0x5573fdc7c510 2024-04-24T23:35:27.273 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.270+0000 7f2e91df7740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2e881f5290 con 0x5573fdc771e0 2024-04-24T23:35:27.273 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.270+0000 7f2e91df7740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2e881f58e0 con 0x5573fdc98ad0 2024-04-24T23:35:27.273 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.270+0000 7f2e87fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573fdc7c510 0x5573fdc98090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.273 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.270+0000 7f2e87fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573fdc7c510 0x5573fdc98090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40586/0 (socket says 172.21.15.157:40586) 2024-04-24T23:35:27.273 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.270+0000 7f2e87fff640 1 -- 172.21.15.157:0/461386219 learned_addr learned my addr 172.21.15.157:0/461386219 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:27.273 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.270+0000 7f2e8d5af640 1 --2- 172.21.15.157:0/461386219 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5573fdc98ad0 0x5573fdcbac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.274 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.270+0000 7f2e8cdae640 1 --2- 172.21.15.157:0/461386219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5573fdc771e0 0x5573fdc56550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.274 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.271+0000 7f2e87fff640 1 -- 172.21.15.157:0/461386219 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5573fdc98ad0 msgr2=0x5573fdcbac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:27.274 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.271+0000 7f2e87fff640 1 --2- 172.21.15.157:0/461386219 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5573fdc98ad0 0x5573fdcbac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.274 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.271+0000 7f2e87fff640 1 -- 172.21.15.157:0/461386219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5573fdc771e0 msgr2=0x5573fdc56550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:27.274 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.271+0000 7f2e87fff640 1 --2- 172.21.15.157:0/461386219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5573fdc771e0 0x5573fdc56550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.274 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.271+0000 7f2e87fff640 1 -- 172.21.15.157:0/461386219 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573fdc76a40 con 0x5573fdc7c510 2024-04-24T23:35:27.274 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.271+0000 7f2e87fff640 1 --2- 172.21.15.157:0/461386219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573fdc7c510 0x5573fdc98090 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c022230 tx=0x7f2e7c01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d196c34e3b8907ab server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:27.274 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.271+0000 7f2e6dffb640 1 -- 172.21.15.157:0/461386219 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c03c070 con 0x5573fdc7c510 2024-04-24T23:35:27.274 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.271+0000 7f2e6dffb640 1 -- 172.21.15.157:0/461386219 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e7c01b070 con 0x5573fdc7c510 2024-04-24T23:35:27.274 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.271+0000 7f2e6dffb640 1 -- 172.21.15.157:0/461386219 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c027040 con 0x5573fdc7c510 2024-04-24T23:35:27.274 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.271+0000 7f2e91df7740 1 -- 172.21.15.157:0/461386219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573fdc7c510 msgr2=0x5573fdc98090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:27.275 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.271+0000 7f2e91df7740 1 --2- 172.21.15.157:0/461386219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573fdc7c510 0x5573fdc98090 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c022230 tx=0x7f2e7c01b000 comp rx=0 tx=0).stop 2024-04-24T23:35:27.275 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.271+0000 7f2e91df7740 1 -- 172.21.15.157:0/461386219 shutdown_connections 2024-04-24T23:35:27.275 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.271+0000 7f2e91df7740 1 --2- 172.21.15.157:0/461386219 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5573fdc98ad0 0x5573fdcbac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.275 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.271+0000 7f2e91df7740 1 --2- 172.21.15.157:0/461386219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573fdc7c510 0x5573fdc98090 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.275 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.271+0000 7f2e91df7740 1 --2- 172.21.15.157:0/461386219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5573fdc771e0 0x5573fdc56550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.275 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.271+0000 7f2e91df7740 1 -- 172.21.15.157:0/461386219 >> 172.21.15.157:0/461386219 conn(0x5573fdc5b8e0 msgr2=0x5573fdc935c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:27.275 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.271+0000 7f2e91df7740 1 -- 172.21.15.157:0/461386219 shutdown_connections 2024-04-24T23:35:27.275 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.271+0000 7f2e91df7740 1 -- 172.21.15.157:0/461386219 wait complete. 2024-04-24T23:35:27.275 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.272+0000 7f2e91df7740 1 -- start start 2024-04-24T23:35:27.275 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.272+0000 7f2e91df7740 1 Processor -- start 2024-04-24T23:35:27.279 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.273+0000 7f2e91df7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573fdc771e0 0x5573fdc7a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.279 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.273+0000 7f2e91df7740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5573fdc7c510 0x5573fdd14b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.279 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.273+0000 7f2e91df7740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5573fdc98ad0 0x5573fdd0ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.279 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.273+0000 7f2e91df7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5573fdc96550 con 0x5573fdc771e0 2024-04-24T23:35:27.279 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.273+0000 7f2e91df7740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5573fdc598d0 con 0x5573fdc7c510 2024-04-24T23:35:27.279 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.273+0000 7f2e91df7740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5573fdcaa700 con 0x5573fdc98ad0 2024-04-24T23:35:27.279 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.273+0000 7f2e91df7740 10 client.0 _ll_register_callbacks cb 0x7f2e7c014670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:35:27.279 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.273+0000 7f2e8cdae640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573fdc771e0 0x5573fdc7a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.280 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.273+0000 7f2e8cdae640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573fdc771e0 0x5573fdc7a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40600/0 (socket says 172.21.15.157:40600) 2024-04-24T23:35:27.280 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.273+0000 7f2e8cdae640 1 -- 172.21.15.157:0/2900610497 learned_addr learned my addr 172.21.15.157:0/2900610497 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:27.280 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.273+0000 7f2e87fff640 1 --2- 172.21.15.157:0/2900610497 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5573fdc7c510 0x5573fdd14b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.280 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.273+0000 7f2e8d5af640 1 --2- 172.21.15.157:0/2900610497 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5573fdc98ad0 0x5573fdd0ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.280 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.273+0000 7f2e8cdae640 1 -- 172.21.15.157:0/2900610497 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5573fdc98ad0 msgr2=0x5573fdd0ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:27.280 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.273+0000 7f2e8cdae640 1 --2- 172.21.15.157:0/2900610497 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5573fdc98ad0 0x5573fdd0ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.280 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.273+0000 7f2e8cdae640 1 -- 172.21.15.157:0/2900610497 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5573fdc7c510 msgr2=0x5573fdd14b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:27.280 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.273+0000 7f2e8cdae640 1 --2- 172.21.15.157:0/2900610497 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5573fdc7c510 0x5573fdd14b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.280 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.273+0000 7f2e8cdae640 1 -- 172.21.15.157:0/2900610497 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573fdc8ddb0 con 0x5573fdc771e0 2024-04-24T23:35:27.280 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.273+0000 7f2e8cdae640 1 --2- 172.21.15.157:0/2900610497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573fdc771e0 0x5573fdc7a5d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2e78000fb0 tx=0x7f2e78000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d08b24c77c0e1e92 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:27.280 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.273+0000 7f2e677fe640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-04-24T23:35:27.280 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.273+0000 7f2e677fe640 1 -- 172.21.15.157:0/2900610497 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e78054b30 con 0x5573fdc771e0 2024-04-24T23:35:27.280 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.273+0000 7f2e677fe640 1 -- 172.21.15.157:0/2900610497 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f2e78058af0 con 0x5573fdc771e0 2024-04-24T23:35:27.281 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.273+0000 7f2e91df7740 1 -- 172.21.15.157:0/2900610497 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e7c01b070 con 0x5573fdc771e0 2024-04-24T23:35:27.281 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.274+0000 7f2e677fe640 1 -- 172.21.15.157:0/2900610497 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f2e78054f00 con 0x5573fdc771e0 2024-04-24T23:35:27.281 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.274+0000 7f2e91df7740 10 client.4809 fetch_fsmap learned FSMap version 9 2024-04-24T23:35:27.281 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.274+0000 7f2e91df7740 1 -- 172.21.15.157:0/2900610497 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x5573fdca46c0 con 0x5573fdc771e0 2024-04-24T23:35:27.281 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.274+0000 7f2e677fe640 1 -- 172.21.15.157:0/2900610497 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e7c01b070 con 0x5573fdc771e0 2024-04-24T23:35:27.281 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.274+0000 7f2e677fe640 1 -- 172.21.15.157:0/2900610497 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7f2e78060f30 con 0x5573fdc771e0 2024-04-24T23:35:27.281 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.274+0000 7f2e91df7740 10 client.4809 fetch_fsmap finished waiting for FSMap version 9 2024-04-24T23:35:27.281 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.274+0000 7f2e91df7740 10 client.4809 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T23:35:27.281 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.274+0000 7f2e91df7740 20 client.4809 populate_metadata read hostname 'smithi157' 2024-04-24T23:35:27.281 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.274+0000 7f2e91df7740 1 --2- 172.21.15.157:0/2900610497 >> [v2:172.21.15.169:6835/3782746080,v1:172.21.15.169:6836/3782746080] conn(0x5573fdca3600 0x5573fdc95420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.282 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.274+0000 7f2e91df7740 4 client.4809 mds_command: new command op to 4270 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:35:27.282 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.274+0000 7f2e91df7740 1 -- 172.21.15.157:0/2900610497 --> [v2:172.21.15.169:6835/3782746080,v1:172.21.15.169:6836/3782746080] -- command(tid 0: {"prefix": "status"}) -- 0x5573fdbc5790 con 0x5573fdca3600 2024-04-24T23:35:27.282 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.274+0000 7f2e87fff640 1 --2- 172.21.15.157:0/2900610497 >> [v2:172.21.15.169:6835/3782746080,v1:172.21.15.169:6836/3782746080] conn(0x5573fdca3600 0x5573fdc95420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.282 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.274+0000 7f2e677fe640 10 client.4809 ms_handle_connect on v2:172.21.15.169:6835/3782746080 2024-04-24T23:35:27.282 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.274+0000 7f2e87fff640 1 --2- 172.21.15.157:0/2900610497 >> [v2:172.21.15.169:6835/3782746080,v1:172.21.15.169:6836/3782746080] conn(0x5573fdca3600 0x5573fdc95420 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=156d8fab4230a1b5 server_cookie=5f9634587e35d3ba in_seq=0 out_seq=0 2024-04-24T23:35:27.282 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.275+0000 7f2e677fe640 1 -- 172.21.15.157:0/2900610497 <== mds.0 v2:172.21.15.169:6835/3782746080 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x5573fdbc5790 con 0x5573fdca3600 2024-04-24T23:35:27.282 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.275+0000 7f2e677fe640 10 client.4809 handle_command_reply: tid=0 2024-04-24T23:35:27.282 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.275+0000 7f2e91df7740 1 client.4809 shutdown 2024-04-24T23:35:27.282 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.275+0000 7f2e91df7740 1 -- 172.21.15.157:0/2900610497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573fdc771e0 msgr2=0x5573fdc7a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:27.282 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.275+0000 7f2e91df7740 1 --2- 172.21.15.157:0/2900610497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573fdc771e0 0x5573fdc7a5d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2e78000fb0 tx=0x7f2e78000bc0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.282 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.275+0000 7f2e91df7740 1 -- 172.21.15.157:0/2900610497 shutdown_connections 2024-04-24T23:35:27.282 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.275+0000 7f2e91df7740 1 --2- 172.21.15.157:0/2900610497 >> [v2:172.21.15.169:6835/3782746080,v1:172.21.15.169:6836/3782746080] conn(0x5573fdca3600 0x5573fdc95420 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.283 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.275+0000 7f2e91df7740 1 --2- 172.21.15.157:0/2900610497 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5573fdc98ad0 0x5573fdd0ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.283 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.275+0000 7f2e91df7740 1 --2- 172.21.15.157:0/2900610497 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5573fdc7c510 0x5573fdd14b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.283 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.275+0000 7f2e677fe640 0 client.4809 ms_handle_reset on v2:172.21.15.169:6835/3782746080 2024-04-24T23:35:27.283 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.275+0000 7f2e91df7740 1 --2- 172.21.15.157:0/2900610497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573fdc771e0 0x5573fdc7a5d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.283 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.275+0000 7f2e91df7740 1 -- 172.21.15.157:0/2900610497 >> 172.21.15.157:0/2900610497 conn(0x5573fdc5b8e0 msgr2=0x5573fdc56550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:27.283 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.275+0000 7f2e91df7740 1 -- 172.21.15.157:0/2900610497 shutdown_connections 2024-04-24T23:35:27.283 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.275+0000 7f2e91df7740 1 -- 172.21.15.157:0/2900610497 wait complete. 2024-04-24T23:35:27.283 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.275+0000 7f2e91df7740 20 client.4809 trim_cache size 0 max 16384 2024-04-24T23:35:27.283 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-04-24T23:35:27.283 INFO:teuthology.orchestra.run.smithi157.stdout: "cluster_fsid": "223685fb-5b64-46a5-9857-3791c9c07df2", 2024-04-24T23:35:27.284 INFO:teuthology.orchestra.run.smithi157.stdout: "whoami": 0, 2024-04-24T23:35:27.284 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 4270, 2024-04-24T23:35:27.284 INFO:teuthology.orchestra.run.smithi157.stdout: "want_state": "up:active", 2024-04-24T23:35:27.284 INFO:teuthology.orchestra.run.smithi157.stdout: "state": "up:active", 2024-04-24T23:35:27.284 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs", 2024-04-24T23:35:27.284 INFO:teuthology.orchestra.run.smithi157.stdout: "rank_uptime": 32.916870885999998, 2024-04-24T23:35:27.284 INFO:teuthology.orchestra.run.smithi157.stdout: "mdsmap_epoch": 9, 2024-04-24T23:35:27.284 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_epoch": 25, 2024-04-24T23:35:27.284 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_epoch_barrier": 25, 2024-04-24T23:35:27.284 INFO:teuthology.orchestra.run.smithi157.stdout: "uptime": 39.200286806999998 2024-04-24T23:35:27.284 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-04-24T23:35:27.285 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.275+0000 7f2e8f012640 1 -- 172.21.15.157:0/2241649672 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f2e7406be30 msgr2=0x7f2e7406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:27.285 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.275+0000 7f2e8f012640 1 --2- 172.21.15.157:0/2241649672 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f2e7406be30 0x7f2e7406e2f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2e880c3150 tx=0x7f2e7c009240 comp rx=0 tx=0).stop 2024-04-24T23:35:27.285 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.275+0000 7f2e8f012640 1 -- 172.21.15.157:0/2241649672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e88072bc0 msgr2=0x7f2e880c2880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:27.285 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.275+0000 7f2e8f012640 1 --2- 172.21.15.157:0/2241649672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e88072bc0 0x7f2e880c2880 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2e7800c400 tx=0x7f2e7800c8c0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.285 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.276+0000 7f2e8f012640 1 -- 172.21.15.157:0/2241649672 shutdown_connections 2024-04-24T23:35:27.286 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.276+0000 7f2e8f012640 1 --2- 172.21.15.157:0/2241649672 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f2e7406be30 0x7f2e7406e2f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.286 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.276+0000 7f2e8f012640 1 --2- 172.21.15.157:0/2241649672 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e8814f960 0x7f2e881526d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.286 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.276+0000 7f2e8f012640 1 --2- 172.21.15.157:0/2241649672 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e88073110 0x7f2e88152190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.286 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.276+0000 7f2e8f012640 1 --2- 172.21.15.157:0/2241649672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e88072bc0 0x7f2e880c2880 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.286 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.276+0000 7f2e8f012640 1 -- 172.21.15.157:0/2241649672 >> 172.21.15.157:0/2241649672 conn(0x7f2e880a8690 msgr2=0x7f2e880c57e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:27.286 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.276+0000 7f2e8f012640 1 -- 172.21.15.157:0/2241649672 shutdown_connections 2024-04-24T23:35:27.286 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.276+0000 7f2e8f012640 1 -- 172.21.15.157:0/2241649672 wait complete. 2024-04-24T23:35:27.287 INFO:teuthology.run_tasks:Running task sequential... 2024-04-24T23:35:27.297 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-24T23:35:27.297 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-24T23:35:27.297 INFO:teuthology.task.print:Enabling mgr modules 2024-04-24T23:35:27.297 INFO:teuthology.run_tasks:Running task parallel... 2024-04-24T23:35:27.341 INFO:teuthology.task.parallel:starting parallel... 2024-04-24T23:35:27.341 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-04-24T23:35:27.341 INFO:teuthology.task.sequential:In sequential, running task ceph-fuse... 2024-04-24T23:35:27.342 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-24T23:35:27.342 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-04-24T23:35:27.343 INFO:tasks.ceph_fuse:client.0 config is {} 2024-04-24T23:35:27.343 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-24T23:35:27.343 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T23:35:27.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:35:27.343 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-04-24T23:35:27.343 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-04-24T23:35:27.344 INFO:teuthology.task.sequential:In sequential, running task kclient... 2024-04-24T23:35:27.347 INFO:tasks.kclient:Mounting kernel clients... 2024-04-24T23:35:27.348 INFO:tasks.kclient:config is {'client.1': {}} 2024-04-24T23:35:27.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:35:27.348 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2024-04-24T23:35:27.363 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-24T23:35:27.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:35:27.364 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-24T23:35:27.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:35:27.366 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link delete ceph-brx 2024-04-24T23:35:27.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:35:27.427 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link delete ceph-brx 2024-04-24T23:35:27.437 INFO:teuthology.orchestra.run.smithi169.stderr:Cannot find device "ceph-brx" 2024-04-24T23:35:27.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:35:27.439 INFO:tasks.kclient:client.1 config is {} 2024-04-24T23:35:27.439 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-24T23:35:27.439 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-24T23:35:27.439 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T23:35:27.544 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-24T23:35:27.544 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T23:35:27.568 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.568+0000 7f0be394f640 1 Processor -- start 2024-04-24T23:35:27.568 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.568+0000 7f0be394f640 1 -- start start 2024-04-24T23:35:27.571 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.568+0000 7f0be394f640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0bdc14b950 0x7f0bdc0c2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.571 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.568+0000 7f0be394f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0bdc151ca0 0x7f0bdc1520c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.571 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.568+0000 7f0be394f640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bdc14bd30 con 0x7f0bdc154e60 2024-04-24T23:35:27.572 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.568+0000 7f0be394f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f0bdc06c480 con 0x7f0bdc151ca0 2024-04-24T23:35:27.572 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.568+0000 7f0be394f640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f0bdc1552f0 con 0x7f0bdc14b950 2024-04-24T23:35:27.572 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.569+0000 7f0be1eec640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0bdc151ca0 0x7f0bdc1520c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.572 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.569+0000 7f0be1eec640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0bdc151ca0 0x7f0bdc1520c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40614/0 (socket says 172.21.15.157:40614) 2024-04-24T23:35:27.572 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.569+0000 7f0be1eec640 1 -- 172.21.15.157:0/2246837733 learned_addr learned my addr 172.21.15.157:0/2246837733 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:27.572 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.569+0000 7f0be16eb640 1 --2- 172.21.15.157:0/2246837733 >> v2:172.21.15.169:3300/0 conn(0x7f0bdc14b950 0x7f0bdc0c2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.572 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.569+0000 7f0be1eec640 1 -- 172.21.15.157:0/2246837733 >> v2:172.21.15.169:3300/0 conn(0x7f0bdc14b950 msgr2=0x7f0bdc0c2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:27.572 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.569+0000 7f0be1eec640 1 --2- 172.21.15.157:0/2246837733 >> v2:172.21.15.169:3300/0 conn(0x7f0bdc14b950 0x7f0bdc0c2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.572 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.569+0000 7f0be1eec640 1 -- 172.21.15.157:0/2246837733 >> v1:172.21.15.169:6789/0 conn(0x7f0bdc154e60 legacy=0x7f0bdc0c5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:27.572 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.569+0000 7f0be1eec640 1 -- 172.21.15.157:0/2246837733 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bdc1555d0 con 0x7f0bdc151ca0 2024-04-24T23:35:27.572 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.569+0000 7f0be16eb640 1 --2- 172.21.15.157:0/2246837733 >> v2:172.21.15.169:3300/0 conn(0x7f0bdc14b950 0x7f0bdc0c2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:35:27.572 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.570+0000 7f0be1eec640 1 --2- 172.21.15.157:0/2246837733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0bdc151ca0 0x7f0bdc1520c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc0c2830 tx=0x7f0bd402a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab62dc8921709797 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:27.572 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.570+0000 7f0bd3fff640 1 -- 172.21.15.157:0/2246837733 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd4036070 con 0x7f0bdc151ca0 2024-04-24T23:35:27.572 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.570+0000 7f0bd3fff640 1 -- 172.21.15.157:0/2246837733 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bd4031600 con 0x7f0bdc151ca0 2024-04-24T23:35:27.573 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.570+0000 7f0bd3fff640 1 -- 172.21.15.157:0/2246837733 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd40319d0 con 0x7f0bdc151ca0 2024-04-24T23:35:27.573 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.571+0000 7f0be394f640 1 -- 172.21.15.157:0/2246837733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0bdc151ca0 msgr2=0x7f0bdc1520c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:27.573 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.571+0000 7f0be394f640 1 --2- 172.21.15.157:0/2246837733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0bdc151ca0 0x7f0bdc1520c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc0c2830 tx=0x7f0bd402a630 comp rx=0 tx=0).stop 2024-04-24T23:35:27.573 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.571+0000 7f0be394f640 1 -- 172.21.15.157:0/2246837733 shutdown_connections 2024-04-24T23:35:27.573 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.571+0000 7f0be394f640 1 --2- 172.21.15.157:0/2246837733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0bdc151ca0 0x7f0bdc1520c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.573 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.571+0000 7f0be394f640 1 --2- 172.21.15.157:0/2246837733 >> v2:172.21.15.169:3300/0 conn(0x7f0bdc14b950 0x7f0bdc0c2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.573 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.571+0000 7f0be394f640 1 -- 172.21.15.157:0/2246837733 >> 172.21.15.157:0/2246837733 conn(0x7f0bdc0a8720 msgr2=0x7f0bdc0a8b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:27.573 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.571+0000 7f0be394f640 1 -- 172.21.15.157:0/2246837733 shutdown_connections 2024-04-24T23:35:27.573 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.571+0000 7f0be394f640 1 -- 172.21.15.157:0/2246837733 wait complete. 2024-04-24T23:35:27.575 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.572+0000 7f0be394f640 1 Processor -- start 2024-04-24T23:35:27.575 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.572+0000 7f0be394f640 1 -- start start 2024-04-24T23:35:27.575 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.573+0000 7f0be394f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0bdc14b950 0x7f0bdc149310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.576 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.573+0000 7f0be394f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bdc151ca0 0x7f0bdc0728f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.576 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.573+0000 7f0be394f640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0bdc153ca0 0x7f0bdc14f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.576 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.573+0000 7f0be394f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0bdc0c5810 con 0x7f0bdc14b950 2024-04-24T23:35:27.576 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.573+0000 7f0be394f640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0bdc155b90 con 0x7f0bdc151ca0 2024-04-24T23:35:27.576 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.573+0000 7f0be394f640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0bdc0c5990 con 0x7f0bdc153ca0 2024-04-24T23:35:27.576 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.573+0000 7f0be16eb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0bdc14b950 0x7f0bdc149310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.576 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.573+0000 7f0be16eb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0bdc14b950 0x7f0bdc149310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40622/0 (socket says 172.21.15.157:40622) 2024-04-24T23:35:27.576 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.573+0000 7f0be16eb640 1 -- 172.21.15.157:0/1925449038 learned_addr learned my addr 172.21.15.157:0/1925449038 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:27.576 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.573+0000 7f0be1eec640 1 --2- 172.21.15.157:0/1925449038 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0bdc153ca0 0x7f0bdc14f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.576 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.573+0000 7f0be0eea640 1 --2- 172.21.15.157:0/1925449038 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bdc151ca0 0x7f0bdc0728f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.576 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.573+0000 7f0be16eb640 1 -- 172.21.15.157:0/1925449038 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0bdc153ca0 msgr2=0x7f0bdc14f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:27.576 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.573+0000 7f0be16eb640 1 --2- 172.21.15.157:0/1925449038 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0bdc153ca0 0x7f0bdc14f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.576 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.573+0000 7f0be16eb640 1 -- 172.21.15.157:0/1925449038 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bdc151ca0 msgr2=0x7f0bdc0728f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:27.577 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.573+0000 7f0be16eb640 1 --2- 172.21.15.157:0/1925449038 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bdc151ca0 0x7f0bdc0728f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.577 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.573+0000 7f0be16eb640 1 -- 172.21.15.157:0/1925449038 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bdc1455a0 con 0x7f0bdc14b950 2024-04-24T23:35:27.577 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.573+0000 7f0be16eb640 1 --2- 172.21.15.157:0/1925449038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0bdc14b950 0x7f0bdc149310 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0bd800c7f0 tx=0x7f0bd800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:27.577 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.573+0000 7f0bd1ffb640 1 -- 172.21.15.157:0/1925449038 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd8015070 con 0x7f0bdc14b950 2024-04-24T23:35:27.577 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.573+0000 7f0bd1ffb640 1 -- 172.21.15.157:0/1925449038 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bd8007820 con 0x7f0bdc14b950 2024-04-24T23:35:27.577 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.573+0000 7f0bd1ffb640 1 -- 172.21.15.157:0/1925449038 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd8007bd0 con 0x7f0bdc14b950 2024-04-24T23:35:27.577 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.573+0000 7f0be394f640 1 -- 172.21.15.157:0/1925449038 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bdc154840 con 0x7f0bdc14b950 2024-04-24T23:35:27.577 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.574+0000 7f0be394f640 1 -- 172.21.15.157:0/1925449038 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bdc153780 con 0x7f0bdc14b950 2024-04-24T23:35:27.577 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.575+0000 7f0be394f640 1 -- 172.21.15.157:0/1925449038 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bdc0681c0 con 0x7f0bdc14b950 2024-04-24T23:35:27.577 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.575+0000 7f0bd1ffb640 1 -- 172.21.15.157:0/1925449038 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0bd8018070 con 0x7f0bdc14b950 2024-04-24T23:35:27.578 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.575+0000 7f0bd1ffb640 1 --2- 172.21.15.157:0/1925449038 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f0bc006be60 0x7f0bc006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.578 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.575+0000 7f0bd1ffb640 1 -- 172.21.15.157:0/1925449038 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f0bd808b160 con 0x7f0bdc14b950 2024-04-24T23:35:27.578 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.575+0000 7f0be0eea640 1 --2- 172.21.15.157:0/1925449038 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f0bc006be60 0x7f0bc006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.578 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.576+0000 7f0be0eea640 1 --2- 172.21.15.157:0/1925449038 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f0bc006be60 0x7f0bc006e320 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc06cb10 tx=0x7f0bcc007e30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:27.580 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.580+0000 7f0bd1ffb640 1 -- 172.21.15.157:0/1925449038 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0bd8059210 con 0x7f0bdc14b950 2024-04-24T23:35:27.668 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.668+0000 7f583c31b640 1 Processor -- start 2024-04-24T23:35:27.669 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.668+0000 7f583c31b640 1 -- start start 2024-04-24T23:35:27.670 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.669+0000 7f583c31b640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58340c5850 0x7f58340c3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.670 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.669+0000 7f583c31b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f58340c7690 0x7f58341559d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.670 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.669+0000 7f583c31b640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58340c42d0 con 0x7f5834070950 2024-04-24T23:35:27.670 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.669+0000 7f583c31b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f5834159580 con 0x7f58340c7690 2024-04-24T23:35:27.670 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.669+0000 7f583c31b640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5834159860 con 0x7f58340c5850 2024-04-24T23:35:27.671 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.669+0000 7f583a8b8640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f58340c7690 0x7f58341559d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.671 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.669+0000 7f583a8b8640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f58340c7690 0x7f58341559d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40636/0 (socket says 172.21.15.157:40636) 2024-04-24T23:35:27.671 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.669+0000 7f583a8b8640 1 -- 172.21.15.157:0/1696864358 learned_addr learned my addr 172.21.15.157:0/1696864358 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:27.671 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.669+0000 7f583a0b7640 1 --2- 172.21.15.157:0/1696864358 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58340c5850 0x7f58340c3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.671 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.669+0000 7f583a8b8640 1 -- 172.21.15.157:0/1696864358 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58340c5850 msgr2=0x7f58340c3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:27.671 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.669+0000 7f583a8b8640 1 --2- 172.21.15.157:0/1696864358 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58340c5850 0x7f58340c3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.671 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.669+0000 7f583a8b8640 1 -- 172.21.15.157:0/1696864358 >> v1:172.21.15.157:6789/0 conn(0x7f5834070950 legacy=0x7f58340c6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:27.671 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.669+0000 7f583a8b8640 1 -- 172.21.15.157:0/1696864358 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58340c85a0 con 0x7f58340c7690 2024-04-24T23:35:27.672 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.670+0000 7f583a8b8640 1 --2- 172.21.15.157:0/1696864358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58340c7690 0x7f58341559d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f582c009e50 tx=0x7f582c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd44301b9a570b6d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:27.673 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.670+0000 7f58390b5640 1 -- 172.21.15.157:0/1696864358 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582c036070 con 0x7f58340c7690 2024-04-24T23:35:27.673 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.670+0000 7f58390b5640 1 -- 172.21.15.157:0/1696864358 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f582c031600 con 0x7f58340c7690 2024-04-24T23:35:27.673 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.670+0000 7f58390b5640 1 -- 172.21.15.157:0/1696864358 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582c0319d0 con 0x7f58340c7690 2024-04-24T23:35:27.673 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.671+0000 7f583c31b640 1 -- 172.21.15.157:0/1696864358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58340c7690 msgr2=0x7f58341559d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:27.673 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.671+0000 7f583c31b640 1 --2- 172.21.15.157:0/1696864358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58340c7690 0x7f58341559d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f582c009e50 tx=0x7f582c02a680 comp rx=0 tx=0).stop 2024-04-24T23:35:27.673 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.671+0000 7f583c31b640 1 -- 172.21.15.157:0/1696864358 shutdown_connections 2024-04-24T23:35:27.673 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.671+0000 7f583c31b640 1 --2- 172.21.15.157:0/1696864358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58340c7690 0x7f58341559d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.673 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.671+0000 7f583c31b640 1 --2- 172.21.15.157:0/1696864358 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58340c5850 0x7f58340c3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.673 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.671+0000 7f583c31b640 1 -- 172.21.15.157:0/1696864358 >> 172.21.15.157:0/1696864358 conn(0x7f58340c8ba0 msgr2=0x7f58340c8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:27.673 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.671+0000 7f583c31b640 1 -- 172.21.15.157:0/1696864358 shutdown_connections 2024-04-24T23:35:27.673 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.671+0000 7f583c31b640 1 -- 172.21.15.157:0/1696864358 wait complete. 2024-04-24T23:35:27.673 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.672+0000 7f583c31b640 1 Processor -- start 2024-04-24T23:35:27.674 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.672+0000 7f583c31b640 1 -- start start 2024-04-24T23:35:27.676 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.672+0000 7f583c31b640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5834070950 0x7f58341578c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.676 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.673+0000 7f583c31b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58340c5850 0x7f5834158460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.676 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.673+0000 7f583c31b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58340c7690 0x7f58341def80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.676 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.673+0000 7f583c31b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f58340c7c80 con 0x7f58340c5850 2024-04-24T23:35:27.676 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.673+0000 7f583c31b640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f58340c7b00 con 0x7f58340c7690 2024-04-24T23:35:27.676 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.673+0000 7f583c31b640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f58340c5c30 con 0x7f5834070950 2024-04-24T23:35:27.677 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.673+0000 7f58398b6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58340c5850 0x7f5834158460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.677 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.673+0000 7f58398b6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58340c5850 0x7f5834158460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40652/0 (socket says 172.21.15.157:40652) 2024-04-24T23:35:27.677 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.673+0000 7f58398b6640 1 -- 172.21.15.157:0/3037027804 learned_addr learned my addr 172.21.15.157:0/3037027804 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:35:27.677 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.673+0000 7f583a0b7640 1 --2- 172.21.15.157:0/3037027804 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5834070950 0x7f58341578c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.677 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.673+0000 7f583a8b8640 1 --2- 172.21.15.157:0/3037027804 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58340c7690 0x7f58341def80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.677 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.673+0000 7f58398b6640 1 -- 172.21.15.157:0/3037027804 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5834070950 msgr2=0x7f58341578c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:27.677 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.673+0000 7f58398b6640 1 --2- 172.21.15.157:0/3037027804 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5834070950 0x7f58341578c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.677 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.673+0000 7f58398b6640 1 -- 172.21.15.157:0/3037027804 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58340c7690 msgr2=0x7f58341def80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:27.677 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.673+0000 7f58398b6640 1 --2- 172.21.15.157:0/3037027804 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58340c7690 0x7f58341def80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.677 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.673+0000 7f58398b6640 1 -- 172.21.15.157:0/3037027804 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5834157e00 con 0x7f58340c5850 2024-04-24T23:35:27.677 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.673+0000 7f58398b6640 1 --2- 172.21.15.157:0/3037027804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58340c5850 0x7f5834158460 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f583000c440 tx=0x7f5830001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:27.677 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.673+0000 7f582affd640 1 -- 172.21.15.157:0/3037027804 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58300052d0 con 0x7f58340c5850 2024-04-24T23:35:27.677 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.673+0000 7f582affd640 1 -- 172.21.15.157:0/3037027804 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5830005470 con 0x7f58340c5850 2024-04-24T23:35:27.678 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.673+0000 7f583c31b640 1 -- 172.21.15.157:0/3037027804 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58341e2410 con 0x7f58340c5850 2024-04-24T23:35:27.678 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.673+0000 7f582affd640 1 -- 172.21.15.157:0/3037027804 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5830007260 con 0x7f58340c5850 2024-04-24T23:35:27.678 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.673+0000 7f583c31b640 1 -- 172.21.15.157:0/3037027804 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58341e2110 con 0x7f58340c5850 2024-04-24T23:35:27.678 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.675+0000 7f583c31b640 1 -- 172.21.15.157:0/3037027804 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5834156ed0 con 0x7f58340c5850 2024-04-24T23:35:27.678 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.675+0000 7f582affd640 1 -- 172.21.15.157:0/3037027804 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5830007400 con 0x7f58340c5850 2024-04-24T23:35:27.678 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.675+0000 7f582affd640 1 --2- 172.21.15.157:0/3037027804 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f581806be30 0x7f581806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.678 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.675+0000 7f582affd640 1 -- 172.21.15.157:0/3037027804 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f583008a620 con 0x7f58340c5850 2024-04-24T23:35:27.678 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.675+0000 7f583a0b7640 1 --2- 172.21.15.157:0/3037027804 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f581806be30 0x7f581806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.678 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.676+0000 7f583a0b7640 1 --2- 172.21.15.157:0/3037027804 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f581806be30 0x7f581806e2f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5824006fd0 tx=0x7f5824008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:27.680 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.680+0000 7f582affd640 1 -- 172.21.15.157:0/3037027804 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5834157e00 con 0x7f58340c5850 2024-04-24T23:35:27.802 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.802+0000 7f0be394f640 1 -- 172.21.15.157:0/1925449038 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0bdc0c6620 con 0x7f0bdc14b950 2024-04-24T23:35:27.803 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.802+0000 7f0bd1ffb640 1 -- 172.21.15.157:0/1925449038 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f0bd805d510 con 0x7f0bdc14b950 2024-04-24T23:35:27.803 INFO:teuthology.orchestra.run.smithi157.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-24T23:35:27.805 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.804+0000 7f0be394f640 1 -- 172.21.15.157:0/1925449038 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f0bc006be60 msgr2=0x7f0bc006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:27.805 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.804+0000 7f0be394f640 1 --2- 172.21.15.157:0/1925449038 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f0bc006be60 0x7f0bc006e320 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc06cb10 tx=0x7f0bcc007e30 comp rx=0 tx=0).stop 2024-04-24T23:35:27.806 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.804+0000 7f0be394f640 1 -- 172.21.15.157:0/1925449038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0bdc14b950 msgr2=0x7f0bdc149310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:27.806 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.804+0000 7f0be394f640 1 --2- 172.21.15.157:0/1925449038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0bdc14b950 0x7f0bdc149310 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0bd800c7f0 tx=0x7f0bd800ccb0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.806 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.804+0000 7f0be394f640 1 -- 172.21.15.157:0/1925449038 shutdown_connections 2024-04-24T23:35:27.806 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.804+0000 7f0be394f640 1 --2- 172.21.15.157:0/1925449038 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f0bc006be60 0x7f0bc006e320 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.806 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.804+0000 7f0be394f640 1 --2- 172.21.15.157:0/1925449038 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0bdc153ca0 0x7f0bdc14f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.806 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.804+0000 7f0be394f640 1 --2- 172.21.15.157:0/1925449038 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bdc151ca0 0x7f0bdc0728f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.806 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.804+0000 7f0be394f640 1 --2- 172.21.15.157:0/1925449038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0bdc14b950 0x7f0bdc149310 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.806 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.804+0000 7f0be394f640 1 -- 172.21.15.157:0/1925449038 >> 172.21.15.157:0/1925449038 conn(0x7f0bdc0a8720 msgr2=0x7f0bdc0c25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:27.806 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.804+0000 7f0be394f640 1 -- 172.21.15.157:0/1925449038 shutdown_connections 2024-04-24T23:35:27.806 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.804+0000 7f0be394f640 1 -- 172.21.15.157:0/1925449038 wait complete. 2024-04-24T23:35:27.816 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-24T23:35:27.816 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T23:35:27.816 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi169.front.sepia.ceph.com 2024-04-24T23:35:27.816 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-24T23:35:27.816 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-24T23:35:27.816 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-24T23:35:27.817 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T23:35:27.817 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T23:35:27.817 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-24T23:35:27.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:35:27.817 DEBUG:teuthology.orchestra.run.smithi169:> ip addr 2024-04-24T23:35:27.836 INFO:teuthology.orchestra.run.smithi169.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T23:35:27.836 INFO:teuthology.orchestra.run.smithi169.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T23:35:27.836 INFO:teuthology.orchestra.run.smithi169.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T23:35:27.836 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:35:27.836 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 ::1/128 scope host 2024-04-24T23:35:27.836 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:35:27.836 INFO:teuthology.orchestra.run.smithi169.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:35:27.836 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:d9:3b:b4 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:35:27.836 INFO:teuthology.orchestra.run.smithi169.stdout: altname enp7s0f0 2024-04-24T23:35:27.836 INFO:teuthology.orchestra.run.smithi169.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:35:27.836 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:d9:3b:b5 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:35:27.836 INFO:teuthology.orchestra.run.smithi169.stdout: altname enp7s0f1 2024-04-24T23:35:27.837 INFO:teuthology.orchestra.run.smithi169.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:35:27.837 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:88:7f:4c brd ff:ff:ff:ff:ff:ff 2024-04-24T23:35:27.837 INFO:teuthology.orchestra.run.smithi169.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T23:35:27.837 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:88:7f:4d brd ff:ff:ff:ff:ff:ff 2024-04-24T23:35:27.837 INFO:teuthology.orchestra.run.smithi169.stdout: inet 172.21.15.169/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-24T23:35:27.837 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft 42683sec preferred_lft 42683sec 2024-04-24T23:35:27.837 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::616f:8a78:350b:aba0/64 scope link noprefixroute 2024-04-24T23:35:27.837 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:35:27.837 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-24T23:35:27.837 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:35:27.837 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2024-04-24T23:35:27.838 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link add name ceph-brx type bridge 2024-04-24T23:35:27.838 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip addr flush dev ceph-brx 2024-04-24T23:35:27.838 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link set ceph-brx up 2024-04-24T23:35:27.838 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-24T23:35:27.838 DEBUG:teuthology.orchestra.run.smithi169:> ') 2024-04-24T23:35:27.899 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.898+0000 7f583c31b640 1 -- 172.21.15.157:0/3037027804 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f58340c90b0 con 0x7f58340c5850 2024-04-24T23:35:27.899 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.899+0000 7f582affd640 1 -- 172.21.15.157:0/3037027804 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f5830058d20 con 0x7f58340c5850 2024-04-24T23:35:27.899 INFO:teuthology.orchestra.run.smithi157.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-24T23:35:27.902 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.901+0000 7f583c31b640 1 -- 172.21.15.157:0/3037027804 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f581806be30 msgr2=0x7f581806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:27.902 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.901+0000 7f583c31b640 1 --2- 172.21.15.157:0/3037027804 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f581806be30 0x7f581806e2f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5824006fd0 tx=0x7f5824008040 comp rx=0 tx=0).stop 2024-04-24T23:35:27.902 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.901+0000 7f583c31b640 1 -- 172.21.15.157:0/3037027804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58340c5850 msgr2=0x7f5834158460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:27.902 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.901+0000 7f583c31b640 1 --2- 172.21.15.157:0/3037027804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58340c5850 0x7f5834158460 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f583000c440 tx=0x7f5830001e70 comp rx=0 tx=0).stop 2024-04-24T23:35:27.902 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.901+0000 7f583c31b640 1 -- 172.21.15.157:0/3037027804 shutdown_connections 2024-04-24T23:35:27.902 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.901+0000 7f583c31b640 1 --2- 172.21.15.157:0/3037027804 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f581806be30 0x7f581806e2f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.902 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.901+0000 7f583c31b640 1 --2- 172.21.15.157:0/3037027804 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58340c7690 0x7f58341def80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.902 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.901+0000 7f583c31b640 1 --2- 172.21.15.157:0/3037027804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58340c5850 0x7f5834158460 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.902 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.901+0000 7f583c31b640 1 --2- 172.21.15.157:0/3037027804 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5834070950 0x7f58341578c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.902 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.901+0000 7f583c31b640 1 -- 172.21.15.157:0/3037027804 >> 172.21.15.157:0/3037027804 conn(0x7f58340c8ba0 msgr2=0x7f5834155340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:27.902 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.901+0000 7f583c31b640 1 -- 172.21.15.157:0/3037027804 shutdown_connections 2024-04-24T23:35:27.902 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:27.901+0000 7f583c31b640 1 -- 172.21.15.157:0/3037027804 wait complete. 2024-04-24T23:35:27.912 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-24T23:35:27.912 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T23:35:27.912 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2024-04-24T23:35:27.912 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-24T23:35:27.912 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T23:35:27.912 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-24T23:35:27.912 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T23:35:27.912 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T23:35:27.913 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:35:27.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:35:27.913 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2024-04-24T23:35:27.928 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T23:35:27.928 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T23:35:27.928 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T23:35:27.928 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:35:27.928 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-04-24T23:35:27.928 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:35:27.928 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:35:27.928 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:35:27.928 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2024-04-24T23:35:27.929 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:35:27.929 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:35:27.929 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2024-04-24T23:35:27.929 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:35:27.929 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:35:27.929 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T23:35:27.929 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:35:27.929 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-24T23:35:27.929 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 42717sec preferred_lft 42717sec 2024-04-24T23:35:27.929 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::3a09:adda:a6cf:8530/64 scope link noprefixroute 2024-04-24T23:35:27.929 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:35:27.929 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-24T23:35:27.930 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:35:27.930 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-24T23:35:27.930 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link add name ceph-brx type bridge 2024-04-24T23:35:27.930 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip addr flush dev ceph-brx 2024-04-24T23:35:27.930 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set ceph-brx up 2024-04-24T23:35:27.930 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-24T23:35:27.930 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-24T23:35:27.931 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:35:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:35:28.015 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:35:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:35:28.016 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:35:28.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:35:28.022 DEBUG:teuthology.orchestra.run.smithi169:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T23:35:28.047 INFO:teuthology.orchestra.run.smithi169.stdout:1 2024-04-24T23:35:28.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:35:28.049 DEBUG:teuthology.orchestra.run.smithi169:> route 2024-04-24T23:35:28.055 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:35:28.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:35:28.061 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T23:35:28.083 INFO:teuthology.orchestra.run.smithi157.stdout:1 2024-04-24T23:35:28.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:35:28.085 DEBUG:teuthology.orchestra.run.smithi157:> route 2024-04-24T23:35:28.111 INFO:teuthology.orchestra.run.smithi169.stdout:Kernel IP routing table 2024-04-24T23:35:28.111 INFO:teuthology.orchestra.run.smithi169.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T23:35:28.111 INFO:teuthology.orchestra.run.smithi169.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-24T23:35:28.111 INFO:teuthology.orchestra.run.smithi169.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-24T23:35:28.112 INFO:teuthology.orchestra.run.smithi169.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T23:35:28.112 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:35:28.112 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2024-04-24T23:35:28.112 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-24T23:35:28.112 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-24T23:35:28.112 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-24T23:35:28.112 DEBUG:teuthology.orchestra.run.smithi169:> ') 2024-04-24T23:35:28.140 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2024-04-24T23:35:28.140 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T23:35:28.140 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-24T23:35:28.140 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-24T23:35:28.140 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T23:35:28.140 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:35:28.140 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-24T23:35:28.140 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-24T23:35:28.141 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-24T23:35:28.141 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-24T23:35:28.141 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-24T23:35:28.200 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:35:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:35:28.228 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:35:28.258 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:35:28.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:35:28.264 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-04-24T23:35:28.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:35:28.279 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list-id 2024-04-24T23:35:28.334 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:35:28.334 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-24T23:35:28.334 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T23:35:28.334 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-24T23:35:28.334 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-24T23:35:28.368 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:35:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:35:28.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:35:28.375 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2024-04-24T23:35:28.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:35:28.390 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list-id 2024-04-24T23:35:28.422 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:35:28.443 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:35:28.446 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:35:28.446 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2024-04-24T23:35:28.446 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-24T23:35:28.446 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-24T23:35:28.446 DEBUG:teuthology.orchestra.run.smithi169:> ') 2024-04-24T23:35:28.449 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-24T23:35:28.449 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:35:28.450 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-24T23:35:28.450 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-24T23:35:28.450 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-24T23:35:28.450 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-24T23:35:28.450 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-24T23:35:28.450 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-24T23:35:28.450 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-24T23:35:28.498 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:35:28.534 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:35:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:35:28.552 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:35:28.557 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:35:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:35:28.558 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:35:28.559 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-24T23:35:28.559 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2024-04-24T23:35:28.559 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set dev brx.0 master ceph-brx 2024-04-24T23:35:28.559 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-24T23:35:28.564 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-24T23:35:28.564 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:35:28.564 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2024-04-24T23:35:28.564 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-24T23:35:28.564 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-24T23:35:28.564 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-24T23:35:28.564 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-24T23:35:28.564 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-24T23:35:28.564 DEBUG:teuthology.orchestra.run.smithi169:> ') 2024-04-24T23:35:28.606 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:35:28.612 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:35:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:35:28.627 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:35:28.633 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-24T23:35:28.633 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:35:28.633 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-24T23:35:28.648 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:35:28.648 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:35:28.648 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-24T23:35:28.682 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:35:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:35:28.690 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:35:28.690 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2024-04-24T23:35:28.690 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link set brx.0 up 2024-04-24T23:35:28.690 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link set dev brx.0 master ceph-brx 2024-04-24T23:35:28.690 DEBUG:teuthology.orchestra.run.smithi169:> ') 2024-04-24T23:35:28.700 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2024-04-24T23:35:28.738 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:35:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:35:28.759 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:35:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:35:28.764 DEBUG:teuthology.orchestra.run.smithi157:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T23:35:28.765 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:35:28.765 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-24T23:35:28.780 INFO:teuthology.orchestra.run.smithi169.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-24T23:35:28.781 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:35:28.781 DEBUG:teuthology.orchestra.run.smithi169:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-24T23:35:28.820 INFO:teuthology.orchestra.run.smithi157.stdout:/proc 2024-04-24T23:35:28.820 INFO:teuthology.orchestra.run.smithi157.stdout:/sys 2024-04-24T23:35:28.820 INFO:teuthology.orchestra.run.smithi157.stdout:/dev 2024-04-24T23:35:28.820 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/security 2024-04-24T23:35:28.820 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/shm 2024-04-24T23:35:28.820 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/pts 2024-04-24T23:35:28.820 INFO:teuthology.orchestra.run.smithi157.stdout:/run 2024-04-24T23:35:28.820 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/cgroup 2024-04-24T23:35:28.820 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/pstore 2024-04-24T23:35:28.820 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/bpf 2024-04-24T23:35:28.820 INFO:teuthology.orchestra.run.smithi157.stdout:/ 2024-04-24T23:35:28.820 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/selinux 2024-04-24T23:35:28.821 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T23:35:28.821 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/hugepages 2024-04-24T23:35:28.821 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/mqueue 2024-04-24T23:35:28.821 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/debug 2024-04-24T23:35:28.821 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/tracing 2024-04-24T23:35:28.821 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/config 2024-04-24T23:35:28.821 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-sysusers.service 2024-04-24T23:35:28.821 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/fuse/connections 2024-04-24T23:35:28.821 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T23:35:28.821 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T23:35:28.821 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T23:35:28.821 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T23:35:28.821 INFO:teuthology.orchestra.run.smithi157.stdout:/run/user/1000 2024-04-24T23:35:28.822 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph 2024-04-24T23:35:28.822 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-0 2024-04-24T23:35:28.822 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-1 2024-04-24T23:35:28.822 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-2 2024-04-24T23:35:28.822 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-3 2024-04-24T23:35:28.822 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns 2024-04-24T23:35:28.822 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T23:35:28.822 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T23:35:28.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:35:28.822 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2024-04-24T23:35:28.836 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-04-24T23:35:28.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:35:28.837 DEBUG:teuthology.orchestra.run.smithi169:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:35:28.875 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-24T23:35:28.875 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-24T23:35:28.917 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2024-04-24T23:35:28.927 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:35:28.927 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-24T23:35:28.940 DEBUG:teuthology.orchestra.run.smithi157:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T23:35:28.996 INFO:teuthology.orchestra.run.smithi157.stdout:/proc 2024-04-24T23:35:28.996 INFO:teuthology.orchestra.run.smithi157.stdout:/sys 2024-04-24T23:35:28.996 INFO:teuthology.orchestra.run.smithi157.stdout:/dev 2024-04-24T23:35:28.996 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/security 2024-04-24T23:35:28.996 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/shm 2024-04-24T23:35:28.996 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/pts 2024-04-24T23:35:28.996 INFO:teuthology.orchestra.run.smithi157.stdout:/run 2024-04-24T23:35:28.996 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/cgroup 2024-04-24T23:35:28.996 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/pstore 2024-04-24T23:35:28.996 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/bpf 2024-04-24T23:35:28.996 INFO:teuthology.orchestra.run.smithi157.stdout:/ 2024-04-24T23:35:28.996 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/selinux 2024-04-24T23:35:28.996 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T23:35:28.996 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/hugepages 2024-04-24T23:35:28.997 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/mqueue 2024-04-24T23:35:28.997 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/debug 2024-04-24T23:35:28.997 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/tracing 2024-04-24T23:35:28.997 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/config 2024-04-24T23:35:28.997 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-sysusers.service 2024-04-24T23:35:28.997 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/fuse/connections 2024-04-24T23:35:28.997 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T23:35:28.997 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T23:35:28.997 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T23:35:28.997 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T23:35:28.997 INFO:teuthology.orchestra.run.smithi157.stdout:/run/user/1000 2024-04-24T23:35:28.997 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph 2024-04-24T23:35:28.997 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-0 2024-04-24T23:35:28.997 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-1 2024-04-24T23:35:28.997 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-2 2024-04-24T23:35:28.998 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-3 2024-04-24T23:35:28.998 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns 2024-04-24T23:35:28.998 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T23:35:28.998 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T23:35:28.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:35:28.998 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2024-04-24T23:35:28.999 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi157.stderr:2024-04-24T23:35:28.998+0000 7f338a7d7480 -1 init, newargv = 0x556765d2ebc0 newargc=15 2024-04-24T23:35:28.999 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi157.stderr:ceph-fuse[42972]: starting ceph client 2024-04-24T23:35:29.002 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi157.stderr:ceph-fuse[42972]: starting fuse 2024-04-24T23:35:29.013 INFO:teuthology.orchestra.run.smithi157.stdout:43 2024-04-24T23:35:29.013 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-04-24T23:35:29.014 DEBUG:teuthology.orchestra.run.smithi157:> sudo stdin-killer -- python3 -c ' 2024-04-24T23:35:29.014 DEBUG:teuthology.orchestra.run.smithi157:> import glob 2024-04-24T23:35:29.014 DEBUG:teuthology.orchestra.run.smithi157:> import re 2024-04-24T23:35:29.014 DEBUG:teuthology.orchestra.run.smithi157:> import os 2024-04-24T23:35:29.014 DEBUG:teuthology.orchestra.run.smithi157:> import subprocess 2024-04-24T23:35:29.014 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-24T23:35:29.014 DEBUG:teuthology.orchestra.run.smithi157:> def _find_admin_socket(client_name): 2024-04-24T23:35:29.015 DEBUG:teuthology.orchestra.run.smithi157:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-24T23:35:29.015 DEBUG:teuthology.orchestra.run.smithi157:> files = glob.glob(asok_path) 2024-04-24T23:35:29.015 DEBUG:teuthology.orchestra.run.smithi157:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-24T23:35:29.015 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-24T23:35:29.015 DEBUG:teuthology.orchestra.run.smithi157:> # Given a non-glob path, it better be there 2024-04-24T23:35:29.015 DEBUG:teuthology.orchestra.run.smithi157:> if "*" not in asok_path: 2024-04-24T23:35:29.015 DEBUG:teuthology.orchestra.run.smithi157:> assert(len(files) == 1) 2024-04-24T23:35:29.015 DEBUG:teuthology.orchestra.run.smithi157:> return files[0] 2024-04-24T23:35:29.015 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-24T23:35:29.015 DEBUG:teuthology.orchestra.run.smithi157:> for f in files: 2024-04-24T23:35:29.015 DEBUG:teuthology.orchestra.run.smithi157:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-24T23:35:29.015 DEBUG:teuthology.orchestra.run.smithi157:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-24T23:35:29.015 DEBUG:teuthology.orchestra.run.smithi157:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-24T23:35:29.015 DEBUG:teuthology.orchestra.run.smithi157:> contents = proc_f.read() 2024-04-24T23:35:29.015 DEBUG:teuthology.orchestra.run.smithi157:> if mountpoint in contents: 2024-04-24T23:35:29.016 DEBUG:teuthology.orchestra.run.smithi157:> return f 2024-04-24T23:35:29.016 DEBUG:teuthology.orchestra.run.smithi157:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-24T23:35:29.016 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-24T23:35:29.016 DEBUG:teuthology.orchestra.run.smithi157:> print(_find_admin_socket("client.0")) 2024-04-24T23:35:29.016 DEBUG:teuthology.orchestra.run.smithi157:> ' 2024-04-24T23:35:29.032 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:35:29.032 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-24T23:35:29.032 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: resolved to: "172.21.15.157:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-24T23:35:29.032 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: trying mount with new device syntax: 1@223685fb-5b64-46a5-9857-3791c9c07df2.cephfs=/ 2024-04-24T23:35:29.032 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.157:3300/172.21.15.169:3300/172.21.15.169:3301" will pass to kernel 2024-04-24T23:35:29.032 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:35:29.033 INFO:teuthology.orchestra.run.smithi169.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-24T23:35:29.033 INFO:teuthology.orchestra.run.smithi169.stdout: You just mounted a file system that supports labels which does not 2024-04-24T23:35:29.033 INFO:teuthology.orchestra.run.smithi169.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-24T23:35:29.033 INFO:teuthology.orchestra.run.smithi169.stdout: applications will generate AVC messages and not be allowed access to 2024-04-24T23:35:29.033 INFO:teuthology.orchestra.run.smithi169.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-24T23:35:29.035 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-24T23:35:29.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:35:29.035 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-24T23:35:29.056 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:35:29.059 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:35:29.131 INFO:teuthology.orchestra.run.smithi157.stdout:/var/run/ceph/ceph-client.0.42972.asok 2024-04-24T23:35:29.134 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:35:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:35:29.141 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.42972.asok 2024-04-24T23:35:29.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:35:29.142 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.42972.asok status 2024-04-24T23:35:29.147 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:35:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:35:29.149 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:35:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:35:29.154 INFO:teuthology.orchestra.run.smithi169.stdout:client4756 2024-04-24T23:35:29.154 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-24T23:35:29.155 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:35:29.218 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:35:29.273 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-04-24T23:35:29.274 INFO:teuthology.orchestra.run.smithi157.stdout: "metadata": { 2024-04-24T23:35:29.274 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", 2024-04-24T23:35:29.274 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-24T23:35:29.274 INFO:teuthology.orchestra.run.smithi157.stdout: "entity_id": "0", 2024-04-24T23:35:29.274 INFO:teuthology.orchestra.run.smithi157.stdout: "hostname": "smithi157", 2024-04-24T23:35:29.274 INFO:teuthology.orchestra.run.smithi157.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-24T23:35:29.274 INFO:teuthology.orchestra.run.smithi157.stdout: "pid": "42972", 2024-04-24T23:35:29.274 INFO:teuthology.orchestra.run.smithi157.stdout: "root": "/" 2024-04-24T23:35:29.274 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-24T23:35:29.274 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_count": 0, 2024-04-24T23:35:29.274 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_pinned_count": 0, 2024-04-24T23:35:29.274 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 4830, 2024-04-24T23:35:29.275 INFO:teuthology.orchestra.run.smithi157.stdout: "inst": { 2024-04-24T23:35:29.275 INFO:teuthology.orchestra.run.smithi157.stdout: "name": { 2024-04-24T23:35:29.275 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "client", 2024-04-24T23:35:29.275 INFO:teuthology.orchestra.run.smithi157.stdout: "num": 4830 2024-04-24T23:35:29.275 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-24T23:35:29.275 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2024-04-24T23:35:29.275 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2024-04-24T23:35:29.275 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2024-04-24T23:35:29.275 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 2866666380 2024-04-24T23:35:29.275 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-24T23:35:29.275 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-24T23:35:29.275 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2024-04-24T23:35:29.275 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2024-04-24T23:35:29.275 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2024-04-24T23:35:29.276 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 2866666380 2024-04-24T23:35:29.276 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-24T23:35:29.276 INFO:teuthology.orchestra.run.smithi157.stdout: "inst_str": "client.4830 192.168.0.1:0/2866666380", 2024-04-24T23:35:29.276 INFO:teuthology.orchestra.run.smithi157.stdout: "addr_str": "192.168.0.1:0/2866666380", 2024-04-24T23:35:29.276 INFO:teuthology.orchestra.run.smithi157.stdout: "inode_count": 1, 2024-04-24T23:35:29.276 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_epoch": 9, 2024-04-24T23:35:29.276 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch": 25, 2024-04-24T23:35:29.276 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch_barrier": 0, 2024-04-24T23:35:29.276 INFO:teuthology.orchestra.run.smithi157.stdout: "blocklisted": false, 2024-04-24T23:35:29.276 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs" 2024-04-24T23:35:29.276 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-04-24T23:35:29.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:35:29.282 DEBUG:teuthology.orchestra.run.smithi157:> stat --file-system '--printf=%T 2024-04-24T23:35:29.282 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-24T23:35:29.297 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2024-04-24T23:35:29.297 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-24T23:35:29.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:35:29.298 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-24T23:35:29.308 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:35:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:35:29.309 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:35:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:35:29.315 INFO:teuthology.orchestra.run.smithi169.stdout:client4756 2024-04-24T23:35:29.315 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:35:29.361 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-04-24T23:35:29.364 INFO:tasks.workunit:Pulling workunits from ref d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:35:29.365 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-24T23:35:29.365 DEBUG:teuthology.orchestra.run.smithi157:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-24T23:35:29.365 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:35:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:35:29.366 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:35:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:35:29.370 INFO:teuthology.orchestra.run.smithi169.stdout:client4756 2024-04-24T23:35:29.370 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/223685fb-5b64-46a5-9857-3791c9c07df2.client4756/status') 2024-04-24T23:35:29.417 INFO:teuthology.orchestra.run.smithi157.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-24T23:35:29.417 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-04-24T23:35:29.417 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 2bh/43d Inode: 1 Links: 2 2024-04-24T23:35:29.417 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:35:29.417 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:fusefs_t:s0 2024-04-24T23:35:29.417 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-04-24T23:35:29.417 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-04-24 23:34:54.360336260 +0000 2024-04-24T23:35:29.417 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-04-24 23:35:03.672751808 +0000 2024-04-24T23:35:29.417 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: - 2024-04-24T23:35:29.418 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-24T23:35:29.418 DEBUG:teuthology.orchestra.run.smithi157:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-24T23:35:29.418 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:35:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:35:29.428 INFO:teuthology.orchestra.run.smithi169.stdout:instance: client.4756 (3)192.168.0.1:0/2662475946 2024-04-24T23:35:29.428 INFO:teuthology.orchestra.run.smithi169.stdout:blocklisted: false 2024-04-24T23:35:29.428 INFO:teuthology.orchestra.run.smithi169.stderr:0+1 records in 2024-04-24T23:35:29.428 INFO:teuthology.orchestra.run.smithi169.stderr:0+1 records out 2024-04-24T23:35:29.428 INFO:teuthology.orchestra.run.smithi169.stderr:69 bytes copied, 3.8457e-05 s, 1.8 MB/s 2024-04-24T23:35:29.429 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-24T23:35:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:35:29.435 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-04-24T23:35:29.436 INFO:tasks.workunit:Pulling workunits from ref d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:35:29.436 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-24T23:35:29.436 DEBUG:teuthology.orchestra.run.smithi169:> stat -- /home/ubuntu/cephtest/mnt.1 2024-04-24T23:35:29.451 INFO:teuthology.orchestra.run.smithi169.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-04-24T23:35:29.451 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-24T23:35:29.451 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 2bh/43d Inode: 1 Links: 2 2024-04-24T23:35:29.451 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:35:29.451 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-24T23:35:29.451 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-04-24T23:35:29.451 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-04-24 23:34:54.360336260 +0000 2024-04-24T23:35:29.451 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-04-24 23:35:03.672751808 +0000 2024-04-24T23:35:29.452 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-04-24 23:34:54.360336260 +0000 2024-04-24T23:35:29.452 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-04-24T23:35:29.452 DEBUG:teuthology.orchestra.run.smithi169:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-04-24T23:35:29.485 INFO:tasks.workunit:timeout=3h 2024-04-24T23:35:29.485 INFO:tasks.workunit:cleanup=True 2024-04-24T23:35:29.485 DEBUG:teuthology.orchestra.run.smithi157:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:35:29.517 INFO:tasks.workunit:timeout=3h 2024-04-24T23:35:29.518 INFO:tasks.workunit:cleanup=True 2024-04-24T23:35:29.518 DEBUG:teuthology.orchestra.run.smithi169:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:35:29.541 INFO:tasks.workunit.client.0.smithi157.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-24T23:35:29.573 INFO:tasks.workunit.client.1.smithi169.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-04-24T23:35:59.164 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:35:59.169 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:36:29.229 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:36:29.233 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:36:40.736 INFO:tasks.workunit.client.0.smithi157.stderr:Note: switching to 'd2c45d977142e024a59b68a8f1dce8f0b9840551'. 2024-04-24T23:36:40.736 INFO:tasks.workunit.client.0.smithi157.stderr: 2024-04-24T23:36:40.736 INFO:tasks.workunit.client.0.smithi157.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-24T23:36:40.736 INFO:tasks.workunit.client.0.smithi157.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-24T23:36:40.736 INFO:tasks.workunit.client.0.smithi157.stderr:state without impacting any branches by switching back to a branch. 2024-04-24T23:36:40.736 INFO:tasks.workunit.client.0.smithi157.stderr: 2024-04-24T23:36:40.736 INFO:tasks.workunit.client.0.smithi157.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-24T23:36:40.736 INFO:tasks.workunit.client.0.smithi157.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-24T23:36:40.736 INFO:tasks.workunit.client.0.smithi157.stderr: 2024-04-24T23:36:40.736 INFO:tasks.workunit.client.0.smithi157.stderr: git switch -c 2024-04-24T23:36:40.737 INFO:tasks.workunit.client.0.smithi157.stderr: 2024-04-24T23:36:40.737 INFO:tasks.workunit.client.0.smithi157.stderr:Or undo this operation with: 2024-04-24T23:36:40.737 INFO:tasks.workunit.client.0.smithi157.stderr: 2024-04-24T23:36:40.737 INFO:tasks.workunit.client.0.smithi157.stderr: git switch - 2024-04-24T23:36:40.737 INFO:tasks.workunit.client.0.smithi157.stderr: 2024-04-24T23:36:40.737 INFO:tasks.workunit.client.0.smithi157.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-24T23:36:40.737 INFO:tasks.workunit.client.0.smithi157.stderr: 2024-04-24T23:36:40.737 INFO:tasks.workunit.client.0.smithi157.stderr:HEAD is now at d2c45d97714 Merge pull request #56528 from adk3798/squid-mypy-bump 2024-04-24T23:36:40.739 DEBUG:teuthology.orchestra.run.smithi157:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-24T23:36:40.765 INFO:tasks.workunit.client.0.smithi157.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-24T23:36:40.767 INFO:tasks.workunit.client.0.smithi157.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-24T23:36:40.767 INFO:tasks.workunit.client.0.smithi157.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-24T23:36:41.101 INFO:tasks.workunit.client.1.smithi169.stderr:Note: switching to 'd2c45d977142e024a59b68a8f1dce8f0b9840551'. 2024-04-24T23:36:41.101 INFO:tasks.workunit.client.1.smithi169.stderr: 2024-04-24T23:36:41.101 INFO:tasks.workunit.client.1.smithi169.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-24T23:36:41.101 INFO:tasks.workunit.client.1.smithi169.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-24T23:36:41.101 INFO:tasks.workunit.client.1.smithi169.stderr:state without impacting any branches by switching back to a branch. 2024-04-24T23:36:41.101 INFO:tasks.workunit.client.1.smithi169.stderr: 2024-04-24T23:36:41.101 INFO:tasks.workunit.client.1.smithi169.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-24T23:36:41.101 INFO:tasks.workunit.client.1.smithi169.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-24T23:36:41.101 INFO:tasks.workunit.client.1.smithi169.stderr: 2024-04-24T23:36:41.101 INFO:tasks.workunit.client.1.smithi169.stderr: git switch -c 2024-04-24T23:36:41.102 INFO:tasks.workunit.client.1.smithi169.stderr: 2024-04-24T23:36:41.102 INFO:tasks.workunit.client.1.smithi169.stderr:Or undo this operation with: 2024-04-24T23:36:41.102 INFO:tasks.workunit.client.1.smithi169.stderr: 2024-04-24T23:36:41.102 INFO:tasks.workunit.client.1.smithi169.stderr: git switch - 2024-04-24T23:36:41.102 INFO:tasks.workunit.client.1.smithi169.stderr: 2024-04-24T23:36:41.102 INFO:tasks.workunit.client.1.smithi169.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-24T23:36:41.102 INFO:tasks.workunit.client.1.smithi169.stderr: 2024-04-24T23:36:41.102 INFO:tasks.workunit.client.1.smithi169.stderr:HEAD is now at d2c45d97714 Merge pull request #56528 from adk3798/squid-mypy-bump 2024-04-24T23:36:41.104 DEBUG:teuthology.orchestra.run.smithi169:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-04-24T23:36:41.197 INFO:tasks.workunit.client.1.smithi169.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-24T23:36:41.199 INFO:tasks.workunit.client.1.smithi169.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-04-24T23:36:41.199 INFO:tasks.workunit.client.1.smithi169.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-24T23:36:41.317 INFO:tasks.workunit.client.0.smithi157.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-24T23:36:41.372 INFO:tasks.workunit.client.0.smithi157.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-24T23:36:41.411 INFO:tasks.workunit.client.0.smithi157.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-24T23:36:41.413 INFO:tasks.workunit.client.0.smithi157.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-24T23:36:41.413 INFO:tasks.workunit.client.0.smithi157.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-24T23:36:41.452 INFO:tasks.workunit.client.0.smithi157.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-24T23:36:41.455 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-24T23:36:41.455 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-24T23:36:41.510 INFO:tasks.workunit:Running workunits matching suites/iozone.sh on client.0... 2024-04-24T23:36:41.512 INFO:tasks.workunit:Running workunit suites/iozone.sh... 2024-04-24T23:36:41.512 DEBUG:teuthology.orchestra.run.smithi157:workunit test suites/iozone.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d2c45d977142e024a59b68a8f1dce8f0b9840551 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iozone.sh 2024-04-24T23:36:41.577 INFO:tasks.workunit.client.0.smithi157.stderr:+ iozone -c -e -s 1024M -r 16K -t 1 -F f1 -i 0 -i 1 2024-04-24T23:36:41.647 INFO:tasks.workunit.client.0.smithi157.stdout: Iozone: Performance Test of File I/O 2024-04-24T23:36:41.647 INFO:tasks.workunit.client.0.smithi157.stdout: Version $Revision: 3.494 $ 2024-04-24T23:36:41.647 INFO:tasks.workunit.client.0.smithi157.stdout: Compiled for 64 bit mode. 2024-04-24T23:36:41.647 INFO:tasks.workunit.client.0.smithi157.stdout: Build: linux-AMD64 2024-04-24T23:36:41.647 INFO:tasks.workunit.client.0.smithi157.stdout: 2024-04-24T23:36:41.647 INFO:tasks.workunit.client.0.smithi157.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-04-24T23:36:41.647 INFO:tasks.workunit.client.0.smithi157.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-04-24T23:36:41.647 INFO:tasks.workunit.client.0.smithi157.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-04-24T23:36:41.648 INFO:tasks.workunit.client.0.smithi157.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-04-24T23:36:41.648 INFO:tasks.workunit.client.0.smithi157.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-04-24T23:36:41.648 INFO:tasks.workunit.client.0.smithi157.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-04-24T23:36:41.648 INFO:tasks.workunit.client.0.smithi157.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-04-24T23:36:41.648 INFO:tasks.workunit.client.0.smithi157.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-04-24T23:36:41.648 INFO:tasks.workunit.client.0.smithi157.stdout: Alexey Skidanov, Sudhir Kumar. 2024-04-24T23:36:41.648 INFO:tasks.workunit.client.0.smithi157.stdout: 2024-04-24T23:36:41.648 INFO:tasks.workunit.client.0.smithi157.stdout: Run began: Wed Apr 24 23:36:41 2024 2024-04-24T23:36:41.648 INFO:tasks.workunit.client.0.smithi157.stdout: 2024-04-24T23:36:41.648 INFO:tasks.workunit.client.0.smithi157.stdout: Include close in write timing 2024-04-24T23:36:41.648 INFO:tasks.workunit.client.0.smithi157.stdout: Include fsync in write timing 2024-04-24T23:36:41.648 INFO:tasks.workunit.client.0.smithi157.stdout: File size set to 1048576 kB 2024-04-24T23:36:41.648 INFO:tasks.workunit.client.0.smithi157.stdout: Record Size 16 kB 2024-04-24T23:36:41.649 INFO:tasks.workunit.client.0.smithi157.stdout: Command line used: iozone -c -e -s 1024M -r 16K -t 1 -F f1 -i 0 -i 1 2024-04-24T23:36:41.649 INFO:tasks.workunit.client.0.smithi157.stdout: Output is in kBytes/sec 2024-04-24T23:36:41.649 INFO:tasks.workunit.client.0.smithi157.stdout: Time Resolution = 0.000001 seconds. 2024-04-24T23:36:41.649 INFO:tasks.workunit.client.0.smithi157.stdout: Processor cache size set to 1024 kBytes. 2024-04-24T23:36:41.649 INFO:tasks.workunit.client.0.smithi157.stdout: Processor cache line size set to 32 bytes. 2024-04-24T23:36:41.649 INFO:tasks.workunit.client.0.smithi157.stdout: File stride size set to 17 * record size. 2024-04-24T23:36:41.649 INFO:tasks.workunit.client.0.smithi157.stdout: Throughput test with 1 process 2024-04-24T23:36:41.649 INFO:tasks.workunit.client.0.smithi157.stdout: Each process writes a 1048576 kByte file in 16 kByte records 2024-04-24T23:36:41.788 INFO:tasks.workunit.client.1.smithi169.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-24T23:36:41.845 INFO:tasks.workunit.client.1.smithi169.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-24T23:36:41.890 INFO:tasks.workunit.client.1.smithi169.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-04-24T23:36:41.893 INFO:tasks.workunit.client.1.smithi169.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-04-24T23:36:41.893 INFO:tasks.workunit.client.1.smithi169.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-24T23:36:41.932 INFO:tasks.workunit.client.1.smithi169.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-04-24T23:36:41.935 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-24T23:36:41.936 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-04-24T23:36:41.951 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.1... 2024-04-24T23:36:41.952 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-04-24T23:36:41.952 DEBUG:teuthology.orchestra.run.smithi169:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d2c45d977142e024a59b68a8f1dce8f0b9840551 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/dbench.sh 2024-04-24T23:36:42.013 INFO:tasks.workunit.client.1.smithi169.stderr:+ dbench 1 2024-04-24T23:36:42.027 INFO:tasks.workunit.client.1.smithi169.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-04-24T23:36:42.027 INFO:tasks.workunit.client.1.smithi169.stdout: 2024-04-24T23:36:42.027 INFO:tasks.workunit.client.1.smithi169.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-04-24T23:36:42.028 INFO:tasks.workunit.client.1.smithi169.stdout:0 of 1 processes prepared for launch 0 sec 2024-04-24T23:36:42.128 INFO:tasks.workunit.client.1.smithi169.stdout:1 of 1 processes prepared for launch 0 sec 2024-04-24T23:36:42.128 INFO:tasks.workunit.client.1.smithi169.stdout:releasing clients 2024-04-24T23:36:43.128 INFO:tasks.workunit.client.1.smithi169.stdout: 1 3797 39.76 MB/sec warmup 1 sec latency 10.728 ms 2024-04-24T23:36:44.128 INFO:tasks.workunit.client.1.smithi169.stdout: 1 7348 29.44 MB/sec warmup 2 sec latency 12.349 ms 2024-04-24T23:36:45.128 INFO:tasks.workunit.client.1.smithi169.stdout: 1 10891 26.00 MB/sec warmup 3 sec latency 14.906 ms 2024-04-24T23:36:46.128 INFO:tasks.workunit.client.1.smithi169.stdout: 1 14013 23.20 MB/sec warmup 4 sec latency 23.273 ms 2024-04-24T23:36:47.128 INFO:tasks.workunit.client.1.smithi169.stdout: 1 17401 22.15 MB/sec warmup 5 sec latency 15.444 ms 2024-04-24T23:36:48.128 INFO:tasks.workunit.client.1.smithi169.stdout: 1 20949 21.63 MB/sec warmup 6 sec latency 13.537 ms 2024-04-24T23:36:49.128 INFO:tasks.workunit.client.1.smithi169.stdout: 1 24245 21.18 MB/sec warmup 7 sec latency 21.513 ms 2024-04-24T23:36:50.128 INFO:tasks.workunit.client.1.smithi169.stdout: 1 27797 20.92 MB/sec warmup 8 sec latency 10.888 ms 2024-04-24T23:36:51.128 INFO:tasks.workunit.client.1.smithi169.stdout: 1 31176 20.55 MB/sec warmup 9 sec latency 30.170 ms 2024-04-24T23:36:51.744 INFO:tasks.workunit.client.0.smithi157.stdout: 2024-04-24T23:36:51.744 INFO:tasks.workunit.client.0.smithi157.stdout: Children see throughput for 1 initial writers = 103871.95 kB/sec 2024-04-24T23:36:51.744 INFO:tasks.workunit.client.0.smithi157.stdout: Parent sees throughput for 1 initial writers = 103870.29 kB/sec 2024-04-24T23:36:51.744 INFO:tasks.workunit.client.0.smithi157.stdout: Min throughput per process = 103871.95 kB/sec 2024-04-24T23:36:51.744 INFO:tasks.workunit.client.0.smithi157.stdout: Max throughput per process = 103871.95 kB/sec 2024-04-24T23:36:51.744 INFO:tasks.workunit.client.0.smithi157.stdout: Avg throughput per process = 103871.95 kB/sec 2024-04-24T23:36:51.744 INFO:tasks.workunit.client.0.smithi157.stdout: Min xfer = 1048576.00 kB 2024-04-24T23:36:52.128 INFO:tasks.workunit.client.1.smithi169.stdout: 1 34974 20.57 MB/sec warmup 10 sec latency 14.444 ms 2024-04-24T23:36:53.129 INFO:tasks.workunit.client.1.smithi169.stdout: 1 38882 20.61 MB/sec warmup 11 sec latency 20.635 ms 2024-04-24T23:36:54.128 INFO:tasks.workunit.client.1.smithi169.stdout: 1 42846 20.85 MB/sec warmup 12 sec latency 11.832 ms 2024-04-24T23:36:55.128 INFO:tasks.workunit.client.1.smithi169.stdout: 1 46393 20.71 MB/sec warmup 13 sec latency 37.277 ms 2024-04-24T23:36:56.129 INFO:tasks.workunit.client.1.smithi169.stdout: 1 49911 20.60 MB/sec warmup 14 sec latency 12.256 ms 2024-04-24T23:36:57.129 INFO:tasks.workunit.client.1.smithi169.stdout: 1 53463 20.50 MB/sec warmup 15 sec latency 15.324 ms 2024-04-24T23:36:58.128 INFO:tasks.workunit.client.1.smithi169.stdout: 1 54305 19.36 MB/sec warmup 16 sec latency 271.954 ms 2024-04-24T23:36:59.128 INFO:tasks.workunit.client.1.smithi169.stdout: 1 57012 19.21 MB/sec warmup 17 sec latency 38.874 ms 2024-04-24T23:36:59.311 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:36:59.315 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:37:00.128 INFO:tasks.workunit.client.1.smithi169.stdout: 1 60288 19.13 MB/sec warmup 18 sec latency 18.691 ms 2024-04-24T23:37:01.129 INFO:tasks.workunit.client.1.smithi169.stdout: 1 63319 18.87 MB/sec warmup 19 sec latency 20.419 ms 2024-04-24T23:37:02.129 INFO:tasks.workunit.client.1.smithi169.stdout: 1 66306 18.62 MB/sec warmup 20 sec latency 17.081 ms 2024-04-24T23:37:03.129 INFO:tasks.workunit.client.1.smithi169.stdout: 1 69029 18.55 MB/sec warmup 21 sec latency 20.048 ms 2024-04-24T23:37:04.129 INFO:tasks.workunit.client.1.smithi169.stdout: 1 72098 18.43 MB/sec warmup 22 sec latency 12.795 ms 2024-04-24T23:37:05.128 INFO:tasks.workunit.client.1.smithi169.stdout: 1 75328 18.39 MB/sec warmup 23 sec latency 19.392 ms 2024-04-24T23:37:06.129 INFO:tasks.workunit.client.1.smithi169.stdout: 1 78689 18.41 MB/sec warmup 24 sec latency 22.575 ms 2024-04-24T23:37:07.129 INFO:tasks.workunit.client.1.smithi169.stdout: 1 82006 18.43 MB/sec warmup 25 sec latency 19.163 ms 2024-04-24T23:37:07.515 INFO:tasks.workunit.client.0.smithi157.stdout: 2024-04-24T23:37:07.515 INFO:tasks.workunit.client.0.smithi157.stdout: Children see throughput for 1 rewriters = 108096.45 kB/sec 2024-04-24T23:37:07.515 INFO:tasks.workunit.client.0.smithi157.stdout: Parent sees throughput for 1 rewriters = 108093.89 kB/sec 2024-04-24T23:37:07.515 INFO:tasks.workunit.client.0.smithi157.stdout: Min throughput per process = 108096.45 kB/sec 2024-04-24T23:37:07.515 INFO:tasks.workunit.client.0.smithi157.stdout: Max throughput per process = 108096.45 kB/sec 2024-04-24T23:37:07.515 INFO:tasks.workunit.client.0.smithi157.stdout: Avg throughput per process = 108096.45 kB/sec 2024-04-24T23:37:07.515 INFO:tasks.workunit.client.0.smithi157.stdout: Min xfer = 1048576.00 kB 2024-04-24T23:37:08.129 INFO:tasks.workunit.client.1.smithi169.stdout: 1 85429 18.45 MB/sec warmup 26 sec latency 19.600 ms 2024-04-24T23:37:09.129 INFO:tasks.workunit.client.1.smithi169.stdout: 1 88967 18.47 MB/sec warmup 27 sec latency 18.758 ms 2024-04-24T23:37:10.129 INFO:tasks.workunit.client.1.smithi169.stdout: 1 92387 18.46 MB/sec warmup 28 sec latency 27.349 ms 2024-04-24T23:37:11.129 INFO:tasks.workunit.client.1.smithi169.stdout: 1 95902 18.48 MB/sec warmup 29 sec latency 14.653 ms 2024-04-24T23:37:12.129 INFO:tasks.workunit.client.1.smithi169.stdout: 1 99342 18.49 MB/sec warmup 30 sec latency 15.697 ms 2024-04-24T23:37:13.129 INFO:tasks.workunit.client.1.smithi169.stdout: 1 102698 18.42 MB/sec warmup 31 sec latency 13.054 ms 2024-04-24T23:37:14.069 INFO:tasks.workunit.client.0.smithi157.stdout: 2024-04-24T23:37:14.069 INFO:tasks.workunit.client.0.smithi157.stdout: Children see throughput for 1 readers = 7396886.00 kB/sec 2024-04-24T23:37:14.069 INFO:tasks.workunit.client.0.smithi157.stdout: Parent sees throughput for 1 readers = 7388250.07 kB/sec 2024-04-24T23:37:14.069 INFO:tasks.workunit.client.0.smithi157.stdout: Min throughput per process = 7396886.00 kB/sec 2024-04-24T23:37:14.069 INFO:tasks.workunit.client.0.smithi157.stdout: Max throughput per process = 7396886.00 kB/sec 2024-04-24T23:37:14.069 INFO:tasks.workunit.client.0.smithi157.stdout: Avg throughput per process = 7396886.00 kB/sec 2024-04-24T23:37:14.069 INFO:tasks.workunit.client.0.smithi157.stdout: Min xfer = 1048576.00 kB 2024-04-24T23:37:14.129 INFO:tasks.workunit.client.1.smithi169.stdout: 1 105687 18.33 MB/sec warmup 32 sec latency 15.064 ms 2024-04-24T23:37:15.129 INFO:tasks.workunit.client.1.smithi169.stdout: 1 108937 18.26 MB/sec warmup 33 sec latency 17.430 ms 2024-04-24T23:37:16.129 INFO:tasks.workunit.client.1.smithi169.stdout: 1 112429 18.26 MB/sec warmup 34 sec latency 23.016 ms 2024-04-24T23:37:16.449 INFO:tasks.workunit.client.0.smithi157.stdout: 2024-04-24T23:37:16.449 INFO:tasks.workunit.client.0.smithi157.stdout: Children see throughput for 1 re-readers = 7418884.00 kB/sec 2024-04-24T23:37:16.449 INFO:tasks.workunit.client.0.smithi157.stdout: Parent sees throughput for 1 re-readers = 7408186.90 kB/sec 2024-04-24T23:37:16.449 INFO:tasks.workunit.client.0.smithi157.stdout: Min throughput per process = 7418884.00 kB/sec 2024-04-24T23:37:16.450 INFO:tasks.workunit.client.0.smithi157.stdout: Max throughput per process = 7418884.00 kB/sec 2024-04-24T23:37:16.450 INFO:tasks.workunit.client.0.smithi157.stdout: Avg throughput per process = 7418884.00 kB/sec 2024-04-24T23:37:16.450 INFO:tasks.workunit.client.0.smithi157.stdout: Min xfer = 1048576.00 kB 2024-04-24T23:37:17.129 INFO:tasks.workunit.client.1.smithi169.stdout: 1 115522 18.24 MB/sec warmup 35 sec latency 14.418 ms 2024-04-24T23:37:18.129 INFO:tasks.workunit.client.1.smithi169.stdout: 1 118730 18.26 MB/sec warmup 36 sec latency 22.702 ms 2024-04-24T23:37:19.129 INFO:tasks.workunit.client.1.smithi169.stdout: 1 122043 18.28 MB/sec warmup 37 sec latency 13.707 ms 2024-04-24T23:37:20.129 INFO:tasks.workunit.client.1.smithi169.stdout: 1 125172 18.19 MB/sec warmup 38 sec latency 20.468 ms 2024-04-24T23:37:21.129 INFO:tasks.workunit.client.1.smithi169.stdout: 1 128595 18.21 MB/sec warmup 39 sec latency 20.760 ms 2024-04-24T23:37:22.129 INFO:tasks.workunit.client.1.smithi169.stdout: 1 131993 18.22 MB/sec warmup 40 sec latency 17.659 ms 2024-04-24T23:37:23.129 INFO:tasks.workunit.client.1.smithi169.stdout: 1 135098 18.23 MB/sec warmup 41 sec latency 18.949 ms 2024-04-24T23:37:24.129 INFO:tasks.workunit.client.1.smithi169.stdout: 1 138524 18.23 MB/sec warmup 42 sec latency 42.238 ms 2024-04-24T23:37:24.139 INFO:tasks.workunit.client.0.smithi157.stdout: 2024-04-24T23:37:24.139 INFO:tasks.workunit.client.0.smithi157.stdout: 2024-04-24T23:37:24.139 INFO:tasks.workunit.client.0.smithi157.stdout: 2024-04-24T23:37:24.140 INFO:tasks.workunit.client.0.smithi157.stdout:iozone test complete. 2024-04-24T23:37:24.140 INFO:tasks.workunit.client.0.smithi157.stderr:+ iozone -c -e -s 1024M -r 1M -t 1 -F f2 -i 0 -i 1 2024-04-24T23:37:24.206 INFO:tasks.workunit.client.0.smithi157.stdout: Iozone: Performance Test of File I/O 2024-04-24T23:37:24.206 INFO:tasks.workunit.client.0.smithi157.stdout: Version $Revision: 3.494 $ 2024-04-24T23:37:24.206 INFO:tasks.workunit.client.0.smithi157.stdout: Compiled for 64 bit mode. 2024-04-24T23:37:24.206 INFO:tasks.workunit.client.0.smithi157.stdout: Build: linux-AMD64 2024-04-24T23:37:24.206 INFO:tasks.workunit.client.0.smithi157.stdout: 2024-04-24T23:37:24.206 INFO:tasks.workunit.client.0.smithi157.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-04-24T23:37:24.206 INFO:tasks.workunit.client.0.smithi157.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-04-24T23:37:24.206 INFO:tasks.workunit.client.0.smithi157.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-04-24T23:37:24.206 INFO:tasks.workunit.client.0.smithi157.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-04-24T23:37:24.207 INFO:tasks.workunit.client.0.smithi157.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-04-24T23:37:24.207 INFO:tasks.workunit.client.0.smithi157.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-04-24T23:37:24.207 INFO:tasks.workunit.client.0.smithi157.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-04-24T23:37:24.207 INFO:tasks.workunit.client.0.smithi157.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-04-24T23:37:24.207 INFO:tasks.workunit.client.0.smithi157.stdout: Alexey Skidanov, Sudhir Kumar. 2024-04-24T23:37:24.207 INFO:tasks.workunit.client.0.smithi157.stdout: 2024-04-24T23:37:24.207 INFO:tasks.workunit.client.0.smithi157.stdout: Run began: Wed Apr 24 23:37:24 2024 2024-04-24T23:37:24.207 INFO:tasks.workunit.client.0.smithi157.stdout: 2024-04-24T23:37:24.207 INFO:tasks.workunit.client.0.smithi157.stdout: Include close in write timing 2024-04-24T23:37:24.207 INFO:tasks.workunit.client.0.smithi157.stdout: Include fsync in write timing 2024-04-24T23:37:24.207 INFO:tasks.workunit.client.0.smithi157.stdout: File size set to 1048576 kB 2024-04-24T23:37:24.207 INFO:tasks.workunit.client.0.smithi157.stdout: Record Size 1024 kB 2024-04-24T23:37:24.207 INFO:tasks.workunit.client.0.smithi157.stdout: Command line used: iozone -c -e -s 1024M -r 1M -t 1 -F f2 -i 0 -i 1 2024-04-24T23:37:24.207 INFO:tasks.workunit.client.0.smithi157.stdout: Output is in kBytes/sec 2024-04-24T23:37:24.208 INFO:tasks.workunit.client.0.smithi157.stdout: Time Resolution = 0.000001 seconds. 2024-04-24T23:37:24.208 INFO:tasks.workunit.client.0.smithi157.stdout: Processor cache size set to 1024 kBytes. 2024-04-24T23:37:24.208 INFO:tasks.workunit.client.0.smithi157.stdout: Processor cache line size set to 32 bytes. 2024-04-24T23:37:24.208 INFO:tasks.workunit.client.0.smithi157.stdout: File stride size set to 17 * record size. 2024-04-24T23:37:24.208 INFO:tasks.workunit.client.0.smithi157.stdout: Throughput test with 1 process 2024-04-24T23:37:24.208 INFO:tasks.workunit.client.0.smithi157.stdout: Each process writes a 1048576 kByte file in 1024 kByte records 2024-04-24T23:37:25.129 INFO:tasks.workunit.client.1.smithi169.stdout: 1 140458 17.97 MB/sec warmup 43 sec latency 23.190 ms 2024-04-24T23:37:26.129 INFO:tasks.workunit.client.1.smithi169.stdout: 1 142477 17.86 MB/sec warmup 44 sec latency 22.055 ms 2024-04-24T23:37:26.166 INFO:tasks.workunit.client.0.smithi157.stdout: 2024-04-24T23:37:26.166 INFO:tasks.workunit.client.0.smithi157.stdout: Children see throughput for 1 initial writers = 543752.44 kB/sec 2024-04-24T23:37:26.166 INFO:tasks.workunit.client.0.smithi157.stdout: Parent sees throughput for 1 initial writers = 543692.10 kB/sec 2024-04-24T23:37:26.166 INFO:tasks.workunit.client.0.smithi157.stdout: Min throughput per process = 543752.44 kB/sec 2024-04-24T23:37:26.166 INFO:tasks.workunit.client.0.smithi157.stdout: Max throughput per process = 543752.44 kB/sec 2024-04-24T23:37:26.166 INFO:tasks.workunit.client.0.smithi157.stdout: Avg throughput per process = 543752.44 kB/sec 2024-04-24T23:37:26.166 INFO:tasks.workunit.client.0.smithi157.stdout: Min xfer = 1048576.00 kB 2024-04-24T23:37:27.129 INFO:tasks.workunit.client.1.smithi169.stdout: 1 145888 17.89 MB/sec warmup 45 sec latency 19.600 ms 2024-04-24T23:37:28.129 INFO:tasks.workunit.client.1.smithi169.stdout: 1 149250 17.91 MB/sec warmup 46 sec latency 15.329 ms 2024-04-24T23:37:29.129 INFO:tasks.workunit.client.1.smithi169.stdout: 1 149479 17.54 MB/sec warmup 47 sec latency 443.915 ms 2024-04-24T23:37:29.508 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:37:29.512 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:37:30.129 INFO:tasks.workunit.client.1.smithi169.stdout: 1 150059 17.21 MB/sec warmup 48 sec latency 472.683 ms 2024-04-24T23:37:31.129 INFO:tasks.workunit.client.1.smithi169.stdout: 1 151200 16.94 MB/sec warmup 49 sec latency 53.816 ms 2024-04-24T23:37:31.277 INFO:tasks.workunit.client.0.smithi157.stdout: 2024-04-24T23:37:31.277 INFO:tasks.workunit.client.0.smithi157.stdout: Children see throughput for 1 rewriters = 504598.53 kB/sec 2024-04-24T23:37:31.277 INFO:tasks.workunit.client.0.smithi157.stdout: Parent sees throughput for 1 rewriters = 504533.71 kB/sec 2024-04-24T23:37:31.277 INFO:tasks.workunit.client.0.smithi157.stdout: Min throughput per process = 504598.53 kB/sec 2024-04-24T23:37:31.277 INFO:tasks.workunit.client.0.smithi157.stdout: Max throughput per process = 504598.53 kB/sec 2024-04-24T23:37:31.277 INFO:tasks.workunit.client.0.smithi157.stdout: Avg throughput per process = 504598.53 kB/sec 2024-04-24T23:37:31.277 INFO:tasks.workunit.client.0.smithi157.stdout: Min xfer = 1048576.00 kB 2024-04-24T23:37:32.129 INFO:tasks.workunit.client.1.smithi169.stdout: 1 153814 16.91 MB/sec warmup 50 sec latency 39.492 ms 2024-04-24T23:37:33.129 INFO:tasks.workunit.client.1.smithi169.stdout: 1 157036 16.93 MB/sec warmup 51 sec latency 22.908 ms 2024-04-24T23:37:34.129 INFO:tasks.workunit.client.1.smithi169.stdout: 1 160325 16.96 MB/sec warmup 52 sec latency 59.626 ms 2024-04-24T23:37:35.130 INFO:tasks.workunit.client.1.smithi169.stdout: 1 163650 16.99 MB/sec warmup 53 sec latency 22.700 ms 2024-04-24T23:37:36.127 INFO:tasks.workunit.client.0.smithi157.stdout: 2024-04-24T23:37:36.128 INFO:tasks.workunit.client.0.smithi157.stdout: Children see throughput for 1 readers = 7472379.00 kB/sec 2024-04-24T23:37:36.128 INFO:tasks.workunit.client.0.smithi157.stdout: Parent sees throughput for 1 readers = 7462476.71 kB/sec 2024-04-24T23:37:36.128 INFO:tasks.workunit.client.0.smithi157.stdout: Min throughput per process = 7472379.00 kB/sec 2024-04-24T23:37:36.128 INFO:tasks.workunit.client.0.smithi157.stdout: Max throughput per process = 7472379.00 kB/sec 2024-04-24T23:37:36.128 INFO:tasks.workunit.client.0.smithi157.stdout: Avg throughput per process = 7472379.00 kB/sec 2024-04-24T23:37:36.128 INFO:tasks.workunit.client.0.smithi157.stdout: Min xfer = 1048576.00 kB 2024-04-24T23:37:36.129 INFO:tasks.workunit.client.1.smithi169.stdout: 1 167099 17.03 MB/sec warmup 54 sec latency 12.782 ms 2024-04-24T23:37:37.130 INFO:tasks.workunit.client.1.smithi169.stdout: 1 170545 17.06 MB/sec warmup 55 sec latency 13.415 ms 2024-04-24T23:37:38.130 INFO:tasks.workunit.client.1.smithi169.stdout: 1 174048 17.10 MB/sec warmup 56 sec latency 19.351 ms 2024-04-24T23:37:38.796 INFO:tasks.workunit.client.0.smithi157.stdout: 2024-04-24T23:37:38.796 INFO:tasks.workunit.client.0.smithi157.stdout: Children see throughput for 1 re-readers = 7508648.00 kB/sec 2024-04-24T23:37:38.797 INFO:tasks.workunit.client.0.smithi157.stdout: Parent sees throughput for 1 re-readers = 7483835.34 kB/sec 2024-04-24T23:37:38.797 INFO:tasks.workunit.client.0.smithi157.stdout: Min throughput per process = 7508648.00 kB/sec 2024-04-24T23:37:38.797 INFO:tasks.workunit.client.0.smithi157.stdout: Max throughput per process = 7508648.00 kB/sec 2024-04-24T23:37:38.797 INFO:tasks.workunit.client.0.smithi157.stdout: Avg throughput per process = 7508648.00 kB/sec 2024-04-24T23:37:38.797 INFO:tasks.workunit.client.0.smithi157.stdout: Min xfer = 1048576.00 kB 2024-04-24T23:37:39.129 INFO:tasks.workunit.client.1.smithi169.stdout: 1 177427 17.12 MB/sec warmup 57 sec latency 15.786 ms 2024-04-24T23:37:40.130 INFO:tasks.workunit.client.1.smithi169.stdout: 1 180703 17.10 MB/sec warmup 58 sec latency 70.937 ms 2024-04-24T23:37:41.130 INFO:tasks.workunit.client.1.smithi169.stdout: 1 183884 17.10 MB/sec warmup 59 sec latency 18.463 ms 2024-04-24T23:37:42.130 INFO:tasks.workunit.client.1.smithi169.stdout: 1 187037 17.06 MB/sec warmup 60 sec latency 15.025 ms 2024-04-24T23:37:43.130 INFO:tasks.workunit.client.1.smithi169.stdout: 1 190577 17.09 MB/sec warmup 61 sec latency 12.387 ms 2024-04-24T23:37:44.130 INFO:tasks.workunit.client.1.smithi169.stdout: 1 193950 17.11 MB/sec warmup 62 sec latency 48.917 ms 2024-04-24T23:37:45.130 INFO:tasks.workunit.client.1.smithi169.stdout: 1 195528 17.03 MB/sec warmup 63 sec latency 111.501 ms 2024-04-24T23:37:46.130 INFO:tasks.workunit.client.1.smithi169.stdout: 1 197972 16.94 MB/sec warmup 64 sec latency 45.223 ms 2024-04-24T23:37:46.639 INFO:tasks.workunit.client.0.smithi157.stdout: 2024-04-24T23:37:46.639 INFO:tasks.workunit.client.0.smithi157.stdout: 2024-04-24T23:37:46.639 INFO:tasks.workunit.client.0.smithi157.stdout: 2024-04-24T23:37:46.639 INFO:tasks.workunit.client.0.smithi157.stdout:iozone test complete. 2024-04-24T23:37:46.640 INFO:tasks.workunit.client.0.smithi157.stderr:+ iozone -c -e -s 10240M -r 1M -t 1 -F f3 -i 0 -i 1 2024-04-24T23:37:46.710 INFO:tasks.workunit.client.0.smithi157.stdout: Iozone: Performance Test of File I/O 2024-04-24T23:37:46.710 INFO:tasks.workunit.client.0.smithi157.stdout: Version $Revision: 3.494 $ 2024-04-24T23:37:46.710 INFO:tasks.workunit.client.0.smithi157.stdout: Compiled for 64 bit mode. 2024-04-24T23:37:46.710 INFO:tasks.workunit.client.0.smithi157.stdout: Build: linux-AMD64 2024-04-24T23:37:46.710 INFO:tasks.workunit.client.0.smithi157.stdout: 2024-04-24T23:37:46.710 INFO:tasks.workunit.client.0.smithi157.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-04-24T23:37:46.710 INFO:tasks.workunit.client.0.smithi157.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-04-24T23:37:46.711 INFO:tasks.workunit.client.0.smithi157.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-04-24T23:37:46.711 INFO:tasks.workunit.client.0.smithi157.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-04-24T23:37:46.711 INFO:tasks.workunit.client.0.smithi157.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-04-24T23:37:46.711 INFO:tasks.workunit.client.0.smithi157.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-04-24T23:37:46.711 INFO:tasks.workunit.client.0.smithi157.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-04-24T23:37:46.711 INFO:tasks.workunit.client.0.smithi157.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-04-24T23:37:46.711 INFO:tasks.workunit.client.0.smithi157.stdout: Alexey Skidanov, Sudhir Kumar. 2024-04-24T23:37:46.711 INFO:tasks.workunit.client.0.smithi157.stdout: 2024-04-24T23:37:46.711 INFO:tasks.workunit.client.0.smithi157.stdout: Run began: Wed Apr 24 23:37:46 2024 2024-04-24T23:37:46.711 INFO:tasks.workunit.client.0.smithi157.stdout: 2024-04-24T23:37:46.711 INFO:tasks.workunit.client.0.smithi157.stdout: Include close in write timing 2024-04-24T23:37:46.711 INFO:tasks.workunit.client.0.smithi157.stdout: Include fsync in write timing 2024-04-24T23:37:46.711 INFO:tasks.workunit.client.0.smithi157.stdout: File size set to 10485760 kB 2024-04-24T23:37:46.711 INFO:tasks.workunit.client.0.smithi157.stdout: Record Size 1024 kB 2024-04-24T23:37:46.712 INFO:tasks.workunit.client.0.smithi157.stdout: Command line used: iozone -c -e -s 10240M -r 1M -t 1 -F f3 -i 0 -i 1 2024-04-24T23:37:46.712 INFO:tasks.workunit.client.0.smithi157.stdout: Output is in kBytes/sec 2024-04-24T23:37:46.712 INFO:tasks.workunit.client.0.smithi157.stdout: Time Resolution = 0.000001 seconds. 2024-04-24T23:37:46.712 INFO:tasks.workunit.client.0.smithi157.stdout: Processor cache size set to 1024 kBytes. 2024-04-24T23:37:46.712 INFO:tasks.workunit.client.0.smithi157.stdout: Processor cache line size set to 32 bytes. 2024-04-24T23:37:46.712 INFO:tasks.workunit.client.0.smithi157.stdout: File stride size set to 17 * record size. 2024-04-24T23:37:46.712 INFO:tasks.workunit.client.0.smithi157.stdout: Throughput test with 1 process 2024-04-24T23:37:46.712 INFO:tasks.workunit.client.0.smithi157.stdout: Each process writes a 10485760 kByte file in 1024 kByte records 2024-04-24T23:37:47.130 INFO:tasks.workunit.client.1.smithi169.stdout: 1 199743 16.83 MB/sec warmup 65 sec latency 19.818 ms 2024-04-24T23:37:48.130 INFO:tasks.workunit.client.1.smithi169.stdout: 1 202408 16.83 MB/sec warmup 66 sec latency 31.211 ms 2024-04-24T23:37:49.130 INFO:tasks.workunit.client.1.smithi169.stdout: 1 203795 16.66 MB/sec warmup 67 sec latency 23.978 ms 2024-04-24T23:37:50.130 INFO:tasks.workunit.client.1.smithi169.stdout: 1 206147 16.62 MB/sec warmup 68 sec latency 16.731 ms 2024-04-24T23:37:51.130 INFO:tasks.workunit.client.1.smithi169.stdout: 1 208146 16.48 MB/sec warmup 69 sec latency 19.034 ms 2024-04-24T23:37:52.130 INFO:tasks.workunit.client.1.smithi169.stdout: 1 209937 16.42 MB/sec warmup 70 sec latency 18.762 ms 2024-04-24T23:37:53.130 INFO:tasks.workunit.client.1.smithi169.stdout: 1 212264 16.36 MB/sec warmup 71 sec latency 16.795 ms 2024-04-24T23:37:54.130 INFO:tasks.workunit.client.1.smithi169.stdout: 1 213755 16.24 MB/sec warmup 72 sec latency 17.324 ms 2024-04-24T23:37:55.130 INFO:tasks.workunit.client.1.smithi169.stdout: 1 216515 16.25 MB/sec warmup 73 sec latency 27.645 ms 2024-04-24T23:37:56.130 INFO:tasks.workunit.client.1.smithi169.stdout: 1 217928 16.12 MB/sec warmup 74 sec latency 40.017 ms 2024-04-24T23:37:57.133 INFO:tasks.workunit.client.1.smithi169.stdout: 1 220340 16.09 MB/sec warmup 75 sec latency 18.847 ms 2024-04-24T23:37:58.130 INFO:tasks.workunit.client.1.smithi169.stdout: 1 222361 15.97 MB/sec warmup 76 sec latency 26.559 ms 2024-04-24T23:37:59.130 INFO:tasks.workunit.client.1.smithi169.stdout: 1 224202 15.92 MB/sec warmup 77 sec latency 16.361 ms 2024-04-24T23:37:59.666 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:37:59.669 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:38:00.130 INFO:tasks.workunit.client.1.smithi169.stdout: 1 226507 15.88 MB/sec warmup 78 sec latency 89.666 ms 2024-04-24T23:38:01.131 INFO:tasks.workunit.client.1.smithi169.stdout: 1 228210 15.78 MB/sec warmup 79 sec latency 38.145 ms 2024-04-24T23:38:02.130 INFO:tasks.workunit.client.1.smithi169.stdout: 1 230912 15.80 MB/sec warmup 80 sec latency 27.229 ms 2024-04-24T23:38:03.131 INFO:tasks.workunit.client.1.smithi169.stdout: 1 233012 15.69 MB/sec warmup 81 sec latency 24.716 ms 2024-04-24T23:38:04.130 INFO:tasks.workunit.client.1.smithi169.stdout: 1 234889 15.65 MB/sec warmup 82 sec latency 18.523 ms 2024-04-24T23:38:05.131 INFO:tasks.workunit.client.1.smithi169.stdout: 1 237771 15.67 MB/sec warmup 83 sec latency 35.181 ms 2024-04-24T23:38:06.130 INFO:tasks.workunit.client.1.smithi169.stdout: 1 238455 15.51 MB/sec warmup 84 sec latency 242.907 ms 2024-04-24T23:38:06.286 INFO:tasks.workunit.client.0.smithi157.stdout: 2024-04-24T23:38:06.286 INFO:tasks.workunit.client.0.smithi157.stdout: Children see throughput for 1 initial writers = 536533.88 kB/sec 2024-04-24T23:38:06.286 INFO:tasks.workunit.client.0.smithi157.stdout: Parent sees throughput for 1 initial writers = 536526.88 kB/sec 2024-04-24T23:38:06.286 INFO:tasks.workunit.client.0.smithi157.stdout: Min throughput per process = 536533.88 kB/sec 2024-04-24T23:38:06.286 INFO:tasks.workunit.client.0.smithi157.stdout: Max throughput per process = 536533.88 kB/sec 2024-04-24T23:38:06.286 INFO:tasks.workunit.client.0.smithi157.stdout: Avg throughput per process = 536533.88 kB/sec 2024-04-24T23:38:06.286 INFO:tasks.workunit.client.0.smithi157.stdout: Min xfer = 10485760.00 kB 2024-04-24T23:38:07.130 INFO:tasks.workunit.client.1.smithi169.stdout: 1 241365 15.53 MB/sec warmup 85 sec latency 81.375 ms 2024-04-24T23:38:08.130 INFO:tasks.workunit.client.1.smithi169.stdout: 1 245006 15.58 MB/sec warmup 86 sec latency 28.670 ms 2024-04-24T23:38:09.130 INFO:tasks.workunit.client.1.smithi169.stdout: 1 248438 15.61 MB/sec warmup 87 sec latency 22.150 ms 2024-04-24T23:38:10.131 INFO:tasks.workunit.client.1.smithi169.stdout: 1 251516 15.60 MB/sec warmup 88 sec latency 86.596 ms 2024-04-24T23:38:11.131 INFO:tasks.workunit.client.1.smithi169.stdout: 1 255073 15.64 MB/sec warmup 89 sec latency 14.607 ms 2024-04-24T23:38:12.131 INFO:tasks.workunit.client.1.smithi169.stdout: 1 258341 15.67 MB/sec warmup 90 sec latency 17.757 ms 2024-04-24T23:38:13.131 INFO:tasks.workunit.client.1.smithi169.stdout: 1 261737 15.69 MB/sec warmup 91 sec latency 31.805 ms 2024-04-24T23:38:14.131 INFO:tasks.workunit.client.1.smithi169.stdout: 1 264942 15.69 MB/sec warmup 92 sec latency 30.105 ms 2024-04-24T23:38:15.131 INFO:tasks.workunit.client.1.smithi169.stdout: 1 267590 15.70 MB/sec warmup 93 sec latency 13.905 ms 2024-04-24T23:38:16.131 INFO:tasks.workunit.client.1.smithi169.stdout: 1 270068 15.68 MB/sec warmup 94 sec latency 24.439 ms 2024-04-24T23:38:17.131 INFO:tasks.workunit.client.1.smithi169.stdout: 1 272038 15.59 MB/sec warmup 95 sec latency 17.667 ms 2024-04-24T23:38:18.131 INFO:tasks.workunit.client.1.smithi169.stdout: 1 273617 15.56 MB/sec warmup 96 sec latency 26.822 ms 2024-04-24T23:38:19.132 INFO:tasks.workunit.client.1.smithi169.stdout: 1 275559 15.47 MB/sec warmup 97 sec latency 23.136 ms 2024-04-24T23:38:20.131 INFO:tasks.workunit.client.1.smithi169.stdout: 1 277103 15.43 MB/sec warmup 98 sec latency 22.891 ms 2024-04-24T23:38:21.132 INFO:tasks.workunit.client.1.smithi169.stdout: 1 278519 15.33 MB/sec warmup 99 sec latency 26.500 ms 2024-04-24T23:38:22.132 INFO:tasks.workunit.client.1.smithi169.stdout: 1 280550 15.31 MB/sec warmup 100 sec latency 17.189 ms 2024-04-24T23:38:23.132 INFO:tasks.workunit.client.1.smithi169.stdout: 1 281801 15.22 MB/sec warmup 101 sec latency 43.541 ms 2024-04-24T23:38:24.132 INFO:tasks.workunit.client.1.smithi169.stdout: 1 284048 15.20 MB/sec warmup 102 sec latency 19.034 ms 2024-04-24T23:38:25.132 INFO:tasks.workunit.client.1.smithi169.stdout: 1 285343 15.11 MB/sec warmup 103 sec latency 43.369 ms 2024-04-24T23:38:26.132 INFO:tasks.workunit.client.1.smithi169.stdout: 1 287585 15.09 MB/sec warmup 104 sec latency 21.938 ms 2024-04-24T23:38:27.132 INFO:tasks.workunit.client.1.smithi169.stdout: 1 287657 14.95 MB/sec warmup 105 sec latency 162.467 ms 2024-04-24T23:38:28.132 INFO:tasks.workunit.client.1.smithi169.stdout: 1 288159 14.82 MB/sec warmup 106 sec latency 83.332 ms 2024-04-24T23:38:29.132 INFO:tasks.workunit.client.1.smithi169.stdout: 1 290319 14.79 MB/sec warmup 107 sec latency 46.399 ms 2024-04-24T23:38:30.009 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:38:30.014 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:38:30.132 INFO:tasks.workunit.client.1.smithi169.stdout: 1 291559 14.71 MB/sec warmup 108 sec latency 29.236 ms 2024-04-24T23:38:31.132 INFO:tasks.workunit.client.1.smithi169.stdout: 1 293478 14.65 MB/sec warmup 109 sec latency 28.012 ms 2024-04-24T23:38:32.132 INFO:tasks.workunit.client.1.smithi169.stdout: 1 294995 14.62 MB/sec warmup 110 sec latency 22.105 ms 2024-04-24T23:38:33.132 INFO:tasks.workunit.client.1.smithi169.stdout: 1 296840 14.55 MB/sec warmup 111 sec latency 27.123 ms 2024-04-24T23:38:34.113 INFO:tasks.workunit.client.0.smithi157.stdout: 2024-04-24T23:38:34.113 INFO:tasks.workunit.client.0.smithi157.stdout: Children see throughput for 1 rewriters = 562571.88 kB/sec 2024-04-24T23:38:34.113 INFO:tasks.workunit.client.0.smithi157.stdout: Parent sees throughput for 1 rewriters = 562564.98 kB/sec 2024-04-24T23:38:34.113 INFO:tasks.workunit.client.0.smithi157.stdout: Min throughput per process = 562571.88 kB/sec 2024-04-24T23:38:34.113 INFO:tasks.workunit.client.0.smithi157.stdout: Max throughput per process = 562571.88 kB/sec 2024-04-24T23:38:34.113 INFO:tasks.workunit.client.0.smithi157.stdout: Avg throughput per process = 562571.88 kB/sec 2024-04-24T23:38:34.113 INFO:tasks.workunit.client.0.smithi157.stdout: Min xfer = 10485760.00 kB 2024-04-24T23:38:34.132 INFO:tasks.workunit.client.1.smithi169.stdout: 1 298365 14.53 MB/sec warmup 112 sec latency 21.335 ms 2024-04-24T23:38:35.132 INFO:tasks.workunit.client.1.smithi169.stdout: 1 301843 14.57 MB/sec warmup 113 sec latency 18.131 ms 2024-04-24T23:38:36.132 INFO:tasks.workunit.client.1.smithi169.stdout: 1 305216 14.60 MB/sec warmup 114 sec latency 19.304 ms 2024-04-24T23:38:37.132 INFO:tasks.workunit.client.1.smithi169.stdout: 1 308593 14.63 MB/sec warmup 115 sec latency 30.912 ms 2024-04-24T23:38:38.132 INFO:tasks.workunit.client.1.smithi169.stdout: 1 311808 14.64 MB/sec warmup 116 sec latency 16.317 ms 2024-04-24T23:38:39.132 INFO:tasks.workunit.client.1.smithi169.stdout: 1 315075 14.67 MB/sec warmup 117 sec latency 20.326 ms 2024-04-24T23:38:40.132 INFO:tasks.workunit.client.1.smithi169.stdout: 1 318263 14.67 MB/sec warmup 118 sec latency 53.537 ms 2024-04-24T23:38:41.132 INFO:tasks.workunit.client.1.smithi169.stdout: 1 321446 14.69 MB/sec warmup 119 sec latency 61.905 ms 2024-04-24T23:38:43.132 INFO:tasks.workunit.client.1.smithi169.stdout: 1 326553 13.59 MB/sec execute 1 sec latency 123.159 ms 2024-04-24T23:38:44.132 INFO:tasks.workunit.client.1.smithi169.stdout: 1 329547 13.62 MB/sec execute 2 sec latency 27.075 ms 2024-04-24T23:38:45.132 INFO:tasks.workunit.client.1.smithi169.stdout: 1 332865 15.19 MB/sec execute 3 sec latency 31.654 ms 2024-04-24T23:38:46.133 INFO:tasks.workunit.client.1.smithi169.stdout: 1 335812 14.79 MB/sec execute 4 sec latency 71.215 ms 2024-04-24T23:38:47.133 INFO:tasks.workunit.client.1.smithi169.stdout: 1 339306 15.63 MB/sec execute 5 sec latency 17.471 ms 2024-04-24T23:38:48.133 INFO:tasks.workunit.client.1.smithi169.stdout: 1 342213 15.94 MB/sec execute 6 sec latency 16.942 ms 2024-04-24T23:38:49.133 INFO:tasks.workunit.client.1.smithi169.stdout: 1 345520 16.37 MB/sec execute 7 sec latency 34.523 ms 2024-04-24T23:38:49.573 INFO:tasks.workunit.client.0.smithi157.stdout: 2024-04-24T23:38:49.573 INFO:tasks.workunit.client.0.smithi157.stdout: Children see throughput for 1 readers = 7448705.50 kB/sec 2024-04-24T23:38:49.573 INFO:tasks.workunit.client.0.smithi157.stdout: Parent sees throughput for 1 readers = 7447680.36 kB/sec 2024-04-24T23:38:49.573 INFO:tasks.workunit.client.0.smithi157.stdout: Min throughput per process = 7448705.50 kB/sec 2024-04-24T23:38:49.573 INFO:tasks.workunit.client.0.smithi157.stdout: Max throughput per process = 7448705.50 kB/sec 2024-04-24T23:38:49.573 INFO:tasks.workunit.client.0.smithi157.stdout: Avg throughput per process = 7448705.50 kB/sec 2024-04-24T23:38:49.573 INFO:tasks.workunit.client.0.smithi157.stdout: Min xfer = 10485760.00 kB 2024-04-24T23:38:50.133 INFO:tasks.workunit.client.1.smithi169.stdout: 1 349063 16.71 MB/sec execute 8 sec latency 15.317 ms 2024-04-24T23:38:51.133 INFO:tasks.workunit.client.1.smithi169.stdout: 1 352226 16.63 MB/sec execute 9 sec latency 39.645 ms 2024-04-24T23:38:52.133 INFO:tasks.workunit.client.1.smithi169.stdout: 1 355680 16.77 MB/sec execute 10 sec latency 32.831 ms 2024-04-24T23:38:53.133 INFO:tasks.workunit.client.1.smithi169.stdout: 1 359123 16.96 MB/sec execute 11 sec latency 20.993 ms 2024-04-24T23:38:53.356 INFO:tasks.workunit.client.0.smithi157.stdout: 2024-04-24T23:38:53.356 INFO:tasks.workunit.client.0.smithi157.stdout: Children see throughput for 1 re-readers = 7688509.50 kB/sec 2024-04-24T23:38:53.356 INFO:tasks.workunit.client.0.smithi157.stdout: Parent sees throughput for 1 re-readers = 7687417.10 kB/sec 2024-04-24T23:38:53.356 INFO:tasks.workunit.client.0.smithi157.stdout: Min throughput per process = 7688509.50 kB/sec 2024-04-24T23:38:53.356 INFO:tasks.workunit.client.0.smithi157.stdout: Max throughput per process = 7688509.50 kB/sec 2024-04-24T23:38:53.356 INFO:tasks.workunit.client.0.smithi157.stdout: Avg throughput per process = 7688509.50 kB/sec 2024-04-24T23:38:53.356 INFO:tasks.workunit.client.0.smithi157.stdout: Min xfer = 10485760.00 kB 2024-04-24T23:38:54.133 INFO:tasks.workunit.client.1.smithi169.stdout: 1 362513 17.11 MB/sec execute 12 sec latency 29.663 ms 2024-04-24T23:38:55.133 INFO:tasks.workunit.client.1.smithi169.stdout: 1 366006 17.26 MB/sec execute 13 sec latency 16.231 ms 2024-04-24T23:38:56.133 INFO:tasks.workunit.client.1.smithi169.stdout: 1 369483 17.38 MB/sec execute 14 sec latency 19.351 ms 2024-04-24T23:38:57.133 INFO:tasks.workunit.client.1.smithi169.stdout: 1 372839 17.50 MB/sec execute 15 sec latency 20.635 ms 2024-04-24T23:38:58.133 INFO:tasks.workunit.client.1.smithi169.stdout: 1 376385 17.60 MB/sec execute 16 sec latency 21.702 ms 2024-04-24T23:38:59.133 INFO:tasks.workunit.client.1.smithi169.stdout: 1 379685 17.61 MB/sec execute 17 sec latency 51.071 ms 2024-04-24T23:39:00.111 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:39:00.114 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:39:00.133 INFO:tasks.workunit.client.1.smithi169.stdout: 1 382856 17.51 MB/sec execute 18 sec latency 16.175 ms 2024-04-24T23:39:01.133 INFO:tasks.workunit.client.1.smithi169.stdout: 1 385506 17.21 MB/sec execute 19 sec latency 24.621 ms 2024-04-24T23:39:01.851 INFO:tasks.workunit.client.0.smithi157.stdout: 2024-04-24T23:39:01.851 INFO:tasks.workunit.client.0.smithi157.stdout: 2024-04-24T23:39:01.851 INFO:tasks.workunit.client.0.smithi157.stdout: 2024-04-24T23:39:01.851 INFO:tasks.workunit.client.0.smithi157.stdout:iozone test complete. 2024-04-24T23:39:01.852 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-24T23:39:01.852 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-24T23:39:01.903 INFO:tasks.workunit:Stopping ['suites/iozone.sh'] on client.0... 2024-04-24T23:39:01.904 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-24T23:39:02.133 INFO:tasks.workunit.client.1.smithi169.stdout: 1 386761 16.88 MB/sec execute 20 sec latency 63.419 ms 2024-04-24T23:39:02.334 DEBUG:teuthology.parallel:result is None 2024-04-24T23:39:02.337 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-24T23:39:02.389 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-24T23:39:02.389 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-24T23:39:02.390 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-24T23:39:02.390 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:39:02.415 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-24T23:39:02.415 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:39:02.483 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:39:02.689 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.688+0000 7f9ff77a2640 1 Processor -- start 2024-04-24T23:39:02.689 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.688+0000 7f9ff77a2640 1 -- start start 2024-04-24T23:39:02.693 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.689+0000 7f9ff77a2640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9ff0152b40 0x7f9ff00c5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:02.693 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.689+0000 7f9ff77a2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ff00c8bb0 con 0x7f9ff00be360 2024-04-24T23:39:02.693 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.689+0000 7f9ff77a2640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ff0159480 con 0x7f9ff00c5b70 2024-04-24T23:39:02.693 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.689+0000 7f9ff77a2640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f9ff00c8e90 con 0x7f9ff0152b40 2024-04-24T23:39:02.693 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.690+0000 7f9ff4d3d640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f9ff00be360 0x7f9ff0155350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:44642/0 (socket says 172.21.15.157:44642) 2024-04-24T23:39:02.693 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.690+0000 7f9ff4d3d640 1 -- 172.21.15.157:0/2014292051 learned_addr learned my addr 172.21.15.157:0/2014292051 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-24T23:39:02.693 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.690+0000 7f9fe7fff640 1 -- 172.21.15.157:0/2014292051 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2667664878 0 0) 0x7f9ff00c8bb0 con 0x7f9ff00be360 2024-04-24T23:39:02.693 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.690+0000 7f9fe7fff640 1 -- 172.21.15.157:0/2014292051 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9fdc003520 con 0x7f9ff00be360 2024-04-24T23:39:02.693 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.691+0000 7f9fe7fff640 1 -- 172.21.15.157:0/2014292051 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 422572345 0 0) 0x7f9fdc003520 con 0x7f9ff00be360 2024-04-24T23:39:02.694 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.691+0000 7f9ff553e640 1 --2- 172.21.15.157:0/2014292051 >> v2:172.21.15.169:3300/0 conn(0x7f9ff0152b40 0x7f9ff00c5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:02.694 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.691+0000 7f9fe7fff640 1 -- 172.21.15.157:0/2014292051 >> v2:172.21.15.169:3300/0 conn(0x7f9ff0152b40 msgr2=0x7f9ff00c5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:39:02.694 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.692+0000 7f9fe7fff640 1 --2- 172.21.15.157:0/2014292051 >> v2:172.21.15.169:3300/0 conn(0x7f9ff0152b40 0x7f9ff00c5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:02.694 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.692+0000 7f9fe7fff640 1 -- 172.21.15.157:0/2014292051 >> v1:172.21.15.169:6789/0 conn(0x7f9ff00c5b70 legacy=0x7f9ff015fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:39:02.694 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.692+0000 7f9fe7fff640 1 -- 172.21.15.157:0/2014292051 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ff0159760 con 0x7f9ff00be360 2024-04-24T23:39:02.694 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.692+0000 7f9fe7fff640 1 -- 172.21.15.157:0/2014292051 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1480525340 0 0) 0x7f9fe80019f0 con 0x7f9ff00be360 2024-04-24T23:39:02.694 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.692+0000 7f9fe7fff640 1 -- 172.21.15.157:0/2014292051 >> v1:172.21.15.157:6789/0 conn(0x7f9ff00be360 legacy=0x7f9ff0155350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:02.694 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.692+0000 7f9fe7fff640 1 --2- 172.21.15.157:0/2014292051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9fdc003a00 0x7f9fdc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:02.694 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.692+0000 7f9fe7fff640 1 --2- 172.21.15.157:0/2014292051 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9fdc004800 0x7f9fdc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:02.694 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.692+0000 7f9fe7fff640 1 --2- 172.21.15.157:0/2014292051 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fdc005410 0x7f9fdc01ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:02.694 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.692+0000 7f9fe7fff640 1 -- 172.21.15.157:0/2014292051 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff00c8e90 con 0x7f9fdc003a00 2024-04-24T23:39:02.694 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.692+0000 7f9fe7fff640 1 -- 172.21.15.157:0/2014292051 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9fdc01d360 con 0x7f9fdc005410 2024-04-24T23:39:02.694 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.692+0000 7f9fe7fff640 1 -- 172.21.15.157:0/2014292051 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9fdc01d560 con 0x7f9fdc004800 2024-04-24T23:39:02.695 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.693+0000 7f9ff553e640 1 --2- 172.21.15.157:0/2014292051 >> v2:172.21.15.169:3300/0 conn(0x7f9ff0152b40 0x7f9ff00c5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:39:02.695 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.693+0000 7f9ff4d3d640 1 --2- 172.21.15.157:0/2014292051 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9fdc004800 0x7f9fdc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:02.695 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.693+0000 7f9ff5d3f640 1 --2- 172.21.15.157:0/2014292051 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fdc005410 0x7f9fdc01ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:02.695 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.693+0000 7f9ff553e640 1 --2- 172.21.15.157:0/2014292051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9fdc003a00 0x7f9fdc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:02.695 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.693+0000 7f9ff4d3d640 1 -- 172.21.15.157:0/2014292051 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fdc005410 msgr2=0x7f9fdc01ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:39:02.695 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.693+0000 7f9ff4d3d640 1 --2- 172.21.15.157:0/2014292051 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fdc005410 0x7f9fdc01ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:02.695 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.693+0000 7f9ff4d3d640 1 -- 172.21.15.157:0/2014292051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9fdc003a00 msgr2=0x7f9fdc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:39:02.695 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.694+0000 7f9ff4d3d640 1 --2- 172.21.15.157:0/2014292051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9fdc003a00 0x7f9fdc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:02.696 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.694+0000 7f9ff4d3d640 1 -- 172.21.15.157:0/2014292051 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fdc003520 con 0x7f9fdc004800 2024-04-24T23:39:02.696 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.694+0000 7f9ff5d3f640 1 -- 172.21.15.157:0/2014292051 reap_dead start 2024-04-24T23:39:02.696 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.694+0000 7f9ff553e640 1 --2- 172.21.15.157:0/2014292051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9fdc003a00 0x7f9fdc003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:39:02.696 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.695+0000 7f9ff4d3d640 1 --2- 172.21.15.157:0/2014292051 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9fdc004800 0x7f9fdc004c80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9fe80097a0 tx=0x7f9fdc005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41912623ba7a7a9d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:02.697 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.695+0000 7f9fe7fff640 1 -- 172.21.15.157:0/2014292051 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe80093f0 con 0x7f9fdc004800 2024-04-24T23:39:02.697 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.695+0000 7f9fe7fff640 1 -- 172.21.15.157:0/2014292051 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fe802f070 con 0x7f9fdc004800 2024-04-24T23:39:02.697 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.695+0000 7f9fe7fff640 1 -- 172.21.15.157:0/2014292051 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe802bca0 con 0x7f9fdc004800 2024-04-24T23:39:02.697 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.696+0000 7f9ff77a2640 1 -- 172.21.15.157:0/2014292051 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9fdc004800 msgr2=0x7f9fdc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:02.697 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.696+0000 7f9ff77a2640 1 --2- 172.21.15.157:0/2014292051 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9fdc004800 0x7f9fdc004c80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9fe80097a0 tx=0x7f9fdc005830 comp rx=0 tx=0).stop 2024-04-24T23:39:02.697 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.696+0000 7f9ff77a2640 1 -- 172.21.15.157:0/2014292051 shutdown_connections 2024-04-24T23:39:02.697 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.696+0000 7f9ff77a2640 1 --2- 172.21.15.157:0/2014292051 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9fdc004800 0x7f9fdc004c80 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:02.697 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.696+0000 7f9ff77a2640 1 -- 172.21.15.157:0/2014292051 >> 172.21.15.157:0/2014292051 conn(0x7f9ff00672c0 msgr2=0x7f9ff00676d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:39:02.697 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.696+0000 7f9ff77a2640 1 -- 172.21.15.157:0/2014292051 shutdown_connections 2024-04-24T23:39:02.697 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.696+0000 7f9ff77a2640 1 -- 172.21.15.157:0/2014292051 wait complete. 2024-04-24T23:39:02.699 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.699+0000 7f9ff77a2640 1 Processor -- start 2024-04-24T23:39:02.699 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.699+0000 7f9ff77a2640 1 -- start start 2024-04-24T23:39:02.702 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.700+0000 7f9ff77a2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fdc004800 0x7f9ff00acca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:02.702 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.700+0000 7f9ff77a2640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ff00c2d00 0x7f9ff01491b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:02.702 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.700+0000 7f9ff77a2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ff0158540 0x7f9ff01e74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:02.702 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.700+0000 7f9ff77a2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff00caa00 con 0x7f9ff0158540 2024-04-24T23:39:02.702 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.700+0000 7f9ff77a2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff00cb210 con 0x7f9fdc004800 2024-04-24T23:39:02.702 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.700+0000 7f9ff77a2640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9ff00cba40 con 0x7f9ff00c2d00 2024-04-24T23:39:02.702 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.700+0000 7f9ff553e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fdc004800 0x7f9ff00acca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:02.702 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.700+0000 7f9ff553e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fdc004800 0x7f9ff00acca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.157:49488/0 (socket says 172.21.15.157:49488) 2024-04-24T23:39:02.703 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.700+0000 7f9ff553e640 1 -- 172.21.15.157:0/334919083 learned_addr learned my addr 172.21.15.157:0/334919083 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:39:02.703 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.701+0000 7f9ff5d3f640 1 --2- 172.21.15.157:0/334919083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ff0158540 0x7f9ff01e74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:02.703 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.701+0000 7f9ff4d3d640 1 --2- 172.21.15.157:0/334919083 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ff00c2d00 0x7f9ff01491b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:02.703 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.701+0000 7f9ff553e640 1 -- 172.21.15.157:0/334919083 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ff00c2d00 msgr2=0x7f9ff01491b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:02.703 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.701+0000 7f9ff553e640 1 --2- 172.21.15.157:0/334919083 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ff00c2d00 0x7f9ff01491b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:02.703 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.701+0000 7f9ff553e640 1 -- 172.21.15.157:0/334919083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ff0158540 msgr2=0x7f9ff01e74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:02.704 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.702+0000 7f9ff553e640 1 --2- 172.21.15.157:0/334919083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ff0158540 0x7f9ff01e74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:02.704 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.702+0000 7f9ff553e640 1 -- 172.21.15.157:0/334919083 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ff01e7010 con 0x7f9fdc004800 2024-04-24T23:39:02.704 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.702+0000 7f9ff5d3f640 1 --2- 172.21.15.157:0/334919083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ff0158540 0x7f9ff01e74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:39:02.704 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.702+0000 7f9ff553e640 1 --2- 172.21.15.157:0/334919083 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fdc004800 0x7f9ff00acca0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9fec00fb50 tx=0x7f9fec00d590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:02.704 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.703+0000 7f9fe5ffb640 1 -- 172.21.15.157:0/334919083 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fec00a640 con 0x7f9fdc004800 2024-04-24T23:39:02.704 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.703+0000 7f9fe5ffb640 1 -- 172.21.15.157:0/334919083 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fec00fe40 con 0x7f9fdc004800 2024-04-24T23:39:02.704 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.703+0000 7f9fe5ffb640 1 -- 172.21.15.157:0/334919083 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fec004220 con 0x7f9fdc004800 2024-04-24T23:39:02.704 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.703+0000 7f9ff77a2640 1 -- 172.21.15.157:0/334919083 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ff0146250 con 0x7f9fdc004800 2024-04-24T23:39:02.705 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.704+0000 7f9ff77a2640 1 -- 172.21.15.157:0/334919083 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ff01eb2e0 con 0x7f9fdc004800 2024-04-24T23:39:02.707 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.707+0000 7f9fe5ffb640 1 -- 172.21.15.157:0/334919083 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9fec005990 con 0x7f9fdc004800 2024-04-24T23:39:02.708 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.707+0000 7f9fe5ffb640 1 --2- 172.21.15.157:0/334919083 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f9fd406bf00 0x7f9fd406e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:02.708 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.708+0000 7f9ff4d3d640 1 --2- 172.21.15.157:0/334919083 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f9fd406bf00 0x7f9fd406e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:02.708 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.708+0000 7f9fe5ffb640 1 -- 172.21.15.157:0/334919083 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f9fec08b030 con 0x7f9fdc004800 2024-04-24T23:39:02.709 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.707+0000 7f9ff77a2640 1 -- 172.21.15.157:0/334919083 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ff01eae30 con 0x7f9fdc004800 2024-04-24T23:39:02.712 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.712+0000 7f9ff4d3d640 1 --2- 172.21.15.157:0/334919083 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f9fd406bf00 0x7f9fd406e3c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8033000 tx=0x7f9fe8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:02.718 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:02.717+0000 7f9fe5ffb640 1 -- 172.21.15.157:0/334919083 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9fec017030 con 0x7f9fdc004800 2024-04-24T23:39:03.089 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.089+0000 7f9ff77a2640 1 -- 172.21.15.157:0/334919083 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9ff01e71e0 con 0x7f9fdc004800 2024-04-24T23:39:03.091 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.092+0000 7f9fe5ffb640 1 -- 172.21.15.157:0/334919083 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18928 (secure 0 0 0) 0x7f9fec059650 con 0x7f9fdc004800 2024-04-24T23:39:03.092 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:39:03.092 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":27,"fsid":"223685fb-5b64-46a5-9857-3791c9c07df2","created":"2024-04-24T23:34:37.504202+0000","modified":"2024-04-24T23:35:43.517316+0000","last_up_change":"2024-04-24T23:34:47.584963+0000","last_in_change":"2024-04-24T23:34:39.632954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:34:47.796655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:34:52.177208+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:34:52.667022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:34:56.395517+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"95d14ac8-fccf-4261-be8e-edafa746f629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6809","nonce":4003822160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6811","nonce":4003822160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6815","nonce":4003822160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6813","nonce":4003822160}]},"public_addr":"172.21.15.157:6809/4003822160","cluster_addr":"172.21.15.157:6811/4003822160","heartbeat_back_addr":"172.21.15.157:6815/4003822160","heartbeat_front_addr":"172.21.15.157:6813/4003822160","state":["exists","up"]},{"osd":1,"uuid":"7be79da9-8eec-4990-9178-71d602e39d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6817","nonce":1557575576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6819","nonce":1557575576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6823","nonce":1557575576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6821","nonce":1557575576}]},"public_addr":"172.21.15.157:6817/1557575576","cluster_addr":"172.21.15.157:6819/1557575576","heartbeat_back_addr":"172.21.15.157:6823/1557575576","heartbeat_front_addr":"172.21.15.157:6821/1557575576","state":["exists","up"]},{"osd":2,"uuid":"0127ed00-b7c3-466d-bfb0-01b0fd315f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6825","nonce":2793948092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6827","nonce":2793948092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6831","nonce":2793948092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6829","nonce":2793948092}]},"public_addr":"172.21.15.157:6825/2793948092","cluster_addr":"172.21.15.157:6827/2793948092","heartbeat_back_addr":"172.21.15.157:6831/2793948092","heartbeat_front_addr":"172.21.15.157:6829/2793948092","state":["exists","up"]},{"osd":3,"uuid":"f2331acc-aa37-433e-868e-3561b6039562","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6801","nonce":422758549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6803","nonce":422758549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6807","nonce":422758549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6805","nonce":422758549}]},"public_addr":"172.21.15.157:6801/422758549","cluster_addr":"172.21.15.157:6803/422758549","heartbeat_back_addr":"172.21.15.157:6807/422758549","heartbeat_front_addr":"172.21.15.157:6805/422758549","state":["exists","up"]},{"osd":4,"uuid":"1998854b-7398-43f5-ab7d-79fa0debcf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6801","nonce":321910099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6803","nonce":321910099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6807","nonce":321910099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6805","nonce":321910099}]},"public_addr":"172.21.15.169:6801/321910099","cluster_addr":"172.21.15.169:6803/321910099","heartbeat_back_addr":"172.21.15.169:6807/321910099","heartbeat_front_addr":"172.21.15.169:6805/321910099","state":["exists","up"]},{"osd":5,"uuid":"c37153ff-9b2b-48b9-a684-2c82b6f39f15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6825","nonce":243321711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6827","nonce":243321711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6831","nonce":243321711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6829","nonce":243321711}]},"public_addr":"172.21.15.169:6825/243321711","cluster_addr":"172.21.15.169:6827/243321711","heartbeat_back_addr":"172.21.15.169:6831/243321711","heartbeat_front_addr":"172.21.15.169:6829/243321711","state":["exists","up"]},{"osd":6,"uuid":"8db79e1c-f36b-49cb-8d8a-a20c2aec57c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6809","nonce":3441420785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6811","nonce":3441420785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6815","nonce":3441420785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6813","nonce":3441420785}]},"public_addr":"172.21.15.169:6809/3441420785","cluster_addr":"172.21.15.169:6811/3441420785","heartbeat_back_addr":"172.21.15.169:6815/3441420785","heartbeat_front_addr":"172.21.15.169:6813/3441420785","state":["exists","up"]},{"osd":7,"uuid":"928586e9-296c-4b8b-9b27-42f9f155dbdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6817","nonce":1663179020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6819","nonce":1663179020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6823","nonce":1663179020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6821","nonce":1663179020}]},"public_addr":"172.21.15.169:6817/1663179020","cluster_addr":"172.21.15.169:6819/1663179020","heartbeat_back_addr":"172.21.15.169:6823/1663179020","heartbeat_front_addr":"172.21.15.169:6821/1663179020","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.848039+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.826489+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.801120+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.727706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.824343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.791915+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.878073+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.810831+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.33","mappings":[{"from":3,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:39:03.095 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.096+0000 7f9ff77a2640 1 -- 172.21.15.157:0/334919083 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f9fd406bf00 msgr2=0x7f9fd406e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:03.095 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.096+0000 7f9ff77a2640 1 --2- 172.21.15.157:0/334919083 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f9fd406bf00 0x7f9fd406e3c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8033000 tx=0x7f9fe8008030 comp rx=0 tx=0).stop 2024-04-24T23:39:03.095 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.096+0000 7f9ff77a2640 1 -- 172.21.15.157:0/334919083 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fdc004800 msgr2=0x7f9ff00acca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:03.095 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.096+0000 7f9ff77a2640 1 --2- 172.21.15.157:0/334919083 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fdc004800 0x7f9ff00acca0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9fec00fb50 tx=0x7f9fec00d590 comp rx=0 tx=0).stop 2024-04-24T23:39:03.096 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.097+0000 7f9ff77a2640 1 -- 172.21.15.157:0/334919083 shutdown_connections 2024-04-24T23:39:03.096 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.097+0000 7f9ff77a2640 1 --2- 172.21.15.157:0/334919083 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f9fd406bf00 0x7f9fd406e3c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:03.096 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.097+0000 7f9ff77a2640 1 --2- 172.21.15.157:0/334919083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ff0158540 0x7f9ff01e74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:03.096 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.097+0000 7f9ff77a2640 1 --2- 172.21.15.157:0/334919083 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ff00c2d00 0x7f9ff01491b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:03.097 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.097+0000 7f9ff77a2640 1 --2- 172.21.15.157:0/334919083 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fdc004800 0x7f9ff00acca0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:03.097 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.097+0000 7f9ff77a2640 1 -- 172.21.15.157:0/334919083 >> 172.21.15.157:0/334919083 conn(0x7f9ff00672c0 msgr2=0x7f9ff0146aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:39:03.097 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.098+0000 7f9ff77a2640 1 -- 172.21.15.157:0/334919083 shutdown_connections 2024-04-24T23:39:03.097 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.098+0000 7f9ff77a2640 1 -- 172.21.15.157:0/334919083 wait complete. 2024-04-24T23:39:03.117 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:34:47.796655+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:34:52.177208+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:34:52.667022+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T23:34:56.395517+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T23:39:03.117 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:39:03.133 INFO:tasks.workunit.client.1.smithi169.stdout: 1 387411 16.16 MB/sec execute 21 sec latency 25.446 ms 2024-04-24T23:39:03.291 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.291+0000 7fcbb1316640 1 Processor -- start 2024-04-24T23:39:03.291 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.291+0000 7fcbb1316640 1 -- start start 2024-04-24T23:39:03.292 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.292+0000 7fcbb1316640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbac152c70 0x7fcbac0c51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:03.292 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.292+0000 7fcbb1316640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcbac06d8c0 0x7fcbac06dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:03.292 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.292+0000 7fcbb1316640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbac153050 con 0x7fcbac0c0690 2024-04-24T23:39:03.292 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.292+0000 7fcbb1316640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fcbac0bffb0 con 0x7fcbac06d8c0 2024-04-24T23:39:03.292 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.292+0000 7fcbb1316640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcbac158c90 con 0x7fcbac152c70 2024-04-24T23:39:03.292 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.293+0000 7fcbaaffd640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbac152c70 0x7fcbac0c51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:03.292 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.293+0000 7fcbab7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcbac06d8c0 0x7fcbac06dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:03.293 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.293+0000 7fcbab7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcbac06d8c0 0x7fcbac06dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58382/0 (socket says 172.21.15.157:58382) 2024-04-24T23:39:03.293 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.293+0000 7fcbab7fe640 1 -- 172.21.15.157:0/1106026125 learned_addr learned my addr 172.21.15.157:0/1106026125 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:39:03.294 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.294+0000 7fcbaaffd640 1 -- 172.21.15.157:0/1106026125 >> v2:172.21.15.157:3300/0 conn(0x7fcbac06d8c0 msgr2=0x7fcbac06dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:39:03.294 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.294+0000 7fcbaaffd640 1 --2- 172.21.15.157:0/1106026125 >> v2:172.21.15.157:3300/0 conn(0x7fcbac06d8c0 0x7fcbac06dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:03.294 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.294+0000 7fcbaaffd640 1 -- 172.21.15.157:0/1106026125 >> v1:172.21.15.157:6789/0 conn(0x7fcbac0c0690 legacy=0x7fcbac0c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:03.294 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.294+0000 7fcbaaffd640 1 -- 172.21.15.157:0/1106026125 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbac158f70 con 0x7fcbac152c70 2024-04-24T23:39:03.295 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.296+0000 7fcbaaffd640 1 --2- 172.21.15.157:0/1106026125 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbac152c70 0x7fcbac0c51f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c0097a0 tx=0x7fcb9c02a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=211f17447be56c93 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:03.295 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.296+0000 7fcba9ffb640 1 -- 172.21.15.157:0/1106026125 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c037070 con 0x7fcbac152c70 2024-04-24T23:39:03.295 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.296+0000 7fcba9ffb640 1 -- 172.21.15.157:0/1106026125 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb9c02adb0 con 0x7fcbac152c70 2024-04-24T23:39:03.295 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.296+0000 7fcba9ffb640 1 -- 172.21.15.157:0/1106026125 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c037070 con 0x7fcbac152c70 2024-04-24T23:39:03.297 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.297+0000 7fcbb1316640 1 -- 172.21.15.157:0/1106026125 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbac152c70 msgr2=0x7fcbac0c51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:03.297 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.297+0000 7fcbb1316640 1 --2- 172.21.15.157:0/1106026125 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbac152c70 0x7fcbac0c51f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c0097a0 tx=0x7fcb9c02a060 comp rx=0 tx=0).stop 2024-04-24T23:39:03.298 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.298+0000 7fcbb1316640 1 -- 172.21.15.157:0/1106026125 shutdown_connections 2024-04-24T23:39:03.298 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.298+0000 7fcbb1316640 1 --2- 172.21.15.157:0/1106026125 >> v2:172.21.15.157:3300/0 conn(0x7fcbac06d8c0 0x7fcbac06dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:03.298 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.298+0000 7fcbb1316640 1 --2- 172.21.15.157:0/1106026125 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbac152c70 0x7fcbac0c51f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:03.298 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.298+0000 7fcbb1316640 1 -- 172.21.15.157:0/1106026125 >> 172.21.15.157:0/1106026125 conn(0x7fcbac0c68e0 msgr2=0x7fcbac0c6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:39:03.298 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.298+0000 7fcbb1316640 1 -- 172.21.15.157:0/1106026125 shutdown_connections 2024-04-24T23:39:03.298 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.298+0000 7fcbb1316640 1 -- 172.21.15.157:0/1106026125 wait complete. 2024-04-24T23:39:03.299 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.300+0000 7fcbb1316640 1 Processor -- start 2024-04-24T23:39:03.300 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.300+0000 7fcbb1316640 1 -- start start 2024-04-24T23:39:03.301 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.301+0000 7fcbb1316640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbac06d8c0 0x7fcbac0a6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:03.301 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.302+0000 7fcbb1316640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbac0c0690 0x7fcbac1ee520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:03.301 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.302+0000 7fcbb1316640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbac152c70 0x7fcbac1e7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:03.301 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.302+0000 7fcbb1316640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcbac0c5ef0 con 0x7fcbac152c70 2024-04-24T23:39:03.302 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.302+0000 7fcbb1316640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcbac159530 con 0x7fcbac0c0690 2024-04-24T23:39:03.302 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.302+0000 7fcbb1316640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcbac0c6070 con 0x7fcbac06d8c0 2024-04-24T23:39:03.303 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.302+0000 7fcbaa7fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbac0c0690 0x7fcbac1ee520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:03.303 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.302+0000 7fcbaa7fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbac0c0690 0x7fcbac1ee520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.157:49490/0 (socket says 172.21.15.157:49490) 2024-04-24T23:39:03.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.302+0000 7fcbaa7fc640 1 -- 172.21.15.157:0/6694059 learned_addr learned my addr 172.21.15.157:0/6694059 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:39:03.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.303+0000 7fcbaaffd640 1 --2- 172.21.15.157:0/6694059 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbac06d8c0 0x7fcbac0a6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:03.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.303+0000 7fcbab7fe640 1 --2- 172.21.15.157:0/6694059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbac152c70 0x7fcbac1e7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:03.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.303+0000 7fcbaa7fc640 1 -- 172.21.15.157:0/6694059 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbac06d8c0 msgr2=0x7fcbac0a6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:03.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.303+0000 7fcbaa7fc640 1 --2- 172.21.15.157:0/6694059 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbac06d8c0 0x7fcbac0a6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:03.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.303+0000 7fcbaa7fc640 1 -- 172.21.15.157:0/6694059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbac152c70 msgr2=0x7fcbac1e7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:03.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.303+0000 7fcbaa7fc640 1 --2- 172.21.15.157:0/6694059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbac152c70 0x7fcbac1e7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:03.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.303+0000 7fcbaa7fc640 1 -- 172.21.15.157:0/6694059 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbac1eeff0 con 0x7fcbac0c0690 2024-04-24T23:39:03.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.303+0000 7fcbab7fe640 1 --2- 172.21.15.157:0/6694059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbac152c70 0x7fcbac1e7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:39:03.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.304+0000 7fcbaa7fc640 1 --2- 172.21.15.157:0/6694059 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbac0c0690 0x7fcbac1ee520 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcba000c450 tx=0x7fcba000c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:03.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.304+0000 7fcb937fe640 1 -- 172.21.15.157:0/6694059 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba00052d0 con 0x7fcbac0c0690 2024-04-24T23:39:03.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.304+0000 7fcb937fe640 1 -- 172.21.15.157:0/6694059 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcba0005470 con 0x7fcbac0c0690 2024-04-24T23:39:03.305 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.305+0000 7fcbb1316640 1 -- 172.21.15.157:0/6694059 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbac070de0 con 0x7fcbac0c0690 2024-04-24T23:39:03.305 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.305+0000 7fcbb1316640 1 -- 172.21.15.157:0/6694059 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbac155f40 con 0x7fcbac0c0690 2024-04-24T23:39:03.305 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.306+0000 7fcb937fe640 1 -- 172.21.15.157:0/6694059 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba0012870 con 0x7fcbac0c0690 2024-04-24T23:39:03.305 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.306+0000 7fcbb1316640 1 -- 172.21.15.157:0/6694059 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbac070c00 con 0x7fcbac0c0690 2024-04-24T23:39:03.307 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.307+0000 7fcb937fe640 1 -- 172.21.15.157:0/6694059 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcba0012a10 con 0x7fcbac0c0690 2024-04-24T23:39:03.307 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.308+0000 7fcb937fe640 1 --2- 172.21.15.157:0/6694059 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fcb9806be80 0x7fcb9806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:03.308 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.308+0000 7fcb937fe640 1 -- 172.21.15.157:0/6694059 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fcba008af80 con 0x7fcbac0c0690 2024-04-24T23:39:03.308 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.309+0000 7fcbaaffd640 1 --2- 172.21.15.157:0/6694059 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fcb9806be80 0x7fcb9806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:03.311 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.311+0000 7fcbaaffd640 1 --2- 172.21.15.157:0/6694059 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fcb9806be80 0x7fcb9806e340 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c034000 tx=0x7fcb9c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:03.314 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.314+0000 7fcb937fe640 1 -- 172.21.15.157:0/6694059 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcbac070c00 con 0x7fcbac0c0690 2024-04-24T23:39:03.648 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.648+0000 7fcbb1316640 1 -- 172.21.15.157:0/6694059 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcbac154a00 con 0x7fcbac0c0690 2024-04-24T23:39:03.648 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.649+0000 7fcb937fe640 1 -- 172.21.15.157:0/6694059 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7fcbac153050 con 0x7fcbac0c0690 2024-04-24T23:39:03.649 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2024-04-24T23:39:03.655 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.655+0000 7fcbb1316640 1 -- 172.21.15.157:0/6694059 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fcb9806be80 msgr2=0x7fcb9806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:03.655 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.655+0000 7fcbb1316640 1 --2- 172.21.15.157:0/6694059 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fcb9806be80 0x7fcb9806e340 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c034000 tx=0x7fcb9c02e000 comp rx=0 tx=0).stop 2024-04-24T23:39:03.655 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.655+0000 7fcbb1316640 1 -- 172.21.15.157:0/6694059 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbac0c0690 msgr2=0x7fcbac1ee520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:03.655 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.655+0000 7fcbb1316640 1 --2- 172.21.15.157:0/6694059 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbac0c0690 0x7fcbac1ee520 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcba000c450 tx=0x7fcba000c910 comp rx=0 tx=0).stop 2024-04-24T23:39:03.655 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.655+0000 7fcbb1316640 1 -- 172.21.15.157:0/6694059 shutdown_connections 2024-04-24T23:39:03.655 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.655+0000 7fcbb1316640 1 --2- 172.21.15.157:0/6694059 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fcb9806be80 0x7fcb9806e340 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:03.655 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.655+0000 7fcbb1316640 1 --2- 172.21.15.157:0/6694059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbac152c70 0x7fcbac1e7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:03.655 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.655+0000 7fcbb1316640 1 --2- 172.21.15.157:0/6694059 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbac0c0690 0x7fcbac1ee520 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:03.655 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.655+0000 7fcbb1316640 1 --2- 172.21.15.157:0/6694059 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbac06d8c0 0x7fcbac0a6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:03.655 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.655+0000 7fcbb1316640 1 -- 172.21.15.157:0/6694059 >> 172.21.15.157:0/6694059 conn(0x7fcbac0c68e0 msgr2=0x7fcbac06dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:39:03.656 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.655+0000 7fcbb1316640 1 -- 172.21.15.157:0/6694059 shutdown_connections 2024-04-24T23:39:03.656 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.655+0000 7fcbb1316640 1 -- 172.21.15.157:0/6694059 wait complete. 2024-04-24T23:39:03.679 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:39:03.891 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.891+0000 7f720b515640 1 Processor -- start 2024-04-24T23:39:03.893 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.892+0000 7f720b515640 1 -- start start 2024-04-24T23:39:03.893 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.893+0000 7f720b515640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f71fc047240 0x7f71fc0479e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:03.893 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.893+0000 7f720b515640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f71fc0deb90 0x7f71fc0defb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:03.893 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.893+0000 7f720b515640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71fc048090 con 0x7f71fc03fb00 2024-04-24T23:39:03.893 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.893+0000 7f720b515640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f71fc048980 con 0x7f71fc047240 2024-04-24T23:39:03.893 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.893+0000 7f720b515640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f71fc047700 con 0x7f71fc0deb90 2024-04-24T23:39:03.893 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.894+0000 7f720ad14640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f71fc0deb90 0x7f71fc0defb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:03.893 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.894+0000 7f7209d12640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f71fc047240 0x7f71fc0479e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:03.893 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.894+0000 7f720ad14640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f71fc0deb90 0x7f71fc0defb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.157:55392/0 (socket says 172.21.15.157:55392) 2024-04-24T23:39:03.893 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.894+0000 7f720ad14640 1 -- 172.21.15.157:0/359019397 learned_addr learned my addr 172.21.15.157:0/359019397 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:39:03.894 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.894+0000 7f7209d12640 1 -- 172.21.15.157:0/359019397 >> v2:172.21.15.169:3300/0 conn(0x7f71fc0deb90 msgr2=0x7f71fc0defb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:39:03.894 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.894+0000 7f7209d12640 1 --2- 172.21.15.157:0/359019397 >> v2:172.21.15.169:3300/0 conn(0x7f71fc0deb90 0x7f71fc0defb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:03.894 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.894+0000 7f7209d12640 1 -- 172.21.15.157:0/359019397 >> v1:172.21.15.157:6789/0 conn(0x7f71fc03fb00 legacy=0x7f71fc0dbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:03.895 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.895+0000 7f7209d12640 1 -- 172.21.15.157:0/359019397 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71fc048330 con 0x7f71fc047240 2024-04-24T23:39:03.897 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.897+0000 7f7209d12640 1 --2- 172.21.15.157:0/359019397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71fc047240 0x7f71fc0479e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f71fc043610 tx=0x7f71f402a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe3a5eee5b77535a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:03.897 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.897+0000 7f7209511640 1 -- 172.21.15.157:0/359019397 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f4037070 con 0x7f71fc047240 2024-04-24T23:39:03.897 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.897+0000 7f7209511640 1 -- 172.21.15.157:0/359019397 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71f402add0 con 0x7f71fc047240 2024-04-24T23:39:03.897 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.897+0000 7f7209511640 1 -- 172.21.15.157:0/359019397 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f4032660 con 0x7f71fc047240 2024-04-24T23:39:03.898 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.898+0000 7f720b515640 1 -- 172.21.15.157:0/359019397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71fc047240 msgr2=0x7f71fc0479e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:03.898 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.898+0000 7f720b515640 1 --2- 172.21.15.157:0/359019397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71fc047240 0x7f71fc0479e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f71fc043610 tx=0x7f71f402a080 comp rx=0 tx=0).stop 2024-04-24T23:39:03.898 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.898+0000 7f720b515640 1 -- 172.21.15.157:0/359019397 shutdown_connections 2024-04-24T23:39:03.898 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.898+0000 7f720b515640 1 --2- 172.21.15.157:0/359019397 >> v2:172.21.15.169:3300/0 conn(0x7f71fc0deb90 0x7f71fc0defb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:03.898 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.898+0000 7f720b515640 1 --2- 172.21.15.157:0/359019397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71fc047240 0x7f71fc0479e0 secure :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f71fc043610 tx=0x7f71f402a080 comp rx=0 tx=0).stop 2024-04-24T23:39:03.898 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.898+0000 7f720b515640 1 -- 172.21.15.157:0/359019397 >> 172.21.15.157:0/359019397 conn(0x7f71fc049340 msgr2=0x7f71fc049750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:39:03.900 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.899+0000 7f720b515640 1 -- 172.21.15.157:0/359019397 shutdown_connections 2024-04-24T23:39:03.901 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.899+0000 7f720b515640 1 -- 172.21.15.157:0/359019397 wait complete. 2024-04-24T23:39:03.901 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.901+0000 7f720b515640 1 Processor -- start 2024-04-24T23:39:03.901 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.901+0000 7f720b515640 1 -- start start 2024-04-24T23:39:03.902 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.902+0000 7f720b515640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71fc03fb00 0x7f71fc031ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:03.902 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.902+0000 7f720b515640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71fc0deb90 0x7f71fc0425c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:03.902 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.902+0000 7f720b515640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71fc0cc6e0 0x7f71fc0dfd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:03.902 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.902+0000 7f720b515640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f71fc04a200 con 0x7f71fc0deb90 2024-04-24T23:39:03.902 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.902+0000 7f720b515640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f71fc04a080 con 0x7f71fc0cc6e0 2024-04-24T23:39:03.902 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.902+0000 7f720b515640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f71fc0dc400 con 0x7f71fc03fb00 2024-04-24T23:39:03.903 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.903+0000 7f7209d12640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71fc0deb90 0x7f71fc0425c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:03.903 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.903+0000 7f7209d12640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71fc0deb90 0x7f71fc0425c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50956/0 (socket says 172.21.15.157:50956) 2024-04-24T23:39:03.904 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.903+0000 7f7209d12640 1 -- 172.21.15.157:0/3362910078 learned_addr learned my addr 172.21.15.157:0/3362910078 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:39:03.904 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.903+0000 7f720a513640 1 --2- 172.21.15.157:0/3362910078 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71fc03fb00 0x7f71fc031ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:03.904 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.903+0000 7f720ad14640 1 --2- 172.21.15.157:0/3362910078 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71fc0cc6e0 0x7f71fc0dfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:03.904 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.904+0000 7f720a513640 1 -- 172.21.15.157:0/3362910078 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71fc0cc6e0 msgr2=0x7f71fc0dfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:03.904 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.904+0000 7f720a513640 1 --2- 172.21.15.157:0/3362910078 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71fc0cc6e0 0x7f71fc0dfd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:03.904 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.904+0000 7f720a513640 1 -- 172.21.15.157:0/3362910078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71fc0deb90 msgr2=0x7f71fc0425c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:03.904 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.904+0000 7f720a513640 1 --2- 172.21.15.157:0/3362910078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71fc0deb90 0x7f71fc0425c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:03.904 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.904+0000 7f720a513640 1 -- 172.21.15.157:0/3362910078 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71fc0cd770 con 0x7f71fc03fb00 2024-04-24T23:39:03.904 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.904+0000 7f7209d12640 1 --2- 172.21.15.157:0/3362910078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71fc0deb90 0x7f71fc0425c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:39:03.904 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.904+0000 7f720a513640 1 --2- 172.21.15.157:0/3362910078 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71fc03fb00 0x7f71fc031ce0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f720000c3a0 tx=0x7f720000c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:03.905 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.905+0000 7f71faffd640 1 -- 172.21.15.157:0/3362910078 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7200017070 con 0x7f71fc03fb00 2024-04-24T23:39:03.905 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.905+0000 7f720b515640 1 -- 172.21.15.157:0/3362910078 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71fc0cdf30 con 0x7f71fc03fb00 2024-04-24T23:39:03.905 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.905+0000 7f720b515640 1 -- 172.21.15.157:0/3362910078 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f71fc0e2a20 con 0x7f71fc03fb00 2024-04-24T23:39:03.906 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.905+0000 7f71faffd640 1 -- 172.21.15.157:0/3362910078 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72000052d0 con 0x7f71fc03fb00 2024-04-24T23:39:03.906 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.905+0000 7f71faffd640 1 -- 172.21.15.157:0/3362910078 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7200003830 con 0x7f71fc03fb00 2024-04-24T23:39:03.907 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.908+0000 7f720b515640 1 -- 172.21.15.157:0/3362910078 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71ec005290 con 0x7f71fc03fb00 2024-04-24T23:39:03.907 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.908+0000 7f71faffd640 1 -- 172.21.15.157:0/3362910078 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f720001a070 con 0x7f71fc03fb00 2024-04-24T23:39:03.908 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.909+0000 7f71faffd640 1 --2- 172.21.15.157:0/3362910078 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f71e806be30 0x7f71e806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:03.908 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.909+0000 7f71faffd640 1 -- 172.21.15.157:0/3362910078 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f720008a450 con 0x7f71fc03fb00 2024-04-24T23:39:03.909 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.910+0000 7f7209d12640 1 --2- 172.21.15.157:0/3362910078 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f71e806be30 0x7f71e806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:03.913 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.914+0000 7f7209d12640 1 --2- 172.21.15.157:0/3362910078 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f71e806be30 0x7f71e806e2f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f71fc00bb70 tx=0x7f71f402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:03.916 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:03.917+0000 7f71faffd640 1 -- 172.21.15.157:0/3362910078 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f72000583a0 con 0x7f71fc03fb00 2024-04-24T23:39:04.133 INFO:tasks.workunit.client.1.smithi169.stdout: 1 387744 15.50 MB/sec execute 22 sec latency 233.388 ms 2024-04-24T23:39:04.182 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.182+0000 7f720b515640 1 -- 172.21.15.157:0/3362910078 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f71ec005d90 con 0x7f71fc03fb00 2024-04-24T23:39:04.182 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.183+0000 7f71faffd640 1 -- 172.21.15.157:0/3362910078 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f720005c6a0 con 0x7f71fc03fb00 2024-04-24T23:39:04.183 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2024-04-24T23:39:04.184 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.185+0000 7f720b515640 1 -- 172.21.15.157:0/3362910078 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f71e806be30 msgr2=0x7f71e806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:04.185 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.185+0000 7f720b515640 1 --2- 172.21.15.157:0/3362910078 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f71e806be30 0x7f71e806e2f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f71fc00bb70 tx=0x7f71f402e000 comp rx=0 tx=0).stop 2024-04-24T23:39:04.185 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.185+0000 7f720b515640 1 -- 172.21.15.157:0/3362910078 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71fc03fb00 msgr2=0x7f71fc031ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:04.185 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.185+0000 7f720b515640 1 --2- 172.21.15.157:0/3362910078 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71fc03fb00 0x7f71fc031ce0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f720000c3a0 tx=0x7f720000c860 comp rx=0 tx=0).stop 2024-04-24T23:39:04.185 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.185+0000 7f720b515640 1 -- 172.21.15.157:0/3362910078 shutdown_connections 2024-04-24T23:39:04.185 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.185+0000 7f720b515640 1 --2- 172.21.15.157:0/3362910078 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f71e806be30 0x7f71e806e2f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:04.185 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.185+0000 7f720b515640 1 --2- 172.21.15.157:0/3362910078 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71fc0cc6e0 0x7f71fc0dfd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:04.185 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.185+0000 7f720b515640 1 --2- 172.21.15.157:0/3362910078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71fc0deb90 0x7f71fc0425c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:04.185 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.185+0000 7f720b515640 1 --2- 172.21.15.157:0/3362910078 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71fc03fb00 0x7f71fc031ce0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:04.185 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.185+0000 7f720b515640 1 -- 172.21.15.157:0/3362910078 >> 172.21.15.157:0/3362910078 conn(0x7f71fc049340 msgr2=0x7f71fc00bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:39:04.185 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.185+0000 7f720b515640 1 -- 172.21.15.157:0/3362910078 shutdown_connections 2024-04-24T23:39:04.185 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.185+0000 7f720b515640 1 -- 172.21.15.157:0/3362910078 wait complete. 2024-04-24T23:39:04.195 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:39:04.296 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.296+0000 7fb9ebbb0640 1 Processor -- start 2024-04-24T23:39:04.296 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.297+0000 7fb9ebbb0640 1 -- start start 2024-04-24T23:39:04.299 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.297+0000 7fb9ebbb0640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb9e40b03e0 0x7fb9e41567e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:04.299 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.297+0000 7fb9ebbb0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb9e40c55d0 0x7fb9e40c5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:04.299 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.297+0000 7fb9ebbb0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9e40cb330 con 0x7fb9e41592f0 2024-04-24T23:39:04.299 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.297+0000 7fb9ebbb0640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fb9e40b07c0 con 0x7fb9e40c55d0 2024-04-24T23:39:04.299 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.297+0000 7fb9ebbb0640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fb9e4156310 con 0x7fb9e40b03e0 2024-04-24T23:39:04.299 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.297+0000 7fb9ea14d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb9e40c55d0 0x7fb9e40c5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:04.299 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.297+0000 7fb9e914b640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fb9e41592f0 0x7fb9e4155c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49272/0 (socket says 172.21.15.157:49272) 2024-04-24T23:39:04.299 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.297+0000 7fb9e994c640 1 --2- 172.21.15.157:0/433952276 >> v2:172.21.15.169:3300/0 conn(0x7fb9e40b03e0 0x7fb9e41567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:04.299 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.297+0000 7fb9e914b640 1 -- 172.21.15.157:0/433952276 learned_addr learned my addr 172.21.15.157:0/433952276 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-24T23:39:04.299 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.297+0000 7fb9ea14d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb9e40c55d0 0x7fb9e40c5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50968/0 (socket says 172.21.15.157:50968) 2024-04-24T23:39:04.300 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.298+0000 7fb9ea14d640 1 -- 172.21.15.157:0/433952276 >> v2:172.21.15.169:3300/0 conn(0x7fb9e40b03e0 msgr2=0x7fb9e41567e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:39:04.300 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.298+0000 7fb9ea14d640 1 --2- 172.21.15.157:0/433952276 >> v2:172.21.15.169:3300/0 conn(0x7fb9e40b03e0 0x7fb9e41567e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:04.300 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.298+0000 7fb9ea14d640 1 -- 172.21.15.157:0/433952276 >> v1:172.21.15.157:6789/0 conn(0x7fb9e41592f0 legacy=0x7fb9e4155c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:04.300 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.298+0000 7fb9e894a640 1 -- 172.21.15.157:0/433952276 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 730253716 0 0) 0x7fb9e40cb330 con 0x7fb9e41592f0 2024-04-24T23:39:04.300 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.298+0000 7fb9ea14d640 1 -- 172.21.15.157:0/433952276 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9e40cb610 con 0x7fb9e40c55d0 2024-04-24T23:39:04.300 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.298+0000 7fb9ea14d640 1 --2- 172.21.15.157:0/433952276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9e40c55d0 0x7fb9e40c5a00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb9e40c8750 tx=0x7fb9dc02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec9f22ce54d3800b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:04.300 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.298+0000 7fb9e894a640 1 -- 172.21.15.157:0/433952276 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9dc037070 con 0x7fb9e40c55d0 2024-04-24T23:39:04.300 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.299+0000 7fb9e894a640 1 -- 172.21.15.157:0/433952276 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9dc032440 con 0x7fb9e40c55d0 2024-04-24T23:39:04.300 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.299+0000 7fb9e894a640 1 -- 172.21.15.157:0/433952276 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9dc02ae60 con 0x7fb9e40c55d0 2024-04-24T23:39:04.300 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.299+0000 7fb9ebbb0640 1 -- 172.21.15.157:0/433952276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9e40c55d0 msgr2=0x7fb9e40c5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:04.300 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.299+0000 7fb9ebbb0640 1 --2- 172.21.15.157:0/433952276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9e40c55d0 0x7fb9e40c5a00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb9e40c8750 tx=0x7fb9dc02a110 comp rx=0 tx=0).stop 2024-04-24T23:39:04.300 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.299+0000 7fb9ebbb0640 1 -- 172.21.15.157:0/433952276 shutdown_connections 2024-04-24T23:39:04.300 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.299+0000 7fb9ebbb0640 1 --2- 172.21.15.157:0/433952276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9e40c55d0 0x7fb9e40c5a00 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:04.301 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.299+0000 7fb9ebbb0640 1 --2- 172.21.15.157:0/433952276 >> v2:172.21.15.169:3300/0 conn(0x7fb9e40b03e0 0x7fb9e41567e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:04.301 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.299+0000 7fb9ebbb0640 1 -- 172.21.15.157:0/433952276 >> 172.21.15.157:0/433952276 conn(0x7fb9e4067080 msgr2=0x7fb9e4067490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:39:04.301 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.299+0000 7fb9ebbb0640 1 -- 172.21.15.157:0/433952276 shutdown_connections 2024-04-24T23:39:04.301 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.299+0000 7fb9ebbb0640 1 -- 172.21.15.157:0/433952276 wait complete. 2024-04-24T23:39:04.303 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.300+0000 7fb9ebbb0640 1 Processor -- start 2024-04-24T23:39:04.303 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.300+0000 7fb9ebbb0640 1 -- start start 2024-04-24T23:39:04.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.301+0000 7fb9ebbb0640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9e40b03e0 0x7fb9e414f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:04.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.301+0000 7fb9ebbb0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9e40c55d0 0x7fb9e40aeb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:04.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.301+0000 7fb9ebbb0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9e41592f0 0x7fb9e40c94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:04.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.301+0000 7fb9ebbb0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e41556e0 con 0x7fb9e41592f0 2024-04-24T23:39:04.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.301+0000 7fb9ebbb0640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e4155400 con 0x7fb9e40c55d0 2024-04-24T23:39:04.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.301+0000 7fb9ebbb0640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb9e4159780 con 0x7fb9e40b03e0 2024-04-24T23:39:04.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.301+0000 7fb9ea14d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9e41592f0 0x7fb9e40c94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:04.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.301+0000 7fb9ea14d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9e41592f0 0x7fb9e40c94f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50978/0 (socket says 172.21.15.157:50978) 2024-04-24T23:39:04.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.301+0000 7fb9ea14d640 1 -- 172.21.15.157:0/3797169344 learned_addr learned my addr 172.21.15.157:0/3797169344 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:39:04.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.301+0000 7fb9e994c640 1 --2- 172.21.15.157:0/3797169344 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9e40b03e0 0x7fb9e414f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:04.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.301+0000 7fb9e914b640 1 --2- 172.21.15.157:0/3797169344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9e40c55d0 0x7fb9e40aeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:04.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.301+0000 7fb9ea14d640 1 -- 172.21.15.157:0/3797169344 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9e40b03e0 msgr2=0x7fb9e414f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:04.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.301+0000 7fb9ea14d640 1 --2- 172.21.15.157:0/3797169344 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9e40b03e0 0x7fb9e414f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:04.305 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.301+0000 7fb9ea14d640 1 -- 172.21.15.157:0/3797169344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9e40c55d0 msgr2=0x7fb9e40aeb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:04.305 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.301+0000 7fb9ea14d640 1 --2- 172.21.15.157:0/3797169344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9e40c55d0 0x7fb9e40aeb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:04.305 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.301+0000 7fb9ea14d640 1 -- 172.21.15.157:0/3797169344 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9e4157820 con 0x7fb9e41592f0 2024-04-24T23:39:04.305 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.302+0000 7fb9ea14d640 1 --2- 172.21.15.157:0/3797169344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9e41592f0 0x7fb9e40c94f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc035040 tx=0x7fb9dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:04.305 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.302+0000 7fb9da7fc640 1 -- 172.21.15.157:0/3797169344 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9dc037040 con 0x7fb9e41592f0 2024-04-24T23:39:04.305 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.302+0000 7fb9da7fc640 1 -- 172.21.15.157:0/3797169344 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9dc042070 con 0x7fb9e41592f0 2024-04-24T23:39:04.305 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.302+0000 7fb9ebbb0640 1 -- 172.21.15.157:0/3797169344 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9e414f640 con 0x7fb9e41592f0 2024-04-24T23:39:04.305 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.302+0000 7fb9da7fc640 1 -- 172.21.15.157:0/3797169344 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9dc0308a0 con 0x7fb9e41592f0 2024-04-24T23:39:04.305 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.302+0000 7fb9ebbb0640 1 -- 172.21.15.157:0/3797169344 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9e414ee60 con 0x7fb9e41592f0 2024-04-24T23:39:04.305 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.303+0000 7fb9ebbb0640 1 -- 172.21.15.157:0/3797169344 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9e40cafc0 con 0x7fb9e41592f0 2024-04-24T23:39:04.305 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.303+0000 7fb9da7fc640 1 -- 172.21.15.157:0/3797169344 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb9dc030a40 con 0x7fb9e41592f0 2024-04-24T23:39:04.305 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.303+0000 7fb9da7fc640 1 --2- 172.21.15.157:0/3797169344 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fb9c806be30 0x7fb9c806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:04.305 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.304+0000 7fb9da7fc640 1 -- 172.21.15.157:0/3797169344 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fb9dc0ab930 con 0x7fb9e41592f0 2024-04-24T23:39:04.305 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.304+0000 7fb9e994c640 1 --2- 172.21.15.157:0/3797169344 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fb9c806be30 0x7fb9c806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:04.306 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.304+0000 7fb9e994c640 1 --2- 172.21.15.157:0/3797169344 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fb9c806be30 0x7fb9c806e2f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb9e414ff60 tx=0x7fb9e000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:04.307 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.308+0000 7fb9da7fc640 1 -- 172.21.15.157:0/3797169344 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb9e40cafc0 con 0x7fb9e41592f0 2024-04-24T23:39:04.490 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.487+0000 7fb9ebbb0640 1 -- 172.21.15.157:0/3797169344 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb9e406d1d0 con 0x7fb9e41592f0 2024-04-24T23:39:04.491 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.488+0000 7fb9da7fc640 1 -- 172.21.15.157:0/3797169344 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7fb9e414f640 con 0x7fb9e41592f0 2024-04-24T23:39:04.491 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 64 2024-04-24T23:39:04.492 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.490+0000 7fb9ebbb0640 1 -- 172.21.15.157:0/3797169344 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fb9c806be30 msgr2=0x7fb9c806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:04.493 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.490+0000 7fb9ebbb0640 1 --2- 172.21.15.157:0/3797169344 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fb9c806be30 0x7fb9c806e2f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb9e414ff60 tx=0x7fb9e000b040 comp rx=0 tx=0).stop 2024-04-24T23:39:04.493 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.490+0000 7fb9ebbb0640 1 -- 172.21.15.157:0/3797169344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9e41592f0 msgr2=0x7fb9e40c94f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:04.493 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.490+0000 7fb9ebbb0640 1 --2- 172.21.15.157:0/3797169344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9e41592f0 0x7fb9e40c94f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc035040 tx=0x7fb9dc0046c0 comp rx=0 tx=0).stop 2024-04-24T23:39:04.493 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.490+0000 7fb9ebbb0640 1 -- 172.21.15.157:0/3797169344 shutdown_connections 2024-04-24T23:39:04.493 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.490+0000 7fb9ebbb0640 1 --2- 172.21.15.157:0/3797169344 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fb9c806be30 0x7fb9c806e2f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:04.493 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.490+0000 7fb9ebbb0640 1 --2- 172.21.15.157:0/3797169344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9e41592f0 0x7fb9e40c94f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:04.493 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.490+0000 7fb9ebbb0640 1 --2- 172.21.15.157:0/3797169344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9e40c55d0 0x7fb9e40aeb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:04.494 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.490+0000 7fb9ebbb0640 1 --2- 172.21.15.157:0/3797169344 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9e40b03e0 0x7fb9e414f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:04.494 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.490+0000 7fb9ebbb0640 1 -- 172.21.15.157:0/3797169344 >> 172.21.15.157:0/3797169344 conn(0x7fb9e4067080 msgr2=0x7fb9e4067460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:39:04.494 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.490+0000 7fb9ebbb0640 1 -- 172.21.15.157:0/3797169344 shutdown_connections 2024-04-24T23:39:04.494 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.490+0000 7fb9ebbb0640 1 -- 172.21.15.157:0/3797169344 wait complete. 2024-04-24T23:39:04.500 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T23:39:04.604 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.605+0000 7f7d42129640 1 Processor -- start 2024-04-24T23:39:04.604 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.605+0000 7f7d42129640 1 -- start start 2024-04-24T23:39:04.606 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.605+0000 7f7d42129640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d3c0b03e0 0x7f7d3c1567e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:04.606 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.605+0000 7f7d42129640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7d3c1592f0 0x7f7d3c155c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:04.606 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.605+0000 7f7d42129640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d3c0cb330 con 0x7f7d3c0c55d0 2024-04-24T23:39:04.606 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.605+0000 7f7d42129640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f7d3c0b07c0 con 0x7f7d3c1592f0 2024-04-24T23:39:04.606 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.605+0000 7f7d42129640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7d3c0cb610 con 0x7f7d3c0b03e0 2024-04-24T23:39:04.606 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.605+0000 7f7d33fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7d3c1592f0 0x7f7d3c155c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:04.606 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.606+0000 7f7d3b7fe640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d3c0b03e0 0x7f7d3c1567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:04.607 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.606+0000 7f7d33fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7d3c1592f0 0x7f7d3c155c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50988/0 (socket says 172.21.15.157:50988) 2024-04-24T23:39:04.607 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.606+0000 7f7d33fff640 1 -- 172.21.15.157:0/3290754874 learned_addr learned my addr 172.21.15.157:0/3290754874 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:39:04.607 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.606+0000 7f7d3bfff640 1 --1- 172.21.15.157:0/3290754874 >> v1:172.21.15.169:6789/0 conn(0x7f7d3c0c55d0 0x7f7d3c0c75f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.157:38666/0 (socket says 172.21.15.157:38666) 2024-04-24T23:39:04.607 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.606+0000 7f7d33fff640 1 -- 172.21.15.157:0/3290754874 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d3c0b03e0 msgr2=0x7f7d3c1567e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:39:04.607 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.606+0000 7f7d33fff640 1 --2- 172.21.15.157:0/3290754874 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d3c0b03e0 0x7f7d3c1567e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:04.607 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.606+0000 7f7d33fff640 1 -- 172.21.15.157:0/3290754874 >> v1:172.21.15.169:6789/0 conn(0x7f7d3c0c55d0 legacy=0x7f7d3c0c75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:04.607 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.606+0000 7f7d3affd640 1 -- 172.21.15.157:0/3290754874 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4076141586 0 0) 0x7f7d3c0cb330 con 0x7f7d3c0c55d0 2024-04-24T23:39:04.607 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.606+0000 7f7d33fff640 1 -- 172.21.15.157:0/3290754874 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d3c0cb8f0 con 0x7f7d3c1592f0 2024-04-24T23:39:04.607 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.606+0000 7f7d33fff640 1 --2- 172.21.15.157:0/3290754874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d3c1592f0 0x7f7d3c155c00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7d24009850 tx=0x7f7d2402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ef38bae12e10d58 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:04.607 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.607+0000 7f7d3affd640 1 -- 172.21.15.157:0/3290754874 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d24004030 con 0x7f7d3c1592f0 2024-04-24T23:39:04.607 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.607+0000 7f7d3affd640 1 -- 172.21.15.157:0/3290754874 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d240041d0 con 0x7f7d3c1592f0 2024-04-24T23:39:04.607 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.607+0000 7f7d3affd640 1 -- 172.21.15.157:0/3290754874 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d24003610 con 0x7f7d3c1592f0 2024-04-24T23:39:04.610 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.607+0000 7f7d42129640 1 -- 172.21.15.157:0/3290754874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d3c1592f0 msgr2=0x7f7d3c155c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:04.610 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.607+0000 7f7d42129640 1 --2- 172.21.15.157:0/3290754874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d3c1592f0 0x7f7d3c155c00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7d24009850 tx=0x7f7d2402a1a0 comp rx=0 tx=0).stop 2024-04-24T23:39:04.610 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.607+0000 7f7d42129640 1 -- 172.21.15.157:0/3290754874 shutdown_connections 2024-04-24T23:39:04.610 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.607+0000 7f7d42129640 1 --2- 172.21.15.157:0/3290754874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d3c1592f0 0x7f7d3c155c00 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:04.610 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.607+0000 7f7d42129640 1 --2- 172.21.15.157:0/3290754874 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d3c0b03e0 0x7f7d3c1567e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:04.610 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.607+0000 7f7d42129640 1 -- 172.21.15.157:0/3290754874 >> 172.21.15.157:0/3290754874 conn(0x7f7d3c067080 msgr2=0x7f7d3c067490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:39:04.611 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.607+0000 7f7d42129640 1 -- 172.21.15.157:0/3290754874 shutdown_connections 2024-04-24T23:39:04.611 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.607+0000 7f7d42129640 1 -- 172.21.15.157:0/3290754874 wait complete. 2024-04-24T23:39:04.611 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.608+0000 7f7d42129640 1 Processor -- start 2024-04-24T23:39:04.611 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.608+0000 7f7d42129640 1 -- start start 2024-04-24T23:39:04.611 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.609+0000 7f7d42129640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d3c0b03e0 0x7f7d3c0abee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:04.611 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.609+0000 7f7d42129640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d3c0c55d0 0x7f7d3c1ead90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:04.611 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.609+0000 7f7d42129640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d3c1592f0 0x7f7d3c0aabb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:04.611 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.609+0000 7f7d42129640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c0cafc0 con 0x7f7d3c0b03e0 2024-04-24T23:39:04.611 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.609+0000 7f7d42129640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c1556e0 con 0x7f7d3c0c55d0 2024-04-24T23:39:04.611 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.609+0000 7f7d42129640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7d3c159750 con 0x7f7d3c1592f0 2024-04-24T23:39:04.611 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.609+0000 7f7d3b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d3c0b03e0 0x7f7d3c0abee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:04.611 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.609+0000 7f7d3b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d3c0b03e0 0x7f7d3c0abee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50998/0 (socket says 172.21.15.157:50998) 2024-04-24T23:39:04.611 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.609+0000 7f7d3b7fe640 1 -- 172.21.15.157:0/1693942403 learned_addr learned my addr 172.21.15.157:0/1693942403 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:39:04.611 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.609+0000 7f7d33fff640 1 --2- 172.21.15.157:0/1693942403 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d3c0c55d0 0x7f7d3c1ead90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:04.612 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.609+0000 7f7d3bfff640 1 --2- 172.21.15.157:0/1693942403 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d3c1592f0 0x7f7d3c0aabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:04.612 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.609+0000 7f7d3b7fe640 1 -- 172.21.15.157:0/1693942403 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d3c1592f0 msgr2=0x7f7d3c0aabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:04.612 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.609+0000 7f7d3b7fe640 1 --2- 172.21.15.157:0/1693942403 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d3c1592f0 0x7f7d3c0aabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:04.612 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.609+0000 7f7d3b7fe640 1 -- 172.21.15.157:0/1693942403 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d3c0c55d0 msgr2=0x7f7d3c1ead90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:04.612 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.609+0000 7f7d3b7fe640 1 --2- 172.21.15.157:0/1693942403 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d3c0c55d0 0x7f7d3c1ead90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:04.612 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.609+0000 7f7d3b7fe640 1 -- 172.21.15.157:0/1693942403 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d3c0abb40 con 0x7f7d3c0b03e0 2024-04-24T23:39:04.612 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.610+0000 7f7d3b7fe640 1 --2- 172.21.15.157:0/1693942403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d3c0b03e0 0x7f7d3c0abee0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c00e930 tx=0x7f7d2c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:04.612 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.610+0000 7f7d38ff9640 1 -- 172.21.15.157:0/1693942403 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c017070 con 0x7f7d3c0b03e0 2024-04-24T23:39:04.612 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.610+0000 7f7d38ff9640 1 -- 172.21.15.157:0/1693942403 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d2c00ca00 con 0x7f7d3c0b03e0 2024-04-24T23:39:04.612 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.610+0000 7f7d42129640 1 -- 172.21.15.157:0/1693942403 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d3c0ab580 con 0x7f7d3c0b03e0 2024-04-24T23:39:04.612 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.610+0000 7f7d38ff9640 1 -- 172.21.15.157:0/1693942403 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c00cdf0 con 0x7f7d3c0b03e0 2024-04-24T23:39:04.612 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.610+0000 7f7d42129640 1 -- 172.21.15.157:0/1693942403 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d3c0aa690 con 0x7f7d3c0b03e0 2024-04-24T23:39:04.612 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.611+0000 7f7d42129640 1 -- 172.21.15.157:0/1693942403 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d3c0681c0 con 0x7f7d3c0b03e0 2024-04-24T23:39:04.613 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.611+0000 7f7d38ff9640 1 -- 172.21.15.157:0/1693942403 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7d2c01a070 con 0x7f7d3c0b03e0 2024-04-24T23:39:04.613 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.612+0000 7f7d38ff9640 1 --2- 172.21.15.157:0/1693942403 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f7d1c06be30 0x7f7d1c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:04.613 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.612+0000 7f7d38ff9640 1 -- 172.21.15.157:0/1693942403 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f7d2c08b180 con 0x7f7d3c0b03e0 2024-04-24T23:39:04.613 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.612+0000 7f7d33fff640 1 --2- 172.21.15.157:0/1693942403 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f7d1c06be30 0x7f7d1c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:04.613 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.612+0000 7f7d33fff640 1 --2- 172.21.15.157:0/1693942403 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f7d1c06be30 0x7f7d1c06e2f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7d24033000 tx=0x7f7d2402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:04.615 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.616+0000 7f7d38ff9640 1 -- 172.21.15.157:0/1693942403 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7d2c059230 con 0x7f7d3c0b03e0 2024-04-24T23:39:04.808 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.809+0000 7f7d42129640 1 -- 172.21.15.157:0/1693942403 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f7d3c0c15b0 con 0x7f7d3c0b03e0 2024-04-24T23:39:04.809 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.809+0000 7f7d38ff9640 1 -- 172.21.15.157:0/1693942403 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f7d2c05d530 con 0x7f7d3c0b03e0 2024-04-24T23:39:04.809 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2024-04-24T23:39:04.811 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.811+0000 7f7d42129640 1 -- 172.21.15.157:0/1693942403 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f7d1c06be30 msgr2=0x7f7d1c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:04.811 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.811+0000 7f7d42129640 1 --2- 172.21.15.157:0/1693942403 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f7d1c06be30 0x7f7d1c06e2f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7d24033000 tx=0x7f7d2402e000 comp rx=0 tx=0).stop 2024-04-24T23:39:04.811 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.811+0000 7f7d42129640 1 -- 172.21.15.157:0/1693942403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d3c0b03e0 msgr2=0x7f7d3c0abee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:04.811 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.811+0000 7f7d42129640 1 --2- 172.21.15.157:0/1693942403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d3c0b03e0 0x7f7d3c0abee0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c00e930 tx=0x7f7d2c00edf0 comp rx=0 tx=0).stop 2024-04-24T23:39:04.811 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.811+0000 7f7d42129640 1 -- 172.21.15.157:0/1693942403 shutdown_connections 2024-04-24T23:39:04.811 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.811+0000 7f7d42129640 1 --2- 172.21.15.157:0/1693942403 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f7d1c06be30 0x7f7d1c06e2f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:04.811 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.811+0000 7f7d42129640 1 --2- 172.21.15.157:0/1693942403 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d3c1592f0 0x7f7d3c0aabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:04.812 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.811+0000 7f7d42129640 1 --2- 172.21.15.157:0/1693942403 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d3c0c55d0 0x7f7d3c1ead90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:04.812 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.811+0000 7f7d42129640 1 --2- 172.21.15.157:0/1693942403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d3c0b03e0 0x7f7d3c0abee0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:04.812 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.811+0000 7f7d42129640 1 -- 172.21.15.157:0/1693942403 >> 172.21.15.157:0/1693942403 conn(0x7f7d3c067080 msgr2=0x7f7d3c0c63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:39:04.812 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.811+0000 7f7d42129640 1 -- 172.21.15.157:0/1693942403 shutdown_connections 2024-04-24T23:39:04.812 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.811+0000 7f7d42129640 1 -- 172.21.15.157:0/1693942403 wait complete. 2024-04-24T23:39:04.822 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T23:39:04.924 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.925+0000 7f6de0b63640 1 Processor -- start 2024-04-24T23:39:04.925 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.925+0000 7f6de0b63640 1 -- start start 2024-04-24T23:39:04.926 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.926+0000 7f6de0b63640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6ddc073110 0x7f6ddc0c12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:04.926 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.926+0000 7f6de0b63640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ddc156240 con 0x7f6ddc14f960 2024-04-24T23:39:04.926 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.926+0000 7f6de0b63640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ddc0a63f0 con 0x7f6ddc072bc0 2024-04-24T23:39:04.927 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.926+0000 7f6de0b63640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f6ddc06ed30 con 0x7f6ddc073110 2024-04-24T23:39:04.927 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.926+0000 7f6dda59c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f6ddc14f960 0x7f6ddc0a5ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49288/0 (socket says 172.21.15.157:49288) 2024-04-24T23:39:04.927 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.926+0000 7f6dda59c640 1 -- 172.21.15.157:0/4233022516 learned_addr learned my addr 172.21.15.157:0/4233022516 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-24T23:39:04.927 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.926+0000 7f6ddad9d640 1 --2- 172.21.15.157:0/4233022516 >> v2:172.21.15.169:3300/0 conn(0x7f6ddc073110 0x7f6ddc0c12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:04.927 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.926+0000 7f6dd9d9b640 1 -- 172.21.15.157:0/4233022516 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2751375702 0 0) 0x7f6ddc156240 con 0x7f6ddc14f960 2024-04-24T23:39:04.927 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.926+0000 7f6dd9d9b640 1 -- 172.21.15.157:0/4233022516 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6dc8003520 con 0x7f6ddc14f960 2024-04-24T23:39:04.927 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.926+0000 7f6dd9d9b640 1 -- 172.21.15.157:0/4233022516 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2002631342 0 0) 0x7f6ddc0a63f0 con 0x7f6ddc072bc0 2024-04-24T23:39:04.927 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.926+0000 7f6dd9d9b640 1 -- 172.21.15.157:0/4233022516 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ddc156240 con 0x7f6ddc072bc0 2024-04-24T23:39:04.927 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.926+0000 7f6ddad9d640 1 -- 172.21.15.157:0/4233022516 >> v1:172.21.15.169:6789/0 conn(0x7f6ddc072bc0 legacy=0x7f6ddc155790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:04.927 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.926+0000 7f6ddad9d640 1 -- 172.21.15.157:0/4233022516 >> v1:172.21.15.157:6789/0 conn(0x7f6ddc14f960 legacy=0x7f6ddc0a5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:04.927 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.926+0000 7f6dd9d9b640 1 -- 172.21.15.157:0/4233022516 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3850541915 0 0) 0x7f6dc8003520 con 0x7f6ddc14f960 2024-04-24T23:39:04.927 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.926+0000 7f6ddad9d640 1 -- 172.21.15.157:0/4233022516 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ddc1564e0 con 0x7f6ddc073110 2024-04-24T23:39:04.927 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.927+0000 7f6ddad9d640 1 --2- 172.21.15.157:0/4233022516 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ddc073110 0x7f6ddc0c12e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4009d90 tx=0x7f6dd402a560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3170025e9bbefd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:04.928 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.927+0000 7f6dd9d9b640 1 -- 172.21.15.157:0/4233022516 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc001670 con 0x7f6ddc073110 2024-04-24T23:39:04.928 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.927+0000 7f6dd9d9b640 1 -- 172.21.15.157:0/4233022516 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dd4007540 con 0x7f6ddc073110 2024-04-24T23:39:04.928 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.927+0000 7f6dd9d9b640 1 -- 172.21.15.157:0/4233022516 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd4007910 con 0x7f6ddc073110 2024-04-24T23:39:04.931 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.928+0000 7f6de0b63640 1 -- 172.21.15.157:0/4233022516 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ddc073110 msgr2=0x7f6ddc0c12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:04.931 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.928+0000 7f6de0b63640 1 --2- 172.21.15.157:0/4233022516 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ddc073110 0x7f6ddc0c12e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4009d90 tx=0x7f6dd402a560 comp rx=0 tx=0).stop 2024-04-24T23:39:04.931 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.928+0000 7f6de0b63640 1 -- 172.21.15.157:0/4233022516 shutdown_connections 2024-04-24T23:39:04.931 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.928+0000 7f6de0b63640 1 --2- 172.21.15.157:0/4233022516 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ddc073110 0x7f6ddc0c12e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:04.931 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.928+0000 7f6de0b63640 1 -- 172.21.15.157:0/4233022516 >> 172.21.15.157:0/4233022516 conn(0x7f6ddc0a8690 msgr2=0x7f6ddc0a8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:39:04.931 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.928+0000 7f6de0b63640 1 -- 172.21.15.157:0/4233022516 shutdown_connections 2024-04-24T23:39:04.931 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.928+0000 7f6de0b63640 1 -- 172.21.15.157:0/4233022516 wait complete. 2024-04-24T23:39:04.931 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.929+0000 7f6de0b63640 1 Processor -- start 2024-04-24T23:39:04.931 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.929+0000 7f6de0b63640 1 -- start start 2024-04-24T23:39:04.931 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.930+0000 7f6de0b63640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ddc072bc0 0x7f6ddc1eec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:04.932 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.930+0000 7f6de0b63640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ddc073110 0x7f6ddc154af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:04.932 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.930+0000 7f6de0b63640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ddc14f960 0x7f6ddc151900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:04.932 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.930+0000 7f6de0b63640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6ddc0c4dd0 con 0x7f6ddc072bc0 2024-04-24T23:39:04.932 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.930+0000 7f6de0b63640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6ddc072470 con 0x7f6ddc14f960 2024-04-24T23:39:04.932 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.930+0000 7f6de0b63640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6ddc0c4a60 con 0x7f6ddc073110 2024-04-24T23:39:04.932 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.930+0000 7f6dda59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ddc072bc0 0x7f6ddc1eec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:04.932 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.930+0000 7f6dda59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ddc072bc0 0x7f6ddc1eec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51000/0 (socket says 172.21.15.157:51000) 2024-04-24T23:39:04.932 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.930+0000 7f6dda59c640 1 -- 172.21.15.157:0/4063877173 learned_addr learned my addr 172.21.15.157:0/4063877173 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:39:04.932 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.930+0000 7f6ddad9d640 1 --2- 172.21.15.157:0/4063877173 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ddc14f960 0x7f6ddc151900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:04.932 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.930+0000 7f6dd3fff640 1 --2- 172.21.15.157:0/4063877173 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ddc073110 0x7f6ddc154af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:04.932 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.930+0000 7f6dda59c640 1 -- 172.21.15.157:0/4063877173 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ddc073110 msgr2=0x7f6ddc154af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:04.932 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.930+0000 7f6dda59c640 1 --2- 172.21.15.157:0/4063877173 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ddc073110 0x7f6ddc154af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:04.932 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.930+0000 7f6dda59c640 1 -- 172.21.15.157:0/4063877173 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ddc14f960 msgr2=0x7f6ddc151900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:04.932 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.930+0000 7f6dda59c640 1 --2- 172.21.15.157:0/4063877173 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ddc14f960 0x7f6ddc151900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:04.933 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.930+0000 7f6dda59c640 1 -- 172.21.15.157:0/4063877173 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ddc0c0980 con 0x7f6ddc072bc0 2024-04-24T23:39:04.933 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.930+0000 7f6dda59c640 1 --2- 172.21.15.157:0/4063877173 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ddc072bc0 0x7f6ddc1eec30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc00c5a0 tx=0x7f6dcc00ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:04.933 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.930+0000 7f6dd2ffd640 1 -- 172.21.15.157:0/4063877173 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc017070 con 0x7f6ddc072bc0 2024-04-24T23:39:04.933 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.930+0000 7f6dd2ffd640 1 -- 172.21.15.157:0/4063877173 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dcc005ce0 con 0x7f6ddc072bc0 2024-04-24T23:39:04.933 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.930+0000 7f6de0b63640 1 -- 172.21.15.157:0/4063877173 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ddc0c0c60 con 0x7f6ddc072bc0 2024-04-24T23:39:04.933 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.930+0000 7f6dd2ffd640 1 -- 172.21.15.157:0/4063877173 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc0072b0 con 0x7f6ddc072bc0 2024-04-24T23:39:04.933 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.930+0000 7f6de0b63640 1 -- 172.21.15.157:0/4063877173 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ddc1513e0 con 0x7f6ddc072bc0 2024-04-24T23:39:04.933 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.931+0000 7f6de0b63640 1 -- 172.21.15.157:0/4063877173 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ddc070970 con 0x7f6ddc072bc0 2024-04-24T23:39:04.933 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.932+0000 7f6dd2ffd640 1 -- 172.21.15.157:0/4063877173 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6dcc01a070 con 0x7f6ddc072bc0 2024-04-24T23:39:04.934 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.932+0000 7f6dd2ffd640 1 --2- 172.21.15.157:0/4063877173 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f6dbc06be30 0x7f6dbc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:04.934 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.932+0000 7f6dd2ffd640 1 -- 172.21.15.157:0/4063877173 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f6dcc08a740 con 0x7f6ddc072bc0 2024-04-24T23:39:04.934 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.932+0000 7f6dd3fff640 1 --2- 172.21.15.157:0/4063877173 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f6dbc06be30 0x7f6dbc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:04.934 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.933+0000 7f6dd3fff640 1 --2- 172.21.15.157:0/4063877173 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f6dbc06be30 0x7f6dbc06e2f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc1ebde0 tx=0x7f6dc4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:04.936 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:04.937+0000 7f6dd2ffd640 1 -- 172.21.15.157:0/4063877173 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6ddc070970 con 0x7f6ddc072bc0 2024-04-24T23:39:05.133 INFO:tasks.workunit.client.1.smithi169.stdout: 1 390522 15.56 MB/sec execute 23 sec latency 99.601 ms 2024-04-24T23:39:05.201 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.202+0000 7f6de0b63640 1 -- 172.21.15.157:0/4063877173 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6ddc0681c0 con 0x7f6ddc072bc0 2024-04-24T23:39:05.201 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.202+0000 7f6dd2ffd640 1 -- 172.21.15.157:0/4063877173 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f6dcc058610 con 0x7f6ddc072bc0 2024-04-24T23:39:05.202 INFO:teuthology.orchestra.run.smithi157.stderr:listed 0 entries 2024-04-24T23:39:05.206 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.206+0000 7f6de0b63640 1 -- 172.21.15.157:0/4063877173 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f6dbc06be30 msgr2=0x7f6dbc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:05.206 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.206+0000 7f6de0b63640 1 --2- 172.21.15.157:0/4063877173 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f6dbc06be30 0x7f6dbc06e2f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc1ebde0 tx=0x7f6dc4009040 comp rx=0 tx=0).stop 2024-04-24T23:39:05.206 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.206+0000 7f6de0b63640 1 -- 172.21.15.157:0/4063877173 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ddc072bc0 msgr2=0x7f6ddc1eec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:05.206 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.206+0000 7f6de0b63640 1 --2- 172.21.15.157:0/4063877173 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ddc072bc0 0x7f6ddc1eec30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc00c5a0 tx=0x7f6dcc00ca60 comp rx=0 tx=0).stop 2024-04-24T23:39:05.206 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.207+0000 7f6de0b63640 1 -- 172.21.15.157:0/4063877173 shutdown_connections 2024-04-24T23:39:05.206 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.207+0000 7f6de0b63640 1 --2- 172.21.15.157:0/4063877173 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f6dbc06be30 0x7f6dbc06e2f0 secure :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc1ebde0 tx=0x7f6dc4009040 comp rx=0 tx=0).stop 2024-04-24T23:39:05.207 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.207+0000 7f6de0b63640 1 --2- 172.21.15.157:0/4063877173 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ddc14f960 0x7f6ddc151900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:05.207 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.207+0000 7f6de0b63640 1 --2- 172.21.15.157:0/4063877173 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ddc073110 0x7f6ddc154af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:05.207 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.207+0000 7f6de0b63640 1 --2- 172.21.15.157:0/4063877173 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ddc072bc0 0x7f6ddc1eec30 secure :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc00c5a0 tx=0x7f6dcc00ca60 comp rx=0 tx=0).stop 2024-04-24T23:39:05.207 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.207+0000 7f6de0b63640 1 -- 172.21.15.157:0/4063877173 >> 172.21.15.157:0/4063877173 conn(0x7f6ddc0a8690 msgr2=0x7f6ddc0c3cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:39:05.207 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.208+0000 7f6de0b63640 1 -- 172.21.15.157:0/4063877173 shutdown_connections 2024-04-24T23:39:05.207 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.208+0000 7f6de0b63640 1 -- 172.21.15.157:0/4063877173 wait complete. 2024-04-24T23:39:05.225 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-24T23:39:05.225 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:39:05.254 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:39:05.461 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.461+0000 7f015a59d640 1 Processor -- start 2024-04-24T23:39:05.461 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.461+0000 7f015a59d640 1 -- start start 2024-04-24T23:39:05.463 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.462+0000 7f015a59d640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01540c1f80 0x7f0154159720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:05.463 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.463+0000 7f015a59d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f015406c400 0x7f01540c2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:05.463 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.463+0000 7f015a59d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01540c1ce0 con 0x7f015415cd10 2024-04-24T23:39:05.463 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.463+0000 7f015a59d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f015406d680 con 0x7f015406c400 2024-04-24T23:39:05.463 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.463+0000 7f015a59d640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f015406ca70 con 0x7f01540c1f80 2024-04-24T23:39:05.463 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.463+0000 7f0158b3a640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f015415cd10 0x7f015415d1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49302/0 (socket says 172.21.15.157:49302) 2024-04-24T23:39:05.463 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.463+0000 7f0158b3a640 1 -- 172.21.15.157:0/3763472334 learned_addr learned my addr 172.21.15.157:0/3763472334 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-24T23:39:05.463 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.463+0000 7f01537fe640 1 --2- 172.21.15.157:0/3763472334 >> v2:172.21.15.157:3300/0 conn(0x7f015406c400 0x7f01540c2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:05.463 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.463+0000 7f0153fff640 1 --2- 172.21.15.157:0/3763472334 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01540c1f80 0x7f0154159720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:05.464 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.464+0000 7f0152ffd640 1 -- 172.21.15.157:0/3763472334 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4064905725 0 0) 0x7f01540c1ce0 con 0x7f015415cd10 2024-04-24T23:39:05.464 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.464+0000 7f0152ffd640 1 -- 172.21.15.157:0/3763472334 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f013c003520 con 0x7f015415cd10 2024-04-24T23:39:05.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.464+0000 7f0152ffd640 1 -- 172.21.15.157:0/3763472334 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3148279257 0 0) 0x7f013c003520 con 0x7f015415cd10 2024-04-24T23:39:05.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.464+0000 7f0152ffd640 1 -- 172.21.15.157:0/3763472334 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01540c1f80 msgr2=0x7f0154159720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:39:05.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.464+0000 7f0152ffd640 1 --2- 172.21.15.157:0/3763472334 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01540c1f80 0x7f0154159720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:05.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.464+0000 7f0152ffd640 1 -- 172.21.15.157:0/3763472334 >> v2:172.21.15.157:3300/0 conn(0x7f015406c400 msgr2=0x7f01540c2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:39:05.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.464+0000 7f0152ffd640 1 --2- 172.21.15.157:0/3763472334 >> v2:172.21.15.157:3300/0 conn(0x7f015406c400 0x7f01540c2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:05.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.464+0000 7f0152ffd640 1 -- 172.21.15.157:0/3763472334 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f015406cd50 con 0x7f015415cd10 2024-04-24T23:39:05.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.464+0000 7f0152ffd640 1 -- 172.21.15.157:0/3763472334 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1480525340 0 0) 0x7f0148001670 con 0x7f015415cd10 2024-04-24T23:39:05.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.464+0000 7f0152ffd640 1 -- 172.21.15.157:0/3763472334 >> v1:172.21.15.157:6789/0 conn(0x7f015415cd10 legacy=0x7f015415d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:05.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.464+0000 7f0152ffd640 1 --2- 172.21.15.157:0/3763472334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f013c0039b0 0x7f013c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:05.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.464+0000 7f0152ffd640 1 --2- 172.21.15.157:0/3763472334 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f013c004800 0x7f013c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:05.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.464+0000 7f0152ffd640 1 --2- 172.21.15.157:0/3763472334 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f013c0051c0 0x7f013c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:05.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.465+0000 7f0152ffd640 1 -- 172.21.15.157:0/3763472334 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f015406d680 con 0x7f013c0039b0 2024-04-24T23:39:05.466 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.465+0000 7f0152ffd640 1 -- 172.21.15.157:0/3763472334 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f015406ca70 con 0x7f013c0051c0 2024-04-24T23:39:05.466 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.465+0000 7f0152ffd640 1 -- 172.21.15.157:0/3763472334 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f013c00bf80 con 0x7f013c004800 2024-04-24T23:39:05.466 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.465+0000 7f0158b3a640 1 --2- 172.21.15.157:0/3763472334 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f013c0051c0 0x7f013c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:05.466 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.465+0000 7f0153fff640 1 --2- 172.21.15.157:0/3763472334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f013c0039b0 0x7f013c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:05.466 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.465+0000 7f01537fe640 1 --2- 172.21.15.157:0/3763472334 >> v2:172.21.15.157:3300/0 conn(0x7f015406c400 0x7f01540c2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:39:05.466 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.466+0000 7f0158b3a640 1 -- 172.21.15.157:0/3763472334 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f013c004800 msgr2=0x7f013c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:39:05.466 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.466+0000 7f0158b3a640 1 --2- 172.21.15.157:0/3763472334 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f013c004800 0x7f013c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:05.467 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.466+0000 7f0158b3a640 1 -- 172.21.15.157:0/3763472334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f013c0039b0 msgr2=0x7f013c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:39:05.467 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.466+0000 7f0158b3a640 1 --2- 172.21.15.157:0/3763472334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f013c0039b0 0x7f013c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:05.467 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.466+0000 7f0158b3a640 1 -- 172.21.15.157:0/3763472334 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f013c003520 con 0x7f013c0051c0 2024-04-24T23:39:05.467 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.466+0000 7f0153fff640 1 --2- 172.21.15.157:0/3763472334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f013c0039b0 0x7f013c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:39:05.467 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.467+0000 7f0158b3a640 1 --2- 172.21.15.157:0/3763472334 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f013c0051c0 0x7f013c00b9c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f014800c1c0 tx=0x7f013c005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acf6702d9aa67442 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:05.469 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.469+0000 7f0152ffd640 1 -- 172.21.15.157:0/3763472334 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0148007cc0 con 0x7f013c0051c0 2024-04-24T23:39:05.469 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.469+0000 7f0152ffd640 1 -- 172.21.15.157:0/3763472334 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0148005af0 con 0x7f013c0051c0 2024-04-24T23:39:05.470 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.470+0000 7f015a59d640 1 -- 172.21.15.157:0/3763472334 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f013c0051c0 msgr2=0x7f013c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:05.470 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.470+0000 7f015a59d640 1 --2- 172.21.15.157:0/3763472334 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f013c0051c0 0x7f013c00b9c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f014800c1c0 tx=0x7f013c005830 comp rx=0 tx=0).stop 2024-04-24T23:39:05.470 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.470+0000 7f0158b3a640 1 -- 172.21.15.157:0/3763472334 reap_dead start 2024-04-24T23:39:05.470 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.470+0000 7f015a59d640 1 -- 172.21.15.157:0/3763472334 shutdown_connections 2024-04-24T23:39:05.470 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.470+0000 7f015a59d640 1 -- 172.21.15.157:0/3763472334 >> 172.21.15.157:0/3763472334 conn(0x7f01540a8720 msgr2=0x7f01540a8b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:39:05.471 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.471+0000 7f015a59d640 1 -- 172.21.15.157:0/3763472334 shutdown_connections 2024-04-24T23:39:05.471 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.471+0000 7f015a59d640 1 -- 172.21.15.157:0/3763472334 wait complete. 2024-04-24T23:39:05.472 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.472+0000 7f015a59d640 1 Processor -- start 2024-04-24T23:39:05.472 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.473+0000 7f015a59d640 1 -- start start 2024-04-24T23:39:05.474 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.474+0000 7f015a59d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01541ea0d0 0x7f01540c51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:05.474 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.474+0000 7f015a59d640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0154071680 0x7f01541ea4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:05.474 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.474+0000 7f015a59d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0154071b00 0x7f0154148810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:05.474 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.474+0000 7f015a59d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0154153a10 con 0x7f0154071b00 2024-04-24T23:39:05.474 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.474+0000 7f015a59d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f015406d310 con 0x7f01541ea0d0 2024-04-24T23:39:05.474 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.474+0000 7f015a59d640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0154153b90 con 0x7f0154071680 2024-04-24T23:39:05.474 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.474+0000 7f0153fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01541ea0d0 0x7f01540c51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:05.474 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.474+0000 7f0153fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01541ea0d0 0x7f01540c51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.157:55454/0 (socket says 172.21.15.157:55454) 2024-04-24T23:39:05.474 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.474+0000 7f0153fff640 1 -- 172.21.15.157:0/365252342 learned_addr learned my addr 172.21.15.157:0/365252342 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:39:05.474 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.474+0000 7f0158b3a640 1 --2- 172.21.15.157:0/365252342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0154071b00 0x7f0154148810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:05.475 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.475+0000 7f0153fff640 1 -- 172.21.15.157:0/365252342 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0154071680 msgr2=0x7f01541ea4b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-24T23:39:05.475 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.475+0000 7f0153fff640 1 --2- 172.21.15.157:0/365252342 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0154071680 0x7f01541ea4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:05.475 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.475+0000 7f0153fff640 1 -- 172.21.15.157:0/365252342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0154071b00 msgr2=0x7f0154148810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:05.475 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.475+0000 7f0153fff640 1 --2- 172.21.15.157:0/365252342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0154071b00 0x7f0154148810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:05.476 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.475+0000 7f0153fff640 1 -- 172.21.15.157:0/365252342 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0154153540 con 0x7f01541ea0d0 2024-04-24T23:39:05.476 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.475+0000 7f0153fff640 1 --2- 172.21.15.157:0/365252342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01541ea0d0 0x7f01540c51e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0144011c00 tx=0x7f014400f610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:05.476 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.475+0000 7f0150ff9640 1 -- 172.21.15.157:0/365252342 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0144011cf0 con 0x7f01541ea0d0 2024-04-24T23:39:05.476 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.476+0000 7f0150ff9640 1 -- 172.21.15.157:0/365252342 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0144011e90 con 0x7f01541ea0d0 2024-04-24T23:39:05.476 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.476+0000 7f0150ff9640 1 -- 172.21.15.157:0/365252342 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0144014660 con 0x7f01541ea0d0 2024-04-24T23:39:05.476 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.477+0000 7f015a59d640 1 -- 172.21.15.157:0/365252342 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f015414ad10 con 0x7f01541ea0d0 2024-04-24T23:39:05.476 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.477+0000 7f015a59d640 1 -- 172.21.15.157:0/365252342 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01540c47f0 con 0x7f01541ea0d0 2024-04-24T23:39:05.477 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.478+0000 7f015a59d640 1 -- 172.21.15.157:0/365252342 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01540681c0 con 0x7f01541ea0d0 2024-04-24T23:39:05.479 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.479+0000 7f0150ff9640 1 -- 172.21.15.157:0/365252342 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0144012040 con 0x7f01541ea0d0 2024-04-24T23:39:05.479 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.480+0000 7f0150ff9640 1 --2- 172.21.15.157:0/365252342 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f014006be80 0x7f014006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:05.479 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.480+0000 7f0150ff9640 1 -- 172.21.15.157:0/365252342 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f014408c150 con 0x7f01541ea0d0 2024-04-24T23:39:05.481 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.482+0000 7f01537fe640 1 --2- 172.21.15.157:0/365252342 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f014006be80 0x7f014006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:05.483 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.484+0000 7f01537fe640 1 --2- 172.21.15.157:0/365252342 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f014006be80 0x7f014006e340 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f014c004b10 tx=0x7f014c00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:05.487 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.487+0000 7f0150ff9640 1 -- 172.21.15.157:0/365252342 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0144019030 con 0x7f01541ea0d0 2024-04-24T23:39:05.736 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.736+0000 7f015a59d640 1 -- 172.21.15.157:0/365252342 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f01540c1810 con 0x7f01541ea0d0 2024-04-24T23:39:05.737 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.738+0000 7f0150ff9640 1 -- 172.21.15.157:0/365252342 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18928 (secure 0 0 0) 0x7f014405a870 con 0x7f01541ea0d0 2024-04-24T23:39:05.738 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-24T23:39:05.738 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":27,"fsid":"223685fb-5b64-46a5-9857-3791c9c07df2","created":"2024-04-24T23:34:37.504202+0000","modified":"2024-04-24T23:35:43.517316+0000","last_up_change":"2024-04-24T23:34:47.584963+0000","last_in_change":"2024-04-24T23:34:39.632954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:34:47.796655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:34:52.177208+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:34:52.667022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:34:56.395517+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"95d14ac8-fccf-4261-be8e-edafa746f629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6809","nonce":4003822160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6811","nonce":4003822160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6815","nonce":4003822160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6813","nonce":4003822160}]},"public_addr":"172.21.15.157:6809/4003822160","cluster_addr":"172.21.15.157:6811/4003822160","heartbeat_back_addr":"172.21.15.157:6815/4003822160","heartbeat_front_addr":"172.21.15.157:6813/4003822160","state":["exists","up"]},{"osd":1,"uuid":"7be79da9-8eec-4990-9178-71d602e39d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6817","nonce":1557575576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6819","nonce":1557575576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6823","nonce":1557575576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6821","nonce":1557575576}]},"public_addr":"172.21.15.157:6817/1557575576","cluster_addr":"172.21.15.157:6819/1557575576","heartbeat_back_addr":"172.21.15.157:6823/1557575576","heartbeat_front_addr":"172.21.15.157:6821/1557575576","state":["exists","up"]},{"osd":2,"uuid":"0127ed00-b7c3-466d-bfb0-01b0fd315f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6825","nonce":2793948092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6827","nonce":2793948092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6831","nonce":2793948092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6829","nonce":2793948092}]},"public_addr":"172.21.15.157:6825/2793948092","cluster_addr":"172.21.15.157:6827/2793948092","heartbeat_back_addr":"172.21.15.157:6831/2793948092","heartbeat_front_addr":"172.21.15.157:6829/2793948092","state":["exists","up"]},{"osd":3,"uuid":"f2331acc-aa37-433e-868e-3561b6039562","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6801","nonce":422758549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6803","nonce":422758549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6807","nonce":422758549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6805","nonce":422758549}]},"public_addr":"172.21.15.157:6801/422758549","cluster_addr":"172.21.15.157:6803/422758549","heartbeat_back_addr":"172.21.15.157:6807/422758549","heartbeat_front_addr":"172.21.15.157:6805/422758549","state":["exists","up"]},{"osd":4,"uuid":"1998854b-7398-43f5-ab7d-79fa0debcf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6801","nonce":321910099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6803","nonce":321910099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6807","nonce":321910099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6805","nonce":321910099}]},"public_addr":"172.21.15.169:6801/321910099","cluster_addr":"172.21.15.169:6803/321910099","heartbeat_back_addr":"172.21.15.169:6807/321910099","heartbeat_front_addr":"172.21.15.169:6805/321910099","state":["exists","up"]},{"osd":5,"uuid":"c37153ff-9b2b-48b9-a684-2c82b6f39f15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6825","nonce":243321711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6827","nonce":243321711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6831","nonce":243321711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6829","nonce":243321711}]},"public_addr":"172.21.15.169:6825/243321711","cluster_addr":"172.21.15.169:6827/243321711","heartbeat_back_addr":"172.21.15.169:6831/243321711","heartbeat_front_addr":"172.21.15.169:6829/243321711","state":["exists","up"]},{"osd":6,"uuid":"8db79e1c-f36b-49cb-8d8a-a20c2aec57c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6809","nonce":3441420785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6811","nonce":3441420785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6815","nonce":3441420785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6813","nonce":3441420785}]},"public_addr":"172.21.15.169:6809/3441420785","cluster_addr":"172.21.15.169:6811/3441420785","heartbeat_back_addr":"172.21.15.169:6815/3441420785","heartbeat_front_addr":"172.21.15.169:6813/3441420785","state":["exists","up"]},{"osd":7,"uuid":"928586e9-296c-4b8b-9b27-42f9f155dbdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6817","nonce":1663179020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6819","nonce":1663179020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6823","nonce":1663179020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6821","nonce":1663179020}]},"public_addr":"172.21.15.169:6817/1663179020","cluster_addr":"172.21.15.169:6819/1663179020","heartbeat_back_addr":"172.21.15.169:6823/1663179020","heartbeat_front_addr":"172.21.15.169:6821/1663179020","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.848039+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.826489+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.801120+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.727706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.824343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.791915+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.878073+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.810831+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.33","mappings":[{"from":3,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:39:05.740 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.741+0000 7f015a59d640 1 -- 172.21.15.157:0/365252342 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f014006be80 msgr2=0x7f014006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:05.740 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.741+0000 7f015a59d640 1 --2- 172.21.15.157:0/365252342 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f014006be80 0x7f014006e340 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f014c004b10 tx=0x7f014c00b040 comp rx=0 tx=0).stop 2024-04-24T23:39:05.740 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.741+0000 7f015a59d640 1 -- 172.21.15.157:0/365252342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01541ea0d0 msgr2=0x7f01540c51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:05.740 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.741+0000 7f015a59d640 1 --2- 172.21.15.157:0/365252342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01541ea0d0 0x7f01540c51e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0144011c00 tx=0x7f014400f610 comp rx=0 tx=0).stop 2024-04-24T23:39:05.741 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.741+0000 7f015a59d640 1 -- 172.21.15.157:0/365252342 shutdown_connections 2024-04-24T23:39:05.741 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.741+0000 7f015a59d640 1 --2- 172.21.15.157:0/365252342 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f014006be80 0x7f014006e340 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:05.741 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.741+0000 7f015a59d640 1 --2- 172.21.15.157:0/365252342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0154071b00 0x7f0154148810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:05.741 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.741+0000 7f015a59d640 1 --2- 172.21.15.157:0/365252342 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0154071680 0x7f01541ea4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:05.741 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.741+0000 7f015a59d640 1 --2- 172.21.15.157:0/365252342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01541ea0d0 0x7f01540c51e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:05.741 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.741+0000 7f015a59d640 1 -- 172.21.15.157:0/365252342 >> 172.21.15.157:0/365252342 conn(0x7f01540a8720 msgr2=0x7f01540a6840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:39:05.741 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.741+0000 7f015a59d640 1 -- 172.21.15.157:0/365252342 shutdown_connections 2024-04-24T23:39:05.741 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.741+0000 7f015a59d640 1 -- 172.21.15.157:0/365252342 wait complete. 2024-04-24T23:39:05.755 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:34:47.796655+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:34:52.177208+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:34:52.667022+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T23:34:56.395517+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T23:39:05.755 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:39:05.863 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.864+0000 7fb5ffa4d640 1 Processor -- start 2024-04-24T23:39:05.863 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.864+0000 7fb5ffa4d640 1 -- start start 2024-04-24T23:39:05.874 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.865+0000 7fb5ffa4d640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb5f80c3370 0x7fb5f8156390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:05.875 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.865+0000 7fb5ffa4d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb5f80a5c50 0x7fb5f8152cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:05.875 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.865+0000 7fb5ffa4d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5f80c3b80 con 0x7fb5f80c3750 2024-04-24T23:39:05.875 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.865+0000 7fb5ffa4d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fb5f8155f00 con 0x7fb5f80a5c50 2024-04-24T23:39:05.875 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.865+0000 7fb5ffa4d640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fb5f80736b0 con 0x7fb5f80c3370 2024-04-24T23:39:05.875 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.865+0000 7fb5fcfe8640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb5f80a5c50 0x7fb5f8152cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:05.876 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.865+0000 7fb5fdfea640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fb5f80c3750 0x7fb5f80c12e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49310/0 (socket says 172.21.15.157:49310) 2024-04-24T23:39:05.876 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.865+0000 7fb5fcfe8640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb5f80a5c50 0x7fb5f8152cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51032/0 (socket says 172.21.15.157:51032) 2024-04-24T23:39:05.876 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.865+0000 7fb5fdfea640 1 -- 172.21.15.157:0/1332830209 learned_addr learned my addr 172.21.15.157:0/1332830209 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-24T23:39:05.876 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.865+0000 7fb5fd7e9640 1 --2- 172.21.15.157:0/1332830209 >> v2:172.21.15.169:3300/0 conn(0x7fb5f80c3370 0x7fb5f8156390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:05.876 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.865+0000 7fb5fcfe8640 1 -- 172.21.15.157:0/1332830209 >> v2:172.21.15.169:3300/0 conn(0x7fb5f80c3370 msgr2=0x7fb5f8156390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:39:05.876 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.865+0000 7fb5fcfe8640 1 --2- 172.21.15.157:0/1332830209 >> v2:172.21.15.169:3300/0 conn(0x7fb5f80c3370 0x7fb5f8156390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:05.877 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.865+0000 7fb5fcfe8640 1 -- 172.21.15.157:0/1332830209 >> v1:172.21.15.157:6789/0 conn(0x7fb5f80c3750 legacy=0x7fb5f80c12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:05.877 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.865+0000 7fb5fcfe8640 1 -- 172.21.15.157:0/1332830209 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5f80a60e0 con 0x7fb5f80a5c50 2024-04-24T23:39:05.877 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.865+0000 7fb5effff640 1 -- 172.21.15.157:0/1332830209 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 875172845 0 0) 0x7fb5f80c3b80 con 0x7fb5f80c3750 2024-04-24T23:39:05.877 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.866+0000 7fb5fcfe8640 1 --2- 172.21.15.157:0/1332830209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5f80a5c50 0x7fb5f8152cc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb5f806ef70 tx=0x7fb5f402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8127f5cec8580937 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:05.877 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.866+0000 7fb5effff640 1 -- 172.21.15.157:0/1332830209 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f4037070 con 0x7fb5f80a5c50 2024-04-24T23:39:05.877 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.866+0000 7fb5effff640 1 -- 172.21.15.157:0/1332830209 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5f4032440 con 0x7fb5f80a5c50 2024-04-24T23:39:05.877 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.866+0000 7fb5effff640 1 -- 172.21.15.157:0/1332830209 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f4032810 con 0x7fb5f80a5c50 2024-04-24T23:39:05.877 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.867+0000 7fb5ffa4d640 1 -- 172.21.15.157:0/1332830209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5f80a5c50 msgr2=0x7fb5f8152cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:05.877 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.867+0000 7fb5ffa4d640 1 --2- 172.21.15.157:0/1332830209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5f80a5c50 0x7fb5f8152cc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb5f806ef70 tx=0x7fb5f402a1a0 comp rx=0 tx=0).stop 2024-04-24T23:39:05.877 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.867+0000 7fb5ffa4d640 1 -- 172.21.15.157:0/1332830209 shutdown_connections 2024-04-24T23:39:05.877 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.867+0000 7fb5ffa4d640 1 --2- 172.21.15.157:0/1332830209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5f80a5c50 0x7fb5f8152cc0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:05.878 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.867+0000 7fb5ffa4d640 1 --2- 172.21.15.157:0/1332830209 >> v2:172.21.15.169:3300/0 conn(0x7fb5f80c3370 0x7fb5f8156390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:05.878 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.867+0000 7fb5ffa4d640 1 -- 172.21.15.157:0/1332830209 >> 172.21.15.157:0/1332830209 conn(0x7fb5f80c68e0 msgr2=0x7fb5f80c6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:39:05.878 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.867+0000 7fb5ffa4d640 1 -- 172.21.15.157:0/1332830209 shutdown_connections 2024-04-24T23:39:05.878 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.867+0000 7fb5ffa4d640 1 -- 172.21.15.157:0/1332830209 wait complete. 2024-04-24T23:39:05.878 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.868+0000 7fb5ffa4d640 1 Processor -- start 2024-04-24T23:39:05.878 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.868+0000 7fb5ffa4d640 1 -- start start 2024-04-24T23:39:05.878 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.869+0000 7fb5ffa4d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5f80c3370 0x7fb5f81511f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:05.878 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.869+0000 7fb5ffa4d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5f80c3750 0x7fb5f81de670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:05.878 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.869+0000 7fb5ffa4d640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5f81debb0 0x7fb5f81e9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:05.878 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.869+0000 7fb5ffa4d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f8155b90 con 0x7fb5f80c3370 2024-04-24T23:39:05.878 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.869+0000 7fb5ffa4d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f81558b0 con 0x7fb5f80c3750 2024-04-24T23:39:05.878 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.869+0000 7fb5ffa4d640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb5f80c19f0 con 0x7fb5f81debb0 2024-04-24T23:39:05.878 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.869+0000 7fb5fd7e9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5f80c3370 0x7fb5f81511f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:05.879 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.869+0000 7fb5fd7e9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5f80c3370 0x7fb5f81511f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51038/0 (socket says 172.21.15.157:51038) 2024-04-24T23:39:05.879 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.869+0000 7fb5fd7e9640 1 -- 172.21.15.157:0/1415285054 learned_addr learned my addr 172.21.15.157:0/1415285054 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:39:05.879 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.869+0000 7fb5fcfe8640 1 --2- 172.21.15.157:0/1415285054 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5f80c3750 0x7fb5f81de670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:05.879 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.869+0000 7fb5fdfea640 1 --2- 172.21.15.157:0/1415285054 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5f81debb0 0x7fb5f81e9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:05.879 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.869+0000 7fb5fd7e9640 1 -- 172.21.15.157:0/1415285054 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5f81debb0 msgr2=0x7fb5f81e9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:05.879 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.869+0000 7fb5fd7e9640 1 --2- 172.21.15.157:0/1415285054 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5f81debb0 0x7fb5f81e9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:05.879 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.869+0000 7fb5fd7e9640 1 -- 172.21.15.157:0/1415285054 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5f80c3750 msgr2=0x7fb5f81de670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:05.879 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.869+0000 7fb5fd7e9640 1 --2- 172.21.15.157:0/1415285054 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5f80c3750 0x7fb5f81de670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:05.879 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.869+0000 7fb5fd7e9640 1 -- 172.21.15.157:0/1415285054 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5f81ec880 con 0x7fb5f80c3370 2024-04-24T23:39:05.879 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.869+0000 7fb5fdfea640 1 --2- 172.21.15.157:0/1415285054 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5f81debb0 0x7fb5f81e9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:39:05.879 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.869+0000 7fb5fd7e9640 1 --2- 172.21.15.157:0/1415285054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5f80c3370 0x7fb5f81511f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb5e800e9e0 tx=0x7fb5e800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:05.879 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.869+0000 7fb5edffb640 1 -- 172.21.15.157:0/1415285054 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e8017070 con 0x7fb5f80c3370 2024-04-24T23:39:05.879 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.869+0000 7fb5edffb640 1 -- 172.21.15.157:0/1415285054 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5e800c9d0 con 0x7fb5f80c3370 2024-04-24T23:39:05.880 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.869+0000 7fb5ffa4d640 1 -- 172.21.15.157:0/1415285054 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5f81ecb60 con 0x7fb5f80c3370 2024-04-24T23:39:05.880 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.869+0000 7fb5edffb640 1 -- 172.21.15.157:0/1415285054 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e800cdc0 con 0x7fb5f80c3370 2024-04-24T23:39:05.880 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.870+0000 7fb5ffa4d640 1 -- 172.21.15.157:0/1415285054 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5f8151730 con 0x7fb5f80c3370 2024-04-24T23:39:05.880 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.871+0000 7fb5ffa4d640 1 -- 172.21.15.157:0/1415285054 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5f80681c0 con 0x7fb5f80c3370 2024-04-24T23:39:05.880 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.871+0000 7fb5edffb640 1 -- 172.21.15.157:0/1415285054 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb5e801a070 con 0x7fb5f80c3370 2024-04-24T23:39:05.880 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.872+0000 7fb5edffb640 1 --2- 172.21.15.157:0/1415285054 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fb5dc06be30 0x7fb5dc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:05.880 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.872+0000 7fb5edffb640 1 -- 172.21.15.157:0/1415285054 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fb5e808b4a0 con 0x7fb5f80c3370 2024-04-24T23:39:05.880 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.872+0000 7fb5fcfe8640 1 --2- 172.21.15.157:0/1415285054 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fb5dc06be30 0x7fb5dc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:05.880 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.872+0000 7fb5fcfe8640 1 --2- 172.21.15.157:0/1415285054 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fb5dc06be30 0x7fb5dc06e2f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4034000 tx=0x7fb5f402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:05.880 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:05.876+0000 7fb5edffb640 1 -- 172.21.15.157:0/1415285054 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb5e8059490 con 0x7fb5f80c3370 2024-04-24T23:39:06.060 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.061+0000 7fb5ffa4d640 1 -- 172.21.15.157:0/1415285054 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb5f80c4240 con 0x7fb5f80c3370 2024-04-24T23:39:06.061 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.061+0000 7fb5edffb640 1 -- 172.21.15.157:0/1415285054 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7fb5e805d790 con 0x7fb5f80c3370 2024-04-24T23:39:06.061 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2024-04-24T23:39:06.063 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.063+0000 7fb5ffa4d640 1 -- 172.21.15.157:0/1415285054 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fb5dc06be30 msgr2=0x7fb5dc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:06.063 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.063+0000 7fb5ffa4d640 1 --2- 172.21.15.157:0/1415285054 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fb5dc06be30 0x7fb5dc06e2f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4034000 tx=0x7fb5f402e000 comp rx=0 tx=0).stop 2024-04-24T23:39:06.063 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.063+0000 7fb5ffa4d640 1 -- 172.21.15.157:0/1415285054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5f80c3370 msgr2=0x7fb5f81511f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:06.063 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.063+0000 7fb5ffa4d640 1 --2- 172.21.15.157:0/1415285054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5f80c3370 0x7fb5f81511f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb5e800e9e0 tx=0x7fb5e800eea0 comp rx=0 tx=0).stop 2024-04-24T23:39:06.063 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.063+0000 7fb5ffa4d640 1 -- 172.21.15.157:0/1415285054 shutdown_connections 2024-04-24T23:39:06.064 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.063+0000 7fb5ffa4d640 1 --2- 172.21.15.157:0/1415285054 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fb5dc06be30 0x7fb5dc06e2f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:06.064 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.063+0000 7fb5ffa4d640 1 --2- 172.21.15.157:0/1415285054 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5f81debb0 0x7fb5f81e9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:06.064 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.063+0000 7fb5ffa4d640 1 --2- 172.21.15.157:0/1415285054 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5f80c3750 0x7fb5f81de670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:06.064 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.063+0000 7fb5ffa4d640 1 --2- 172.21.15.157:0/1415285054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5f80c3370 0x7fb5f81511f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:06.064 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.063+0000 7fb5ffa4d640 1 -- 172.21.15.157:0/1415285054 >> 172.21.15.157:0/1415285054 conn(0x7fb5f80c68e0 msgr2=0x7fb5f80c0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:39:06.064 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.063+0000 7fb5ffa4d640 1 -- 172.21.15.157:0/1415285054 shutdown_connections 2024-04-24T23:39:06.064 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.063+0000 7fb5ffa4d640 1 -- 172.21.15.157:0/1415285054 wait complete. 2024-04-24T23:39:06.074 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:39:06.133 INFO:tasks.workunit.client.1.smithi169.stdout: 1 392579 15.21 MB/sec execute 24 sec latency 62.371 ms 2024-04-24T23:39:06.181 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.182+0000 7fc2414df640 1 Processor -- start 2024-04-24T23:39:06.181 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.182+0000 7fc2414df640 1 -- start start 2024-04-24T23:39:06.183 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.182+0000 7fc2414df640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc23c0c8c70 0x7fc23c158bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:06.183 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.182+0000 7fc2414df640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc23c159100 0x7fc23c15ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:06.183 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.182+0000 7fc2414df640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc23c0c8730 con 0x7fc23c152d90 2024-04-24T23:39:06.183 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.183+0000 7fc2414df640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fc23c156600 con 0x7fc23c0c8c70 2024-04-24T23:39:06.184 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.183+0000 7fc2414df640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc23c0c67c0 con 0x7fc23c159100 2024-04-24T23:39:06.184 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.183+0000 7fc233fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc23c0c8c70 0x7fc23c158bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:06.184 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.183+0000 7fc233fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc23c0c8c70 0x7fc23c158bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51048/0 (socket says 172.21.15.157:51048) 2024-04-24T23:39:06.184 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.183+0000 7fc233fff640 1 -- 172.21.15.157:0/3700975327 learned_addr learned my addr 172.21.15.157:0/3700975327 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:39:06.184 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.183+0000 7fc23b7fe640 1 --2- 172.21.15.157:0/3700975327 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc23c159100 0x7fc23c15ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:06.184 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.183+0000 7fc233fff640 1 -- 172.21.15.157:0/3700975327 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc23c159100 msgr2=0x7fc23c15ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:39:06.184 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.183+0000 7fc233fff640 1 --2- 172.21.15.157:0/3700975327 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc23c159100 0x7fc23c15ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:06.184 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.183+0000 7fc233fff640 1 -- 172.21.15.157:0/3700975327 >> v1:172.21.15.169:6789/0 conn(0x7fc23c152d90 legacy=0x7fc23c0c7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:06.184 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.183+0000 7fc233fff640 1 -- 172.21.15.157:0/3700975327 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc23c155840 con 0x7fc23c0c8c70 2024-04-24T23:39:06.184 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.184+0000 7fc233fff640 1 --2- 172.21.15.157:0/3700975327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc23c0c8c70 0x7fc23c158bc0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc2240098a0 tx=0x7fc22402a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dd19ebe82d51775 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:06.184 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.184+0000 7fc23a7fc640 1 -- 172.21.15.157:0/3700975327 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc224037070 con 0x7fc23c0c8c70 2024-04-24T23:39:06.184 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.184+0000 7fc23a7fc640 1 -- 172.21.15.157:0/3700975327 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc224032440 con 0x7fc23c0c8c70 2024-04-24T23:39:06.184 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.184+0000 7fc23a7fc640 1 -- 172.21.15.157:0/3700975327 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc224032810 con 0x7fc23c0c8c70 2024-04-24T23:39:06.187 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.184+0000 7fc2414df640 1 -- 172.21.15.157:0/3700975327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc23c0c8c70 msgr2=0x7fc23c158bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:06.187 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.184+0000 7fc2414df640 1 --2- 172.21.15.157:0/3700975327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc23c0c8c70 0x7fc23c158bc0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc2240098a0 tx=0x7fc22402a160 comp rx=0 tx=0).stop 2024-04-24T23:39:06.188 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.184+0000 7fc2414df640 1 -- 172.21.15.157:0/3700975327 shutdown_connections 2024-04-24T23:39:06.188 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.184+0000 7fc2414df640 1 --2- 172.21.15.157:0/3700975327 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc23c159100 0x7fc23c15ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:06.188 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.184+0000 7fc2414df640 1 --2- 172.21.15.157:0/3700975327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc23c0c8c70 0x7fc23c158bc0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:06.188 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.184+0000 7fc2414df640 1 -- 172.21.15.157:0/3700975327 >> 172.21.15.157:0/3700975327 conn(0x7fc23c0ac8e0 msgr2=0x7fc23c0accf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:39:06.188 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.184+0000 7fc2414df640 1 -- 172.21.15.157:0/3700975327 shutdown_connections 2024-04-24T23:39:06.188 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.184+0000 7fc2414df640 1 -- 172.21.15.157:0/3700975327 wait complete. 2024-04-24T23:39:06.188 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.186+0000 7fc2414df640 1 Processor -- start 2024-04-24T23:39:06.188 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.186+0000 7fc2414df640 1 -- start start 2024-04-24T23:39:06.188 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.186+0000 7fc2414df640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc23c0c8c70 0x7fc23c0c3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:06.188 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.186+0000 7fc2414df640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc23c152d90 0x7fc23c1e9240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:06.188 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.186+0000 7fc2414df640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc23c159100 0x7fc23c1eea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:06.188 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.186+0000 7fc2414df640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc23c0c7e90 con 0x7fc23c0c8c70 2024-04-24T23:39:06.189 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.186+0000 7fc2414df640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc23c155e00 con 0x7fc23c152d90 2024-04-24T23:39:06.189 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.186+0000 7fc2414df640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc23c0c8010 con 0x7fc23c159100 2024-04-24T23:39:06.189 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.186+0000 7fc23affd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc23c0c8c70 0x7fc23c0c3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:06.189 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.186+0000 7fc23affd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc23c0c8c70 0x7fc23c0c3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51050/0 (socket says 172.21.15.157:51050) 2024-04-24T23:39:06.189 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.186+0000 7fc23affd640 1 -- 172.21.15.157:0/2944806036 learned_addr learned my addr 172.21.15.157:0/2944806036 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:39:06.189 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.186+0000 7fc233fff640 1 --2- 172.21.15.157:0/2944806036 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc23c152d90 0x7fc23c1e9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:06.189 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.186+0000 7fc23b7fe640 1 --2- 172.21.15.157:0/2944806036 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc23c159100 0x7fc23c1eea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:06.189 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.187+0000 7fc23affd640 1 -- 172.21.15.157:0/2944806036 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc23c159100 msgr2=0x7fc23c1eea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:06.189 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.187+0000 7fc23affd640 1 --2- 172.21.15.157:0/2944806036 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc23c159100 0x7fc23c1eea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:06.189 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.187+0000 7fc23affd640 1 -- 172.21.15.157:0/2944806036 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc23c152d90 msgr2=0x7fc23c1e9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:06.189 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.187+0000 7fc23affd640 1 --2- 172.21.15.157:0/2944806036 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc23c152d90 0x7fc23c1e9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:06.189 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.187+0000 7fc23affd640 1 -- 172.21.15.157:0/2944806036 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc23c0c4cf0 con 0x7fc23c0c8c70 2024-04-24T23:39:06.189 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.187+0000 7fc23affd640 1 --2- 172.21.15.157:0/2944806036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc23c0c8c70 0x7fc23c0c3490 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc22c00c450 tx=0x7fc22c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:06.189 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.187+0000 7fc2337fe640 1 -- 172.21.15.157:0/2944806036 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22c017070 con 0x7fc23c0c8c70 2024-04-24T23:39:06.190 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.187+0000 7fc2337fe640 1 -- 172.21.15.157:0/2944806036 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc22c0052d0 con 0x7fc23c0c8c70 2024-04-24T23:39:06.190 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.187+0000 7fc2414df640 1 -- 172.21.15.157:0/2944806036 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc23c0c29d0 con 0x7fc23c0c8c70 2024-04-24T23:39:06.190 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.187+0000 7fc2337fe640 1 -- 172.21.15.157:0/2944806036 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22c003830 con 0x7fc23c0c8c70 2024-04-24T23:39:06.190 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.187+0000 7fc2414df640 1 -- 172.21.15.157:0/2944806036 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc23c0c2c90 con 0x7fc23c0c8c70 2024-04-24T23:39:06.190 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.188+0000 7fc2414df640 1 -- 172.21.15.157:0/2944806036 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc23c06b620 con 0x7fc23c0c8c70 2024-04-24T23:39:06.191 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.188+0000 7fc2337fe640 1 -- 172.21.15.157:0/2944806036 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc22c01a070 con 0x7fc23c0c8c70 2024-04-24T23:39:06.191 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.189+0000 7fc2337fe640 1 --2- 172.21.15.157:0/2944806036 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fc21c06be30 0x7fc21c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:06.191 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.189+0000 7fc2337fe640 1 -- 172.21.15.157:0/2944806036 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fc22c08a500 con 0x7fc23c0c8c70 2024-04-24T23:39:06.191 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.189+0000 7fc233fff640 1 --2- 172.21.15.157:0/2944806036 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fc21c06be30 0x7fc21c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:06.191 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.189+0000 7fc233fff640 1 --2- 172.21.15.157:0/2944806036 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fc21c06be30 0x7fc21c06e2f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc224034000 tx=0x7fc22402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:06.192 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.193+0000 7fc2337fe640 1 -- 172.21.15.157:0/2944806036 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc23c06b620 con 0x7fc23c0c8c70 2024-04-24T23:39:06.382 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.382+0000 7fc2414df640 1 -- 172.21.15.157:0/2944806036 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc23c0681c0 con 0x7fc23c0c8c70 2024-04-24T23:39:06.382 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.382+0000 7fc2337fe640 1 -- 172.21.15.157:0/2944806036 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7fc22c0584f0 con 0x7fc23c0c8c70 2024-04-24T23:39:06.382 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2024-04-24T23:39:06.384 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.384+0000 7fc2414df640 1 -- 172.21.15.157:0/2944806036 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fc21c06be30 msgr2=0x7fc21c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:06.384 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.384+0000 7fc2414df640 1 --2- 172.21.15.157:0/2944806036 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fc21c06be30 0x7fc21c06e2f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc224034000 tx=0x7fc22402e000 comp rx=0 tx=0).stop 2024-04-24T23:39:06.384 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.384+0000 7fc2414df640 1 -- 172.21.15.157:0/2944806036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc23c0c8c70 msgr2=0x7fc23c0c3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:06.384 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.384+0000 7fc2414df640 1 --2- 172.21.15.157:0/2944806036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc23c0c8c70 0x7fc23c0c3490 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc22c00c450 tx=0x7fc22c00c910 comp rx=0 tx=0).stop 2024-04-24T23:39:06.384 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.384+0000 7fc2414df640 1 -- 172.21.15.157:0/2944806036 shutdown_connections 2024-04-24T23:39:06.384 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.384+0000 7fc2414df640 1 --2- 172.21.15.157:0/2944806036 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fc21c06be30 0x7fc21c06e2f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:06.385 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.384+0000 7fc2414df640 1 --2- 172.21.15.157:0/2944806036 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc23c159100 0x7fc23c1eea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:06.385 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.384+0000 7fc2414df640 1 --2- 172.21.15.157:0/2944806036 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc23c152d90 0x7fc23c1e9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:06.385 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.384+0000 7fc2414df640 1 --2- 172.21.15.157:0/2944806036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc23c0c8c70 0x7fc23c0c3490 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:06.385 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.384+0000 7fc2414df640 1 -- 172.21.15.157:0/2944806036 >> 172.21.15.157:0/2944806036 conn(0x7fc23c0ac8e0 msgr2=0x7fc23c0c7780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:39:06.385 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.384+0000 7fc2414df640 1 -- 172.21.15.157:0/2944806036 shutdown_connections 2024-04-24T23:39:06.385 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.384+0000 7fc2414df640 1 -- 172.21.15.157:0/2944806036 wait complete. 2024-04-24T23:39:06.396 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:39:06.502 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.503+0000 7fb96b542640 1 Processor -- start 2024-04-24T23:39:06.502 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.503+0000 7fb96b542640 1 -- start start 2024-04-24T23:39:06.503 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.504+0000 7fb96b542640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9640732a0 0x7fb96406b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:06.503 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.504+0000 7fb96b542640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb9640c13c0 0x7fb96415fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:06.503 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.504+0000 7fb96b542640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9641555d0 con 0x7fb96406b6a0 2024-04-24T23:39:06.504 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.504+0000 7fb96b542640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fb9640c0690 con 0x7fb9640c13c0 2024-04-24T23:39:06.504 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.504+0000 7fb96b542640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb9640c0f30 con 0x7fb9640732a0 2024-04-24T23:39:06.504 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.504+0000 7fb969adf640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb9640c13c0 0x7fb96415fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:06.504 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.504+0000 7fb9692de640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9640732a0 0x7fb96406b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:06.504 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.504+0000 7fb968add640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fb96406b6a0 0x7fb9640c2770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.157:38704/0 (socket says 172.21.15.157:38704) 2024-04-24T23:39:06.504 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.504+0000 7fb968add640 1 -- 172.21.15.157:0/3829621531 learned_addr learned my addr 172.21.15.157:0/3829621531 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-24T23:39:06.504 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.504+0000 7fb969adf640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb9640c13c0 0x7fb96415fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.157:55472/0 (socket says 172.21.15.157:55472) 2024-04-24T23:39:06.504 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.504+0000 7fb9692de640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9640732a0 0x7fb96406b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.157:40882/0 (socket says 172.21.15.157:40882) 2024-04-24T23:39:06.504 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.504+0000 7fb969adf640 1 -- 172.21.15.157:0/3829621531 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9640732a0 msgr2=0x7fb96406b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:39:06.504 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.504+0000 7fb969adf640 1 --2- 172.21.15.157:0/3829621531 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9640732a0 0x7fb96406b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:06.504 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.504+0000 7fb969adf640 1 -- 172.21.15.157:0/3829621531 >> v1:172.21.15.169:6789/0 conn(0x7fb96406b6a0 legacy=0x7fb9640c2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:06.504 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.504+0000 7fb969adf640 1 -- 172.21.15.157:0/3829621531 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9640c08e0 con 0x7fb9640c13c0 2024-04-24T23:39:06.504 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.504+0000 7fb9692de640 1 --2- 172.21.15.157:0/3829621531 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9640732a0 0x7fb96406b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:39:06.506 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.505+0000 7fb969adf640 1 --2- 172.21.15.157:0/3829621531 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9640c13c0 0x7fb96415fd40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb95c009d90 tx=0x7fb95c02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bff22f10ea1a1ffe server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:06.506 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.505+0000 7fb95bfff640 1 -- 172.21.15.157:0/3829621531 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb95c007740 con 0x7fb9640c13c0 2024-04-24T23:39:06.506 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.505+0000 7fb95bfff640 1 -- 172.21.15.157:0/3829621531 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb95c0078e0 con 0x7fb9640c13c0 2024-04-24T23:39:06.506 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.505+0000 7fb95bfff640 1 -- 172.21.15.157:0/3829621531 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb95c007cb0 con 0x7fb9640c13c0 2024-04-24T23:39:06.506 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.506+0000 7fb96b542640 1 -- 172.21.15.157:0/3829621531 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9640c13c0 msgr2=0x7fb96415fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:06.506 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.506+0000 7fb96b542640 1 --2- 172.21.15.157:0/3829621531 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9640c13c0 0x7fb96415fd40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb95c009d90 tx=0x7fb95c02a660 comp rx=0 tx=0).stop 2024-04-24T23:39:06.506 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.506+0000 7fb96b542640 1 -- 172.21.15.157:0/3829621531 shutdown_connections 2024-04-24T23:39:06.506 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.506+0000 7fb96b542640 1 --2- 172.21.15.157:0/3829621531 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9640c13c0 0x7fb96415fd40 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:06.506 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.506+0000 7fb96b542640 1 --2- 172.21.15.157:0/3829621531 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9640732a0 0x7fb96406b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:06.506 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.506+0000 7fb96b542640 1 -- 172.21.15.157:0/3829621531 >> 172.21.15.157:0/3829621531 conn(0x7fb9640c3d60 msgr2=0x7fb9640c4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:39:06.506 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.506+0000 7fb96b542640 1 -- 172.21.15.157:0/3829621531 shutdown_connections 2024-04-24T23:39:06.506 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.506+0000 7fb96b542640 1 -- 172.21.15.157:0/3829621531 wait complete. 2024-04-24T23:39:06.509 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.507+0000 7fb96b542640 1 Processor -- start 2024-04-24T23:39:06.509 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.507+0000 7fb96b542640 1 -- start start 2024-04-24T23:39:06.509 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.507+0000 7fb96b542640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb96406b6a0 0x7fb9641ec8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:06.509 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.507+0000 7fb96b542640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9640732a0 0x7fb964143ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:06.509 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.507+0000 7fb96b542640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9640c13c0 0x7fb9641ebd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:06.509 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.507+0000 7fb96b542640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb9640c3a00 con 0x7fb96406b6a0 2024-04-24T23:39:06.509 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.507+0000 7fb96b542640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb9640711a0 con 0x7fb9640732a0 2024-04-24T23:39:06.509 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.507+0000 7fb96b542640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb964073680 con 0x7fb9640c13c0 2024-04-24T23:39:06.509 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.508+0000 7fb9692de640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb96406b6a0 0x7fb9641ec8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:06.509 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.508+0000 7fb9692de640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb96406b6a0 0x7fb9641ec8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51060/0 (socket says 172.21.15.157:51060) 2024-04-24T23:39:06.509 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.508+0000 7fb968add640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9640732a0 0x7fb964143ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:06.509 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.508+0000 7fb9692de640 1 -- 172.21.15.157:0/510707943 learned_addr learned my addr 172.21.15.157:0/510707943 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:39:06.509 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.508+0000 7fb969adf640 1 --2- 172.21.15.157:0/510707943 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9640c13c0 0x7fb9641ebd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:06.509 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.508+0000 7fb9692de640 1 -- 172.21.15.157:0/510707943 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9640c13c0 msgr2=0x7fb9641ebd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:06.510 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.508+0000 7fb9692de640 1 --2- 172.21.15.157:0/510707943 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9640c13c0 0x7fb9641ebd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:06.510 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.508+0000 7fb9692de640 1 -- 172.21.15.157:0/510707943 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9640732a0 msgr2=0x7fb964143ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:06.510 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.508+0000 7fb9692de640 1 --2- 172.21.15.157:0/510707943 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9640732a0 0x7fb964143ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:06.510 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.508+0000 7fb9692de640 1 -- 172.21.15.157:0/510707943 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9641550f0 con 0x7fb96406b6a0 2024-04-24T23:39:06.510 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.508+0000 7fb968add640 1 --2- 172.21.15.157:0/510707943 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9640732a0 0x7fb964143ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:39:06.510 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.508+0000 7fb969adf640 1 --2- 172.21.15.157:0/510707943 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9640c13c0 0x7fb9641ebd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:39:06.510 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.508+0000 7fb9692de640 1 --2- 172.21.15.157:0/510707943 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb96406b6a0 0x7fb9641ec8b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb96000ebe0 tx=0x7fb96000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:06.510 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.508+0000 7fb959ffb640 1 -- 172.21.15.157:0/510707943 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb960017070 con 0x7fb96406b6a0 2024-04-24T23:39:06.510 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.508+0000 7fb959ffb640 1 -- 172.21.15.157:0/510707943 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb96000cc30 con 0x7fb96406b6a0 2024-04-24T23:39:06.510 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.508+0000 7fb959ffb640 1 -- 172.21.15.157:0/510707943 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb960013470 con 0x7fb96406b6a0 2024-04-24T23:39:06.510 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.508+0000 7fb96b542640 1 -- 172.21.15.157:0/510707943 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9641eb550 con 0x7fb96406b6a0 2024-04-24T23:39:06.510 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.508+0000 7fb96b542640 1 -- 172.21.15.157:0/510707943 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9641dfc40 con 0x7fb96406b6a0 2024-04-24T23:39:06.510 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.509+0000 7fb96b542640 1 -- 172.21.15.157:0/510707943 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9640681c0 con 0x7fb96406b6a0 2024-04-24T23:39:06.511 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.510+0000 7fb959ffb640 1 -- 172.21.15.157:0/510707943 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb96001a070 con 0x7fb96406b6a0 2024-04-24T23:39:06.511 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.510+0000 7fb959ffb640 1 --2- 172.21.15.157:0/510707943 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fb94806be30 0x7fb94806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:06.511 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.510+0000 7fb959ffb640 1 -- 172.21.15.157:0/510707943 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fb96008b480 con 0x7fb96406b6a0 2024-04-24T23:39:06.511 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.510+0000 7fb968add640 1 --2- 172.21.15.157:0/510707943 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fb94806be30 0x7fb94806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:06.511 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.511+0000 7fb968add640 1 --2- 172.21.15.157:0/510707943 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fb94806be30 0x7fb94806e2f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb964144950 tx=0x7fb954009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:06.514 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.515+0000 7fb959ffb640 1 -- 172.21.15.157:0/510707943 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb9600593f0 con 0x7fb96406b6a0 2024-04-24T23:39:06.699 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.699+0000 7fb96b542640 1 -- 172.21.15.157:0/510707943 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb964152de0 con 0x7fb96406b6a0 2024-04-24T23:39:06.699 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.700+0000 7fb959ffb640 1 -- 172.21.15.157:0/510707943 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7fb96005d6f0 con 0x7fb96406b6a0 2024-04-24T23:39:06.699 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 64 2024-04-24T23:39:06.701 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.702+0000 7fb96b542640 1 -- 172.21.15.157:0/510707943 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fb94806be30 msgr2=0x7fb94806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:06.702 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.702+0000 7fb96b542640 1 --2- 172.21.15.157:0/510707943 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fb94806be30 0x7fb94806e2f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb964144950 tx=0x7fb954009040 comp rx=0 tx=0).stop 2024-04-24T23:39:06.702 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.702+0000 7fb96b542640 1 -- 172.21.15.157:0/510707943 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb96406b6a0 msgr2=0x7fb9641ec8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:06.702 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.702+0000 7fb96b542640 1 --2- 172.21.15.157:0/510707943 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb96406b6a0 0x7fb9641ec8b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb96000ebe0 tx=0x7fb96000c520 comp rx=0 tx=0).stop 2024-04-24T23:39:06.702 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.702+0000 7fb96b542640 1 -- 172.21.15.157:0/510707943 shutdown_connections 2024-04-24T23:39:06.702 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.702+0000 7fb96b542640 1 --2- 172.21.15.157:0/510707943 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fb94806be30 0x7fb94806e2f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:06.702 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.702+0000 7fb96b542640 1 --2- 172.21.15.157:0/510707943 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9640c13c0 0x7fb9641ebd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:06.702 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.702+0000 7fb96b542640 1 --2- 172.21.15.157:0/510707943 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9640732a0 0x7fb964143ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:06.702 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.702+0000 7fb96b542640 1 --2- 172.21.15.157:0/510707943 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb96406b6a0 0x7fb9641ec8b0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:06.702 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.702+0000 7fb96b542640 1 -- 172.21.15.157:0/510707943 >> 172.21.15.157:0/510707943 conn(0x7fb9640c3d60 msgr2=0x7fb9640c5d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:39:06.702 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.702+0000 7fb96b542640 1 -- 172.21.15.157:0/510707943 shutdown_connections 2024-04-24T23:39:06.702 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.702+0000 7fb96b542640 1 -- 172.21.15.157:0/510707943 wait complete. 2024-04-24T23:39:06.712 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T23:39:06.820 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.821+0000 7f0332e2a640 1 Processor -- start 2024-04-24T23:39:06.821 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.821+0000 7f0332e2a640 1 -- start start 2024-04-24T23:39:06.822 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.822+0000 7f0332e2a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f032c0c7f10 0x7f032c0a8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:06.822 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.822+0000 7f0332e2a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f032c0c3d00 con 0x7f032c0c6c40 2024-04-24T23:39:06.822 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.822+0000 7f0332e2a640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f032c1557c0 con 0x7f032c0a7c70 2024-04-24T23:39:06.822 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.822+0000 7f0332e2a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f032c155aa0 con 0x7f032c0c7f10 2024-04-24T23:39:06.822 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.822+0000 7f03313c7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f032c0c7f10 0x7f032c0a8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:06.822 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.822+0000 7f032bfff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f032c0c6c40 0x7f032c0c7800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49316/0 (socket says 172.21.15.157:49316) 2024-04-24T23:39:06.822 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.822+0000 7f03313c7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f032c0c7f10 0x7f032c0a8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51068/0 (socket says 172.21.15.157:51068) 2024-04-24T23:39:06.822 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.822+0000 7f032bfff640 1 -- 172.21.15.157:0/3053418331 learned_addr learned my addr 172.21.15.157:0/3053418331 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-24T23:39:06.822 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.822+0000 7f032b7fe640 1 -- 172.21.15.157:0/3053418331 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1295933396 0 0) 0x7f032c1557c0 con 0x7f032c0a7c70 2024-04-24T23:39:06.822 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.822+0000 7f03313c7640 1 -- 172.21.15.157:0/3053418331 >> v1:172.21.15.169:6789/0 conn(0x7f032c0a7c70 legacy=0x7f032c0c3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:06.823 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.822+0000 7f03313c7640 1 -- 172.21.15.157:0/3053418331 >> v1:172.21.15.157:6789/0 conn(0x7f032c0c6c40 legacy=0x7f032c0c7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:06.823 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.822+0000 7f03313c7640 1 -- 172.21.15.157:0/3053418331 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f032c155d80 con 0x7f032c0c7f10 2024-04-24T23:39:06.824 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.823+0000 7f03313c7640 1 --2- 172.21.15.157:0/3053418331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f032c0c7f10 0x7f032c0a8050 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0324009d90 tx=0x7f032402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb0ff3725c335761 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:06.824 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.823+0000 7f032b7fe640 1 -- 172.21.15.157:0/3053418331 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0324007740 con 0x7f032c0c7f10 2024-04-24T23:39:06.824 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.823+0000 7f032b7fe640 1 -- 172.21.15.157:0/3053418331 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03240078e0 con 0x7f032c0c7f10 2024-04-24T23:39:06.824 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.823+0000 7f032b7fe640 1 -- 172.21.15.157:0/3053418331 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0324007740 con 0x7f032c0c7f10 2024-04-24T23:39:06.824 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.824+0000 7f0332e2a640 1 -- 172.21.15.157:0/3053418331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f032c0c7f10 msgr2=0x7f032c0a8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:06.824 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.824+0000 7f0332e2a640 1 --2- 172.21.15.157:0/3053418331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f032c0c7f10 0x7f032c0a8050 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0324009d90 tx=0x7f032402a660 comp rx=0 tx=0).stop 2024-04-24T23:39:06.824 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.824+0000 7f0332e2a640 1 -- 172.21.15.157:0/3053418331 shutdown_connections 2024-04-24T23:39:06.824 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.824+0000 7f0332e2a640 1 --2- 172.21.15.157:0/3053418331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f032c0c7f10 0x7f032c0a8050 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:06.824 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.824+0000 7f0332e2a640 1 -- 172.21.15.157:0/3053418331 >> 172.21.15.157:0/3053418331 conn(0x7f032c0aa7b0 msgr2=0x7f032c0aab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:39:06.824 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.824+0000 7f0332e2a640 1 -- 172.21.15.157:0/3053418331 shutdown_connections 2024-04-24T23:39:06.825 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.825+0000 7f0332e2a640 1 -- 172.21.15.157:0/3053418331 wait complete. 2024-04-24T23:39:06.826 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.826+0000 7f0332e2a640 1 Processor -- start 2024-04-24T23:39:06.826 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.827+0000 7f0332e2a640 1 -- start start 2024-04-24T23:39:06.828 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.828+0000 7f0332e2a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f032c0a7c70 0x7f032c1ebae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:06.828 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.828+0000 7f0332e2a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f032c0c6c40 0x7f032c0a9a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:06.828 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.828+0000 7f0332e2a640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f032c0c7f10 0x7f032c0abe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:06.829 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.828+0000 7f0332e2a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f032c156420 con 0x7f032c0a7c70 2024-04-24T23:39:06.829 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.828+0000 7f0332e2a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f032c1562a0 con 0x7f032c0c6c40 2024-04-24T23:39:06.829 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.828+0000 7f0332e2a640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f032c160d20 con 0x7f032c0c7f10 2024-04-24T23:39:06.829 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.828+0000 7f0330bc6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f032c0a7c70 0x7f032c1ebae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:06.829 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.828+0000 7f0330bc6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f032c0a7c70 0x7f032c1ebae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51072/0 (socket says 172.21.15.157:51072) 2024-04-24T23:39:06.829 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.828+0000 7f0330bc6640 1 -- 172.21.15.157:0/3274181665 learned_addr learned my addr 172.21.15.157:0/3274181665 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:39:06.829 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.828+0000 7f032bfff640 1 --2- 172.21.15.157:0/3274181665 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f032c0c6c40 0x7f032c0a9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:06.829 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.828+0000 7f03313c7640 1 --2- 172.21.15.157:0/3274181665 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f032c0c7f10 0x7f032c0abe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:06.829 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.828+0000 7f0330bc6640 1 -- 172.21.15.157:0/3274181665 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f032c0c7f10 msgr2=0x7f032c0abe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:06.829 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.828+0000 7f0330bc6640 1 --2- 172.21.15.157:0/3274181665 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f032c0c7f10 0x7f032c0abe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:06.829 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.828+0000 7f0330bc6640 1 -- 172.21.15.157:0/3274181665 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f032c0c6c40 msgr2=0x7f032c0a9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:06.829 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.828+0000 7f0330bc6640 1 --2- 172.21.15.157:0/3274181665 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f032c0c6c40 0x7f032c0a9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:06.829 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.828+0000 7f0330bc6640 1 -- 172.21.15.157:0/3274181665 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f032c06c1c0 con 0x7f032c0a7c70 2024-04-24T23:39:06.829 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.828+0000 7f032bfff640 1 --2- 172.21.15.157:0/3274181665 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f032c0c6c40 0x7f032c0a9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:39:06.830 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.828+0000 7f0330bc6640 1 --2- 172.21.15.157:0/3274181665 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f032c0a7c70 0x7f032c1ebae0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f032000c3c0 tx=0x7f032000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:06.830 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.829+0000 7f03297fa640 1 -- 172.21.15.157:0/3274181665 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0320017070 con 0x7f032c0a7c70 2024-04-24T23:39:06.830 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.829+0000 7f03297fa640 1 -- 172.21.15.157:0/3274181665 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03200035f0 con 0x7f032c0a7c70 2024-04-24T23:39:06.830 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.829+0000 7f03297fa640 1 -- 172.21.15.157:0/3274181665 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03200039e0 con 0x7f032c0a7c70 2024-04-24T23:39:06.830 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.829+0000 7f0332e2a640 1 -- 172.21.15.157:0/3274181665 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f032c0accb0 con 0x7f032c0a7c70 2024-04-24T23:39:06.830 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.829+0000 7f0332e2a640 1 -- 172.21.15.157:0/3274181665 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f032c0ad270 con 0x7f032c0a7c70 2024-04-24T23:39:06.831 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.830+0000 7f0332e2a640 1 -- 172.21.15.157:0/3274181665 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f032c0681c0 con 0x7f032c0a7c70 2024-04-24T23:39:06.831 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.831+0000 7f03297fa640 1 -- 172.21.15.157:0/3274181665 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f032001a070 con 0x7f032c0a7c70 2024-04-24T23:39:06.831 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.831+0000 7f03297fa640 1 --2- 172.21.15.157:0/3274181665 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f031006be80 0x7f031006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:06.831 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.831+0000 7f03297fa640 1 -- 172.21.15.157:0/3274181665 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f032008a400 con 0x7f032c0a7c70 2024-04-24T23:39:06.831 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.831+0000 7f032bfff640 1 --2- 172.21.15.157:0/3274181665 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f031006be80 0x7f031006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:06.832 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.832+0000 7f032bfff640 1 --2- 172.21.15.157:0/3274181665 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f031006be80 0x7f031006e340 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f032c0a8210 tx=0x7f0318007e80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:06.836 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:06.837+0000 7f03297fa640 1 -- 172.21.15.157:0/3274181665 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f03200584d0 con 0x7f032c0a7c70 2024-04-24T23:39:07.077 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.078+0000 7f0332e2a640 1 -- 172.21.15.157:0/3274181665 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f032c0c97d0 con 0x7f032c0a7c70 2024-04-24T23:39:07.077 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.078+0000 7f03297fa640 1 -- 172.21.15.157:0/3274181665 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f032005c7d0 con 0x7f032c0a7c70 2024-04-24T23:39:07.078 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2024-04-24T23:39:07.080 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.080+0000 7f0332e2a640 1 -- 172.21.15.157:0/3274181665 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f031006be80 msgr2=0x7f031006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:07.080 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.080+0000 7f0332e2a640 1 --2- 172.21.15.157:0/3274181665 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f031006be80 0x7f031006e340 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f032c0a8210 tx=0x7f0318007e80 comp rx=0 tx=0).stop 2024-04-24T23:39:07.080 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.080+0000 7f0332e2a640 1 -- 172.21.15.157:0/3274181665 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f032c0a7c70 msgr2=0x7f032c1ebae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:07.080 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.080+0000 7f0332e2a640 1 --2- 172.21.15.157:0/3274181665 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f032c0a7c70 0x7f032c1ebae0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f032000c3c0 tx=0x7f032000c880 comp rx=0 tx=0).stop 2024-04-24T23:39:07.080 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.080+0000 7f0332e2a640 1 -- 172.21.15.157:0/3274181665 shutdown_connections 2024-04-24T23:39:07.080 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.080+0000 7f0332e2a640 1 --2- 172.21.15.157:0/3274181665 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f031006be80 0x7f031006e340 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:07.081 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.080+0000 7f0332e2a640 1 --2- 172.21.15.157:0/3274181665 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f032c0c7f10 0x7f032c0abe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:07.081 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.080+0000 7f0332e2a640 1 --2- 172.21.15.157:0/3274181665 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f032c0c6c40 0x7f032c0a9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:07.081 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.080+0000 7f0332e2a640 1 --2- 172.21.15.157:0/3274181665 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f032c0a7c70 0x7f032c1ebae0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:07.081 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.080+0000 7f0332e2a640 1 -- 172.21.15.157:0/3274181665 >> 172.21.15.157:0/3274181665 conn(0x7f032c0aa7b0 msgr2=0x7f032c153080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:39:07.081 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.080+0000 7f0332e2a640 1 -- 172.21.15.157:0/3274181665 shutdown_connections 2024-04-24T23:39:07.081 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.080+0000 7f0332e2a640 1 -- 172.21.15.157:0/3274181665 wait complete. 2024-04-24T23:39:07.091 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T23:39:07.133 INFO:tasks.workunit.client.1.smithi169.stdout: 1 395361 15.30 MB/sec execute 25 sec latency 24.761 ms 2024-04-24T23:39:07.196 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.197+0000 7f5c37a09640 1 Processor -- start 2024-04-24T23:39:07.197 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.197+0000 7f5c37a09640 1 -- start start 2024-04-24T23:39:07.198 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.198+0000 7f5c37a09640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5c3014efb0 0x7f5c30152370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:07.198 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.198+0000 7f5c37a09640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5c30152da0 0x7f5c30153250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:07.198 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.198+0000 7f5c37a09640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c30154e60 con 0x7f5c300c2510 2024-04-24T23:39:07.199 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.198+0000 7f5c37a09640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f5c3014f390 con 0x7f5c3014efb0 2024-04-24T23:39:07.199 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.198+0000 7f5c37a09640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5c301550f0 con 0x7f5c30152da0 2024-04-24T23:39:07.199 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.198+0000 7f5c34fa4640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f5c300c2510 0x7f5c300c5070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49326/0 (socket says 172.21.15.157:49326) 2024-04-24T23:39:07.199 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.198+0000 7f5c34fa4640 1 -- 172.21.15.157:0/385896476 learned_addr learned my addr 172.21.15.157:0/385896476 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-24T23:39:07.199 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.198+0000 7f5c35fa6640 1 --2- 172.21.15.157:0/385896476 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5c30152da0 0x7f5c30153250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:07.199 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.198+0000 7f5c357a5640 1 --2- 172.21.15.157:0/385896476 >> v2:172.21.15.169:3300/0 conn(0x7f5c3014efb0 0x7f5c30152370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:07.199 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.198+0000 7f5c27fff640 1 -- 172.21.15.157:0/385896476 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4118603436 0 0) 0x7f5c30154e60 con 0x7f5c300c2510 2024-04-24T23:39:07.199 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.198+0000 7f5c27fff640 1 -- 172.21.15.157:0/385896476 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5c1c003520 con 0x7f5c300c2510 2024-04-24T23:39:07.199 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.199+0000 7f5c35fa6640 1 -- 172.21.15.157:0/385896476 >> v2:172.21.15.169:3300/0 conn(0x7f5c3014efb0 msgr2=0x7f5c30152370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:39:07.199 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.199+0000 7f5c35fa6640 1 --2- 172.21.15.157:0/385896476 >> v2:172.21.15.169:3300/0 conn(0x7f5c3014efb0 0x7f5c30152370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:07.199 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.199+0000 7f5c35fa6640 1 -- 172.21.15.157:0/385896476 >> v1:172.21.15.157:6789/0 conn(0x7f5c300c2510 legacy=0x7f5c300c5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:07.199 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.199+0000 7f5c35fa6640 1 -- 172.21.15.157:0/385896476 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c301553d0 con 0x7f5c30152da0 2024-04-24T23:39:07.199 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.199+0000 7f5c357a5640 1 --2- 172.21.15.157:0/385896476 >> v2:172.21.15.169:3300/0 conn(0x7f5c3014efb0 0x7f5c30152370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:39:07.200 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.199+0000 7f5c35fa6640 1 --2- 172.21.15.157:0/385896476 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5c30152da0 0x7f5c30153250 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5c28009d90 tx=0x7f5c2802a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1668a675076e5575 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:07.203 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.199+0000 7f5c27fff640 1 -- 172.21.15.157:0/385896476 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c28031600 con 0x7f5c30152da0 2024-04-24T23:39:07.203 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.199+0000 7f5c27fff640 1 -- 172.21.15.157:0/385896476 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c280317a0 con 0x7f5c30152da0 2024-04-24T23:39:07.203 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.200+0000 7f5c27fff640 1 -- 172.21.15.157:0/385896476 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c28031b70 con 0x7f5c30152da0 2024-04-24T23:39:07.203 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.200+0000 7f5c37a09640 1 -- 172.21.15.157:0/385896476 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5c30152da0 msgr2=0x7f5c30153250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:07.203 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.200+0000 7f5c37a09640 1 --2- 172.21.15.157:0/385896476 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5c30152da0 0x7f5c30153250 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5c28009d90 tx=0x7f5c2802a5d0 comp rx=0 tx=0).stop 2024-04-24T23:39:07.203 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.200+0000 7f5c37a09640 1 -- 172.21.15.157:0/385896476 shutdown_connections 2024-04-24T23:39:07.203 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.200+0000 7f5c37a09640 1 --2- 172.21.15.157:0/385896476 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5c30152da0 0x7f5c30153250 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:07.203 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.200+0000 7f5c37a09640 1 --2- 172.21.15.157:0/385896476 >> v2:172.21.15.169:3300/0 conn(0x7f5c3014efb0 0x7f5c30152370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:07.203 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.200+0000 7f5c37a09640 1 -- 172.21.15.157:0/385896476 >> 172.21.15.157:0/385896476 conn(0x7f5c300a8690 msgr2=0x7f5c300a8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:39:07.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.200+0000 7f5c37a09640 1 -- 172.21.15.157:0/385896476 shutdown_connections 2024-04-24T23:39:07.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.200+0000 7f5c37a09640 1 -- 172.21.15.157:0/385896476 wait complete. 2024-04-24T23:39:07.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.201+0000 7f5c37a09640 1 Processor -- start 2024-04-24T23:39:07.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.201+0000 7f5c37a09640 1 -- start start 2024-04-24T23:39:07.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.202+0000 7f5c37a09640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c300c2510 0x7f5c300abdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:07.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.202+0000 7f5c37a09640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5c3014efb0 0x7f5c300ac310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:07.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.202+0000 7f5c37a09640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c30152da0 0x7f5c301e57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:07.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.202+0000 7f5c37a09640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5c300c3030 con 0x7f5c30152da0 2024-04-24T23:39:07.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.202+0000 7f5c37a09640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5c30155990 con 0x7f5c300c2510 2024-04-24T23:39:07.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.202+0000 7f5c37a09640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5c300c5780 con 0x7f5c3014efb0 2024-04-24T23:39:07.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.202+0000 7f5c35fa6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c30152da0 0x7f5c301e57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:07.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.202+0000 7f5c35fa6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c30152da0 0x7f5c301e57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51078/0 (socket says 172.21.15.157:51078) 2024-04-24T23:39:07.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.202+0000 7f5c35fa6640 1 -- 172.21.15.157:0/3228598640 learned_addr learned my addr 172.21.15.157:0/3228598640 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-24T23:39:07.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.202+0000 7f5c357a5640 1 --2- 172.21.15.157:0/3228598640 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c300c2510 0x7f5c300abdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:07.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.202+0000 7f5c34fa4640 1 --2- 172.21.15.157:0/3228598640 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5c3014efb0 0x7f5c300ac310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:07.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.202+0000 7f5c35fa6640 1 -- 172.21.15.157:0/3228598640 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5c3014efb0 msgr2=0x7f5c300ac310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:07.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.202+0000 7f5c35fa6640 1 --2- 172.21.15.157:0/3228598640 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5c3014efb0 0x7f5c300ac310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:07.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.202+0000 7f5c35fa6640 1 -- 172.21.15.157:0/3228598640 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c300c2510 msgr2=0x7f5c300abdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:07.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.202+0000 7f5c35fa6640 1 --2- 172.21.15.157:0/3228598640 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c300c2510 0x7f5c300abdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:07.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.202+0000 7f5c35fa6640 1 -- 172.21.15.157:0/3228598640 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c300711a0 con 0x7f5c30152da0 2024-04-24T23:39:07.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.202+0000 7f5c357a5640 1 --2- 172.21.15.157:0/3228598640 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c300c2510 0x7f5c300abdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:39:07.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.202+0000 7f5c34fa4640 1 --2- 172.21.15.157:0/3228598640 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5c3014efb0 0x7f5c300ac310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:39:07.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.203+0000 7f5c35fa6640 1 --2- 172.21.15.157:0/3228598640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c30152da0 0x7f5c301e57f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5c2802abc0 tx=0x7f5c28004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:07.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.203+0000 7f5c25ffb640 1 -- 172.21.15.157:0/3228598640 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c28040070 con 0x7f5c30152da0 2024-04-24T23:39:07.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.203+0000 7f5c25ffb640 1 -- 172.21.15.157:0/3228598640 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c2803c830 con 0x7f5c30152da0 2024-04-24T23:39:07.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.203+0000 7f5c25ffb640 1 -- 172.21.15.157:0/3228598640 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2803cbe0 con 0x7f5c30152da0 2024-04-24T23:39:07.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.203+0000 7f5c37a09640 1 -- 172.21.15.157:0/3228598640 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c300aa2c0 con 0x7f5c30152da0 2024-04-24T23:39:07.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.203+0000 7f5c37a09640 1 -- 172.21.15.157:0/3228598640 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c301e94b0 con 0x7f5c30152da0 2024-04-24T23:39:07.206 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.204+0000 7f5c37a09640 1 -- 172.21.15.157:0/3228598640 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c300681c0 con 0x7f5c30152da0 2024-04-24T23:39:07.207 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.204+0000 7f5c25ffb640 1 -- 172.21.15.157:0/3228598640 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5c28039040 con 0x7f5c30152da0 2024-04-24T23:39:07.207 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.205+0000 7f5c25ffb640 1 --2- 172.21.15.157:0/3228598640 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f5c1406be30 0x7f5c1406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:07.207 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.205+0000 7f5c25ffb640 1 -- 172.21.15.157:0/3228598640 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f5c280ac660 con 0x7f5c30152da0 2024-04-24T23:39:07.207 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.205+0000 7f5c357a5640 1 --2- 172.21.15.157:0/3228598640 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f5c1406be30 0x7f5c1406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:07.207 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.205+0000 7f5c357a5640 1 --2- 172.21.15.157:0/3228598640 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f5c1406be30 0x7f5c1406e2f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c009560 tx=0x7f5c2c009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:07.208 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.209+0000 7f5c25ffb640 1 -- 172.21.15.157:0/3228598640 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5c2807a770 con 0x7f5c30152da0 2024-04-24T23:39:07.408 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.407+0000 7f5c37a09640 1 -- 172.21.15.157:0/3228598640 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5c301e8fc0 con 0x7f5c30152da0 2024-04-24T23:39:07.408 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.408+0000 7f5c25ffb640 1 -- 172.21.15.157:0/3228598640 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f5c2807ea70 con 0x7f5c30152da0 2024-04-24T23:39:07.408 INFO:teuthology.orchestra.run.smithi157.stderr:listed 0 entries 2024-04-24T23:39:07.410 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.410+0000 7f5c37a09640 1 -- 172.21.15.157:0/3228598640 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f5c1406be30 msgr2=0x7f5c1406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:07.410 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.410+0000 7f5c37a09640 1 --2- 172.21.15.157:0/3228598640 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f5c1406be30 0x7f5c1406e2f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c009560 tx=0x7f5c2c009190 comp rx=0 tx=0).stop 2024-04-24T23:39:07.410 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.410+0000 7f5c37a09640 1 -- 172.21.15.157:0/3228598640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c30152da0 msgr2=0x7f5c301e57f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:07.410 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.410+0000 7f5c37a09640 1 --2- 172.21.15.157:0/3228598640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c30152da0 0x7f5c301e57f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5c2802abc0 tx=0x7f5c28004a40 comp rx=0 tx=0).stop 2024-04-24T23:39:07.410 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.410+0000 7f5c37a09640 1 -- 172.21.15.157:0/3228598640 shutdown_connections 2024-04-24T23:39:07.410 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.410+0000 7f5c37a09640 1 --2- 172.21.15.157:0/3228598640 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f5c1406be30 0x7f5c1406e2f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:07.410 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.410+0000 7f5c37a09640 1 --2- 172.21.15.157:0/3228598640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c30152da0 0x7f5c301e57f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:07.410 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.410+0000 7f5c37a09640 1 --2- 172.21.15.157:0/3228598640 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5c3014efb0 0x7f5c300ac310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:07.410 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.410+0000 7f5c37a09640 1 --2- 172.21.15.157:0/3228598640 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c300c2510 0x7f5c300abdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:07.410 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.410+0000 7f5c37a09640 1 -- 172.21.15.157:0/3228598640 >> 172.21.15.157:0/3228598640 conn(0x7f5c300a8690 msgr2=0x7f5c300a7f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:39:07.411 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.410+0000 7f5c37a09640 1 -- 172.21.15.157:0/3228598640 shutdown_connections 2024-04-24T23:39:07.411 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-24T23:39:07.410+0000 7f5c37a09640 1 -- 172.21.15.157:0/3228598640 wait complete. 2024-04-24T23:39:07.421 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi157.front.sepia.ceph.com... 2024-04-24T23:39:07.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:39:07.422 DEBUG:teuthology.orchestra.run.smithi157:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-24T23:39:07.447 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:39:07.448 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi157.stderr:ceph-fuse[42972]: fuse finished with error 0 and tester_r 0 2024-04-24T23:39:08.133 INFO:tasks.workunit.client.1.smithi169.stdout: 1 398435 15.32 MB/sec execute 26 sec latency 33.699 ms 2024-04-24T23:39:09.133 INFO:tasks.workunit.client.1.smithi169.stdout: 1 401742 15.42 MB/sec execute 27 sec latency 14.266 ms 2024-04-24T23:39:10.134 INFO:tasks.workunit.client.1.smithi169.stdout: 1 405142 15.54 MB/sec execute 28 sec latency 29.959 ms 2024-04-24T23:39:11.134 INFO:tasks.workunit.client.1.smithi169.stdout: 1 408499 15.65 MB/sec execute 29 sec latency 68.959 ms 2024-04-24T23:39:12.134 INFO:tasks.workunit.client.1.smithi169.stdout: 1 411901 15.76 MB/sec execute 30 sec latency 16.201 ms 2024-04-24T23:39:13.134 INFO:tasks.workunit.client.1.smithi169.stdout: 1 415341 15.86 MB/sec execute 31 sec latency 19.666 ms 2024-04-24T23:39:13.548 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-04-24T23:39:13.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:39:13.549 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-24T23:39:13.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:39:13.565 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-04-24T23:39:13.621 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-24T23:39:13.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:39:13.622 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T23:39:13.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:39:13.687 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link delete ceph-brx 2024-04-24T23:39:13.776 DEBUG:teuthology.parallel:result is None 2024-04-24T23:39:14.134 INFO:tasks.workunit.client.1.smithi169.stdout: 1 418812 15.96 MB/sec execute 32 sec latency 15.712 ms 2024-04-24T23:39:15.134 INFO:tasks.workunit.client.1.smithi169.stdout: 1 421958 15.93 MB/sec execute 33 sec latency 22.582 ms 2024-04-24T23:39:16.134 INFO:tasks.workunit.client.1.smithi169.stdout: 1 425300 15.98 MB/sec execute 34 sec latency 30.888 ms 2024-04-24T23:39:17.134 INFO:tasks.workunit.client.1.smithi169.stdout: 1 428594 16.05 MB/sec execute 35 sec latency 16.742 ms 2024-04-24T23:39:18.134 INFO:tasks.workunit.client.1.smithi169.stdout: 1 431712 16.01 MB/sec execute 36 sec latency 27.602 ms 2024-04-24T23:39:19.134 INFO:tasks.workunit.client.1.smithi169.stdout: 1 433755 15.91 MB/sec execute 37 sec latency 114.957 ms 2024-04-24T23:39:20.134 INFO:tasks.workunit.client.1.smithi169.stdout: 1 436601 15.96 MB/sec execute 38 sec latency 290.568 ms 2024-04-24T23:39:21.134 INFO:tasks.workunit.client.1.smithi169.stdout: 1 439318 15.88 MB/sec execute 39 sec latency 24.204 ms 2024-04-24T23:39:22.134 INFO:tasks.workunit.client.1.smithi169.stdout: 1 442754 15.94 MB/sec execute 40 sec latency 30.387 ms 2024-04-24T23:39:23.134 INFO:tasks.workunit.client.1.smithi169.stdout: 1 445859 15.90 MB/sec execute 41 sec latency 23.294 ms 2024-04-24T23:39:24.134 INFO:tasks.workunit.client.1.smithi169.stdout: 1 449376 15.98 MB/sec execute 42 sec latency 28.453 ms 2024-04-24T23:39:25.134 INFO:tasks.workunit.client.1.smithi169.stdout: 1 452890 16.05 MB/sec execute 43 sec latency 32.039 ms 2024-04-24T23:39:26.134 INFO:tasks.workunit.client.1.smithi169.stdout: 1 455556 16.08 MB/sec execute 44 sec latency 87.781 ms 2024-04-24T23:39:27.134 INFO:tasks.workunit.client.1.smithi169.stdout: 1 458533 16.21 MB/sec execute 45 sec latency 56.540 ms 2024-04-24T23:39:28.134 INFO:tasks.workunit.client.1.smithi169.stdout: 1 462380 16.56 MB/sec execute 46 sec latency 7.369 ms 2024-04-24T23:39:29.134 INFO:tasks.workunit.client.1.smithi169.stdout: 1 465785 16.61 MB/sec execute 47 sec latency 11.790 ms 2024-04-24T23:39:30.134 INFO:tasks.workunit.client.1.smithi169.stdout: 1 467114 16.38 MB/sec execute 48 sec latency 148.032 ms 2024-04-24T23:39:30.168 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:39:30.172 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:39:31.134 INFO:tasks.workunit.client.1.smithi169.stdout: 1 469659 16.33 MB/sec execute 49 sec latency 32.098 ms 2024-04-24T23:39:32.134 INFO:tasks.workunit.client.1.smithi169.stdout: 1 472998 16.38 MB/sec execute 50 sec latency 21.756 ms 2024-04-24T23:39:33.134 INFO:tasks.workunit.client.1.smithi169.stdout: 1 476394 16.43 MB/sec execute 51 sec latency 22.548 ms 2024-04-24T23:39:34.134 INFO:tasks.workunit.client.1.smithi169.stdout: 1 479634 16.46 MB/sec execute 52 sec latency 17.540 ms 2024-04-24T23:39:35.134 INFO:tasks.workunit.client.1.smithi169.stdout: 1 482667 16.41 MB/sec execute 53 sec latency 11.420 ms 2024-04-24T23:39:36.134 INFO:tasks.workunit.client.1.smithi169.stdout: 1 485403 16.34 MB/sec execute 54 sec latency 21.225 ms 2024-04-24T23:39:37.134 INFO:tasks.workunit.client.1.smithi169.stdout: 1 488214 16.38 MB/sec execute 55 sec latency 22.015 ms 2024-04-24T23:39:38.134 INFO:tasks.workunit.client.1.smithi169.stdout: 1 491116 16.34 MB/sec execute 56 sec latency 20.535 ms 2024-04-24T23:39:39.134 INFO:tasks.workunit.client.1.smithi169.stdout: 1 494589 16.39 MB/sec execute 57 sec latency 15.708 ms 2024-04-24T23:39:40.134 INFO:tasks.workunit.client.1.smithi169.stdout: 1 498019 16.43 MB/sec execute 58 sec latency 18.998 ms 2024-04-24T23:39:41.134 INFO:tasks.workunit.client.1.smithi169.stdout: 1 501538 16.48 MB/sec execute 59 sec latency 26.063 ms 2024-04-24T23:39:42.134 INFO:tasks.workunit.client.1.smithi169.stdout: 1 504914 16.52 MB/sec execute 60 sec latency 24.468 ms 2024-04-24T23:39:43.134 INFO:tasks.workunit.client.1.smithi169.stdout: 1 508331 16.56 MB/sec execute 61 sec latency 18.927 ms 2024-04-24T23:39:44.135 INFO:tasks.workunit.client.1.smithi169.stdout: 1 511830 16.60 MB/sec execute 62 sec latency 18.852 ms 2024-04-24T23:39:45.134 INFO:tasks.workunit.client.1.smithi169.stdout: 1 515353 16.64 MB/sec execute 63 sec latency 22.264 ms 2024-04-24T23:39:46.135 INFO:tasks.workunit.client.1.smithi169.stdout: 1 518289 16.59 MB/sec execute 64 sec latency 126.788 ms 2024-04-24T23:39:47.134 INFO:tasks.workunit.client.1.smithi169.stdout: 1 521648 16.62 MB/sec execute 65 sec latency 31.497 ms 2024-04-24T23:39:48.135 INFO:tasks.workunit.client.1.smithi169.stdout: 1 525009 16.63 MB/sec execute 66 sec latency 25.270 ms 2024-04-24T23:39:49.135 INFO:tasks.workunit.client.1.smithi169.stdout: 1 528132 16.61 MB/sec execute 67 sec latency 31.405 ms 2024-04-24T23:39:50.135 INFO:tasks.workunit.client.1.smithi169.stdout: 1 531440 16.63 MB/sec execute 68 sec latency 18.004 ms 2024-04-24T23:39:51.135 INFO:tasks.workunit.client.1.smithi169.stdout: 1 533991 16.61 MB/sec execute 69 sec latency 26.034 ms 2024-04-24T23:39:52.135 INFO:tasks.workunit.client.1.smithi169.stdout: 1 537221 16.63 MB/sec execute 70 sec latency 18.864 ms 2024-04-24T23:39:53.135 INFO:tasks.workunit.client.1.smithi169.stdout: 1 540696 16.66 MB/sec execute 71 sec latency 13.927 ms 2024-04-24T23:39:54.135 INFO:tasks.workunit.client.1.smithi169.stdout: 1 544140 16.69 MB/sec execute 72 sec latency 18.645 ms 2024-04-24T23:39:55.135 INFO:tasks.workunit.client.1.smithi169.stdout: 1 547454 16.72 MB/sec execute 73 sec latency 25.169 ms 2024-04-24T23:39:56.136 INFO:tasks.workunit.client.1.smithi169.stdout: 1 550701 16.74 MB/sec execute 74 sec latency 24.109 ms 2024-04-24T23:39:57.135 INFO:tasks.workunit.client.1.smithi169.stdout: 1 554123 16.76 MB/sec execute 75 sec latency 27.887 ms 2024-04-24T23:39:58.135 INFO:tasks.workunit.client.1.smithi169.stdout: 1 557134 16.73 MB/sec execute 76 sec latency 18.804 ms 2024-04-24T23:39:59.136 INFO:tasks.workunit.client.1.smithi169.stdout: 1 560597 16.75 MB/sec execute 77 sec latency 29.125 ms 2024-04-24T23:40:00.136 INFO:tasks.workunit.client.1.smithi169.stdout: 1 563745 16.73 MB/sec execute 78 sec latency 19.632 ms 2024-04-24T23:40:00.503 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:40:00.508 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:40:01.136 INFO:tasks.workunit.client.1.smithi169.stdout: 1 564779 16.64 MB/sec execute 79 sec latency 335.835 ms 2024-04-24T23:40:02.136 INFO:tasks.workunit.client.1.smithi169.stdout: 1 565973 16.48 MB/sec execute 80 sec latency 267.031 ms 2024-04-24T23:40:03.136 INFO:tasks.workunit.client.1.smithi169.stdout: 1 567249 16.34 MB/sec execute 81 sec latency 57.510 ms 2024-04-24T23:40:04.136 INFO:tasks.workunit.client.1.smithi169.stdout: 1 570690 16.37 MB/sec execute 82 sec latency 22.357 ms 2024-04-24T23:40:05.136 INFO:tasks.workunit.client.1.smithi169.stdout: 1 573867 16.39 MB/sec execute 83 sec latency 23.731 ms 2024-04-24T23:40:06.136 INFO:tasks.workunit.client.1.smithi169.stdout: 1 576993 16.41 MB/sec execute 84 sec latency 34.605 ms 2024-04-24T23:40:07.136 INFO:tasks.workunit.client.1.smithi169.stdout: 1 580261 16.41 MB/sec execute 85 sec latency 24.143 ms 2024-04-24T23:40:08.136 INFO:tasks.workunit.client.1.smithi169.stdout: 1 583595 16.44 MB/sec execute 86 sec latency 30.578 ms 2024-04-24T23:40:09.136 INFO:tasks.workunit.client.1.smithi169.stdout: 1 586984 16.46 MB/sec execute 87 sec latency 16.431 ms 2024-04-24T23:40:10.136 INFO:tasks.workunit.client.1.smithi169.stdout: 1 590401 16.48 MB/sec execute 88 sec latency 16.785 ms 2024-04-24T23:40:11.136 INFO:tasks.workunit.client.1.smithi169.stdout: 1 593643 16.51 MB/sec execute 89 sec latency 55.742 ms 2024-04-24T23:40:12.136 INFO:tasks.workunit.client.1.smithi169.stdout: 1 597036 16.53 MB/sec execute 90 sec latency 25.528 ms 2024-04-24T23:40:13.136 INFO:tasks.workunit.client.1.smithi169.stdout: 1 600382 16.55 MB/sec execute 91 sec latency 26.161 ms 2024-04-24T23:40:14.136 INFO:tasks.workunit.client.1.smithi169.stdout: 1 603267 16.52 MB/sec execute 92 sec latency 23.816 ms 2024-04-24T23:40:15.136 INFO:tasks.workunit.client.1.smithi169.stdout: 1 606189 16.49 MB/sec execute 93 sec latency 14.019 ms 2024-04-24T23:40:16.136 INFO:tasks.workunit.client.1.smithi169.stdout: 1 607609 16.44 MB/sec execute 94 sec latency 156.201 ms 2024-04-24T23:40:17.136 INFO:tasks.workunit.client.1.smithi169.stdout: 1 609545 16.33 MB/sec execute 95 sec latency 115.541 ms 2024-04-24T23:40:18.136 INFO:tasks.workunit.client.1.smithi169.stdout: 1 612562 16.36 MB/sec execute 96 sec latency 21.183 ms 2024-04-24T23:40:19.136 INFO:tasks.workunit.client.1.smithi169.stdout: 1 615592 16.35 MB/sec execute 97 sec latency 22.580 ms 2024-04-24T23:40:20.136 INFO:tasks.workunit.client.1.smithi169.stdout: 1 618672 16.36 MB/sec execute 98 sec latency 21.770 ms 2024-04-24T23:40:21.136 INFO:tasks.workunit.client.1.smithi169.stdout: 1 621839 16.38 MB/sec execute 99 sec latency 16.768 ms 2024-04-24T23:40:22.136 INFO:tasks.workunit.client.1.smithi169.stdout: 1 625061 16.40 MB/sec execute 100 sec latency 14.185 ms 2024-04-24T23:40:23.136 INFO:tasks.workunit.client.1.smithi169.stdout: 1 627672 16.33 MB/sec execute 101 sec latency 27.720 ms 2024-04-24T23:40:24.137 INFO:tasks.workunit.client.1.smithi169.stdout: 1 630226 16.33 MB/sec execute 102 sec latency 16.503 ms 2024-04-24T23:40:25.137 INFO:tasks.workunit.client.1.smithi169.stdout: 1 633090 16.32 MB/sec execute 103 sec latency 19.315 ms 2024-04-24T23:40:26.137 INFO:tasks.workunit.client.1.smithi169.stdout: 1 636500 16.34 MB/sec execute 104 sec latency 29.409 ms 2024-04-24T23:40:27.137 INFO:tasks.workunit.client.1.smithi169.stdout: 1 639824 16.36 MB/sec execute 105 sec latency 22.491 ms 2024-04-24T23:40:28.137 INFO:tasks.workunit.client.1.smithi169.stdout: 1 643127 16.38 MB/sec execute 106 sec latency 21.483 ms 2024-04-24T23:40:29.137 INFO:tasks.workunit.client.1.smithi169.stdout: 1 646589 16.41 MB/sec execute 107 sec latency 17.169 ms 2024-04-24T23:40:30.139 INFO:tasks.workunit.client.1.smithi169.stdout: 1 649890 16.42 MB/sec execute 108 sec latency 23.179 ms 2024-04-24T23:40:30.594 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:40:30.597 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:40:31.137 INFO:tasks.workunit.client.1.smithi169.stdout: 1 652474 16.36 MB/sec execute 109 sec latency 56.525 ms 2024-04-24T23:40:32.137 INFO:tasks.workunit.client.1.smithi169.stdout: 1 653909 16.31 MB/sec execute 110 sec latency 118.290 ms 2024-04-24T23:40:33.137 INFO:tasks.workunit.client.1.smithi169.stdout: 1 655869 16.23 MB/sec execute 111 sec latency 31.021 ms 2024-04-24T23:40:34.137 INFO:tasks.workunit.client.1.smithi169.stdout: 1 658791 16.24 MB/sec execute 112 sec latency 25.931 ms 2024-04-24T23:40:35.137 INFO:tasks.workunit.client.1.smithi169.stdout: 1 661999 16.26 MB/sec execute 113 sec latency 23.603 ms 2024-04-24T23:40:36.137 INFO:tasks.workunit.client.1.smithi169.stdout: 1 665076 16.25 MB/sec execute 114 sec latency 15.875 ms 2024-04-24T23:40:37.137 INFO:tasks.workunit.client.1.smithi169.stdout: 1 668367 16.27 MB/sec execute 115 sec latency 23.431 ms 2024-04-24T23:40:38.137 INFO:tasks.workunit.client.1.smithi169.stdout: 1 671629 16.29 MB/sec execute 116 sec latency 20.472 ms 2024-04-24T23:40:39.137 INFO:tasks.workunit.client.1.smithi169.stdout: 1 675108 16.32 MB/sec execute 117 sec latency 15.987 ms 2024-04-24T23:40:40.137 INFO:tasks.workunit.client.1.smithi169.stdout: 1 678436 16.33 MB/sec execute 118 sec latency 37.890 ms 2024-04-24T23:40:41.137 INFO:tasks.workunit.client.1.smithi169.stdout: 1 681482 16.31 MB/sec execute 119 sec latency 25.004 ms 2024-04-24T23:40:42.137 INFO:tasks.workunit.client.1.smithi169.stdout: 1 684624 16.32 MB/sec execute 120 sec latency 26.789 ms 2024-04-24T23:40:43.138 INFO:tasks.workunit.client.1.smithi169.stdout: 1 687541 16.30 MB/sec execute 121 sec latency 19.666 ms 2024-04-24T23:40:44.137 INFO:tasks.workunit.client.1.smithi169.stdout: 1 690090 16.29 MB/sec execute 122 sec latency 30.175 ms 2024-04-24T23:40:45.137 INFO:tasks.workunit.client.1.smithi169.stdout: 1 693173 16.30 MB/sec execute 123 sec latency 24.008 ms 2024-04-24T23:40:46.137 INFO:tasks.workunit.client.1.smithi169.stdout: 1 696281 16.32 MB/sec execute 124 sec latency 39.917 ms 2024-04-24T23:40:47.137 INFO:tasks.workunit.client.1.smithi169.stdout: 1 698968 16.29 MB/sec execute 125 sec latency 14.553 ms 2024-04-24T23:40:48.137 INFO:tasks.workunit.client.1.smithi169.stdout: 1 701911 16.27 MB/sec execute 126 sec latency 23.620 ms 2024-04-24T23:40:49.137 INFO:tasks.workunit.client.1.smithi169.stdout: 1 704956 16.28 MB/sec execute 127 sec latency 17.291 ms 2024-04-24T23:40:50.137 INFO:tasks.workunit.client.1.smithi169.stdout: 1 708313 16.30 MB/sec execute 128 sec latency 21.089 ms 2024-04-24T23:40:51.137 INFO:tasks.workunit.client.1.smithi169.stdout: 1 711036 16.29 MB/sec execute 129 sec latency 29.598 ms 2024-04-24T23:40:52.137 INFO:tasks.workunit.client.1.smithi169.stdout: 1 714343 16.30 MB/sec execute 130 sec latency 46.144 ms 2024-04-24T23:40:53.138 INFO:tasks.workunit.client.1.smithi169.stdout: 1 717409 16.31 MB/sec execute 131 sec latency 70.018 ms 2024-04-24T23:40:54.137 INFO:tasks.workunit.client.1.smithi169.stdout: 1 718031 16.20 MB/sec execute 132 sec latency 411.826 ms 2024-04-24T23:40:55.137 INFO:tasks.workunit.client.1.smithi169.stdout: 1 719681 16.13 MB/sec execute 133 sec latency 44.596 ms 2024-04-24T23:40:56.137 INFO:tasks.workunit.client.1.smithi169.stdout: 1 722524 16.14 MB/sec execute 134 sec latency 24.480 ms 2024-04-24T23:40:57.137 INFO:tasks.workunit.client.1.smithi169.stdout: 1 725508 16.14 MB/sec execute 135 sec latency 23.701 ms 2024-04-24T23:40:58.137 INFO:tasks.workunit.client.1.smithi169.stdout: 1 728738 16.15 MB/sec execute 136 sec latency 31.766 ms 2024-04-24T23:40:59.138 INFO:tasks.workunit.client.1.smithi169.stdout: 1 731879 16.17 MB/sec execute 137 sec latency 24.022 ms 2024-04-24T23:41:00.138 INFO:tasks.workunit.client.1.smithi169.stdout: 1 735029 16.18 MB/sec execute 138 sec latency 20.323 ms 2024-04-24T23:41:00.886 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:41:00.889 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:41:00.936 INFO:tasks.ceph.mds.b.smithi169.stderr:2024-04-24T23:41:00.937+0000 7f12ddd9e640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 39490) UID: 0 2024-04-24T23:41:00.937 INFO:tasks.ceph.mds.d.smithi169.stderr:2024-04-24T23:41:00.937+0000 7fcbfaa77640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 39490) UID: 0 2024-04-24T23:41:01.137 INFO:tasks.workunit.client.1.smithi169.stdout: 1 737539 16.13 MB/sec execute 139 sec latency 24.050 ms 2024-04-24T23:41:02.138 INFO:tasks.workunit.client.1.smithi169.stdout: 1 740024 16.13 MB/sec execute 140 sec latency 26.005 ms 2024-04-24T23:41:03.138 INFO:tasks.workunit.client.1.smithi169.stdout: 1 742715 16.12 MB/sec execute 141 sec latency 29.708 ms 2024-04-24T23:41:04.138 INFO:tasks.workunit.client.1.smithi169.stdout: 1 745937 16.14 MB/sec execute 142 sec latency 27.146 ms 2024-04-24T23:41:05.138 INFO:tasks.workunit.client.1.smithi169.stdout: 1 749033 16.13 MB/sec execute 143 sec latency 16.408 ms 2024-04-24T23:41:06.138 INFO:tasks.workunit.client.1.smithi169.stdout: 1 751512 16.09 MB/sec execute 144 sec latency 49.865 ms 2024-04-24T23:41:07.138 INFO:tasks.workunit.client.1.smithi169.stdout: 1 754216 16.10 MB/sec execute 145 sec latency 19.080 ms 2024-04-24T23:41:08.138 INFO:tasks.workunit.client.1.smithi169.stdout: 1 757241 16.09 MB/sec execute 146 sec latency 23.650 ms 2024-04-24T23:41:09.138 INFO:tasks.workunit.client.1.smithi169.stdout: 1 760656 16.11 MB/sec execute 147 sec latency 35.068 ms 2024-04-24T23:41:10.138 INFO:tasks.workunit.client.1.smithi169.stdout: 1 764131 16.13 MB/sec execute 148 sec latency 19.544 ms 2024-04-24T23:41:11.138 INFO:tasks.workunit.client.1.smithi169.stdout: 1 767332 16.15 MB/sec execute 149 sec latency 30.805 ms 2024-04-24T23:41:12.138 INFO:tasks.workunit.client.1.smithi169.stdout: 1 770616 16.16 MB/sec execute 150 sec latency 32.324 ms 2024-04-24T23:41:13.138 INFO:tasks.workunit.client.1.smithi169.stdout: 1 773431 16.15 MB/sec execute 151 sec latency 14.018 ms 2024-04-24T23:41:14.138 INFO:tasks.workunit.client.1.smithi169.stdout: 1 776426 16.13 MB/sec execute 152 sec latency 24.963 ms 2024-04-24T23:41:15.138 INFO:tasks.workunit.client.1.smithi169.stdout: 1 778993 16.14 MB/sec execute 153 sec latency 34.537 ms 2024-04-24T23:41:16.138 INFO:tasks.workunit.client.1.smithi169.stdout: 1 781861 16.12 MB/sec execute 154 sec latency 24.336 ms 2024-04-24T23:41:17.138 INFO:tasks.workunit.client.1.smithi169.stdout: 1 785043 16.14 MB/sec execute 155 sec latency 25.494 ms 2024-04-24T23:41:18.138 INFO:tasks.workunit.client.1.smithi169.stdout: 1 788030 16.13 MB/sec execute 156 sec latency 22.863 ms 2024-04-24T23:41:19.138 INFO:tasks.workunit.client.1.smithi169.stdout: 1 790647 16.10 MB/sec execute 157 sec latency 16.186 ms 2024-04-24T23:41:20.138 INFO:tasks.workunit.client.1.smithi169.stdout: 1 793023 16.09 MB/sec execute 158 sec latency 17.124 ms 2024-04-24T23:41:21.138 INFO:tasks.workunit.client.1.smithi169.stdout: 1 795861 16.10 MB/sec execute 159 sec latency 22.912 ms 2024-04-24T23:41:22.138 INFO:tasks.workunit.client.1.smithi169.stdout: 1 798856 16.10 MB/sec execute 160 sec latency 37.536 ms 2024-04-24T23:41:23.138 INFO:tasks.workunit.client.1.smithi169.stdout: 1 801465 16.06 MB/sec execute 161 sec latency 20.234 ms 2024-04-24T23:41:24.138 INFO:tasks.workunit.client.1.smithi169.stdout: 1 804020 16.07 MB/sec execute 162 sec latency 33.781 ms 2024-04-24T23:41:25.138 INFO:tasks.workunit.client.1.smithi169.stdout: 1 806621 16.05 MB/sec execute 163 sec latency 26.433 ms 2024-04-24T23:41:26.138 INFO:tasks.workunit.client.1.smithi169.stdout: 1 809810 16.07 MB/sec execute 164 sec latency 25.184 ms 2024-04-24T23:41:27.138 INFO:tasks.workunit.client.1.smithi169.stdout: 1 813268 16.09 MB/sec execute 165 sec latency 31.348 ms 2024-04-24T23:41:28.138 INFO:tasks.workunit.client.1.smithi169.stdout: 1 816419 16.08 MB/sec execute 166 sec latency 19.466 ms 2024-04-24T23:41:29.138 INFO:tasks.workunit.client.1.smithi169.stdout: 1 819352 16.08 MB/sec execute 167 sec latency 27.185 ms 2024-04-24T23:41:30.138 INFO:tasks.workunit.client.1.smithi169.stdout: 1 821319 16.04 MB/sec execute 168 sec latency 118.610 ms 2024-04-24T23:41:31.138 INFO:tasks.workunit.client.1.smithi169.stdout: 1 824098 16.05 MB/sec execute 169 sec latency 41.002 ms 2024-04-24T23:41:32.138 INFO:tasks.workunit.client.1.smithi169.stdout: 1 827290 16.06 MB/sec execute 170 sec latency 24.025 ms 2024-04-24T23:41:33.138 INFO:tasks.workunit.client.1.smithi169.stdout: 1 829946 16.04 MB/sec execute 171 sec latency 12.789 ms 2024-04-24T23:41:34.138 INFO:tasks.workunit.client.1.smithi169.stdout: 1 832185 16.02 MB/sec execute 172 sec latency 26.555 ms 2024-04-24T23:41:35.138 INFO:tasks.workunit.client.1.smithi169.stdout: 1 834979 16.01 MB/sec execute 173 sec latency 80.427 ms 2024-04-24T23:41:36.138 INFO:tasks.workunit.client.1.smithi169.stdout: 1 837544 15.99 MB/sec execute 174 sec latency 56.865 ms 2024-04-24T23:41:37.139 INFO:tasks.workunit.client.1.smithi169.stdout: 1 840309 15.98 MB/sec execute 175 sec latency 16.752 ms 2024-04-24T23:41:38.139 INFO:tasks.workunit.client.1.smithi169.stdout: 1 842858 15.98 MB/sec execute 176 sec latency 19.932 ms 2024-04-24T23:41:39.139 INFO:tasks.workunit.client.1.smithi169.stdout: 1 845821 15.97 MB/sec execute 177 sec latency 20.632 ms 2024-04-24T23:41:40.139 INFO:tasks.workunit.client.1.smithi169.stdout: 1 849008 15.99 MB/sec execute 178 sec latency 14.642 ms 2024-04-24T23:41:41.139 INFO:tasks.workunit.client.1.smithi169.stdout: 1 849224 15.90 MB/sec execute 179 sec latency 338.206 ms 2024-04-24T23:41:42.139 INFO:tasks.workunit.client.1.smithi169.stdout: 1 849730 15.82 MB/sec execute 180 sec latency 348.165 ms 2024-04-24T23:41:43.139 INFO:tasks.workunit.client.1.smithi169.stdout: 1 852090 15.82 MB/sec execute 181 sec latency 55.894 ms 2024-04-24T23:41:44.139 INFO:tasks.workunit.client.1.smithi169.stdout: 1 853937 15.77 MB/sec execute 182 sec latency 33.733 ms 2024-04-24T23:41:45.139 INFO:tasks.workunit.client.1.smithi169.stdout: 1 856463 15.76 MB/sec execute 183 sec latency 19.595 ms 2024-04-24T23:41:46.139 INFO:tasks.workunit.client.1.smithi169.stdout: 1 859811 15.78 MB/sec execute 184 sec latency 25.771 ms 2024-04-24T23:41:47.139 INFO:tasks.workunit.client.1.smithi169.stdout: 1 863032 15.80 MB/sec execute 185 sec latency 30.481 ms 2024-04-24T23:41:48.139 INFO:tasks.workunit.client.1.smithi169.stdout: 1 866071 15.79 MB/sec execute 186 sec latency 26.964 ms 2024-04-24T23:41:49.139 INFO:tasks.workunit.client.1.smithi169.stdout: 1 868747 15.77 MB/sec execute 187 sec latency 22.297 ms 2024-04-24T23:41:50.139 INFO:tasks.workunit.client.1.smithi169.stdout: 1 871237 15.78 MB/sec execute 188 sec latency 28.893 ms 2024-04-24T23:41:51.139 INFO:tasks.workunit.client.1.smithi169.stdout: 1 874107 15.77 MB/sec execute 189 sec latency 22.409 ms 2024-04-24T23:41:52.139 INFO:tasks.workunit.client.1.smithi169.stdout: 1 877172 15.78 MB/sec execute 190 sec latency 19.535 ms 2024-04-24T23:41:53.139 INFO:tasks.workunit.client.1.smithi169.stdout: 1 879939 15.77 MB/sec execute 191 sec latency 42.460 ms 2024-04-24T23:41:54.139 INFO:tasks.workunit.client.1.smithi169.stdout: 1 882912 15.76 MB/sec execute 192 sec latency 12.858 ms 2024-04-24T23:41:55.139 INFO:tasks.workunit.client.1.smithi169.stdout: 1 885676 15.76 MB/sec execute 193 sec latency 24.462 ms 2024-04-24T23:41:56.139 INFO:tasks.workunit.client.1.smithi169.stdout: 1 888379 15.76 MB/sec execute 194 sec latency 84.523 ms 2024-04-24T23:41:57.139 INFO:tasks.workunit.client.1.smithi169.stdout: 1 891776 15.77 MB/sec execute 195 sec latency 22.793 ms 2024-04-24T23:41:58.139 INFO:tasks.workunit.client.1.smithi169.stdout: 1 895134 15.79 MB/sec execute 196 sec latency 26.272 ms 2024-04-24T23:41:59.139 INFO:tasks.workunit.client.1.smithi169.stdout: 1 898574 15.81 MB/sec execute 197 sec latency 32.863 ms 2024-04-24T23:42:00.139 INFO:tasks.workunit.client.1.smithi169.stdout: 1 901988 15.82 MB/sec execute 198 sec latency 27.019 ms 2024-04-24T23:42:01.139 INFO:tasks.workunit.client.1.smithi169.stdout: 1 904970 15.81 MB/sec execute 199 sec latency 20.847 ms 2024-04-24T23:42:02.139 INFO:tasks.workunit.client.1.smithi169.stdout: 1 908184 15.82 MB/sec execute 200 sec latency 15.412 ms 2024-04-24T23:42:03.139 INFO:tasks.workunit.client.1.smithi169.stdout: 1 911185 15.81 MB/sec execute 201 sec latency 25.898 ms 2024-04-24T23:42:04.139 INFO:tasks.workunit.client.1.smithi169.stdout: 1 913640 15.80 MB/sec execute 202 sec latency 32.346 ms 2024-04-24T23:42:05.139 INFO:tasks.workunit.client.1.smithi169.stdout: 1 916611 15.81 MB/sec execute 203 sec latency 21.553 ms 2024-04-24T23:42:06.139 INFO:tasks.workunit.client.1.smithi169.stdout: 1 919486 15.89 MB/sec execute 204 sec latency 61.418 ms 2024-04-24T23:42:07.140 INFO:tasks.workunit.client.1.smithi169.stdout: 1 920228 15.84 MB/sec execute 205 sec latency 244.697 ms 2024-04-24T23:42:08.140 INFO:tasks.workunit.client.1.smithi169.stdout: 1 920931 15.78 MB/sec execute 206 sec latency 148.088 ms 2024-04-24T23:42:09.140 INFO:tasks.workunit.client.1.smithi169.stdout: 1 923243 15.76 MB/sec execute 207 sec latency 35.761 ms 2024-04-24T23:42:10.140 INFO:tasks.workunit.client.1.smithi169.stdout: 1 926239 15.75 MB/sec execute 208 sec latency 32.320 ms 2024-04-24T23:42:11.140 INFO:tasks.workunit.client.1.smithi169.stdout: 1 928946 15.76 MB/sec execute 209 sec latency 16.054 ms 2024-04-24T23:42:12.140 INFO:tasks.workunit.client.1.smithi169.stdout: 1 931693 15.75 MB/sec execute 210 sec latency 23.629 ms 2024-04-24T23:42:13.140 INFO:tasks.workunit.client.1.smithi169.stdout: 1 934877 15.76 MB/sec execute 211 sec latency 31.067 ms 2024-04-24T23:42:14.140 INFO:tasks.workunit.client.1.smithi169.stdout: 1 938269 15.78 MB/sec execute 212 sec latency 32.300 ms 2024-04-24T23:42:15.140 INFO:tasks.workunit.client.1.smithi169.stdout: 1 941542 15.79 MB/sec execute 213 sec latency 43.522 ms 2024-04-24T23:42:16.140 INFO:tasks.workunit.client.1.smithi169.stdout: 1 944158 15.76 MB/sec execute 214 sec latency 45.266 ms 2024-04-24T23:42:17.140 INFO:tasks.workunit.client.1.smithi169.stdout: 1 947247 15.76 MB/sec execute 215 sec latency 13.124 ms 2024-04-24T23:42:18.140 INFO:tasks.workunit.client.1.smithi169.stdout: 1 949873 15.76 MB/sec execute 216 sec latency 22.032 ms 2024-04-24T23:42:19.140 INFO:tasks.workunit.client.1.smithi169.stdout: 1 953015 15.77 MB/sec execute 217 sec latency 20.672 ms 2024-04-24T23:42:20.140 INFO:tasks.workunit.client.1.smithi169.stdout: 1 956483 15.79 MB/sec execute 218 sec latency 12.614 ms 2024-04-24T23:42:21.140 INFO:tasks.workunit.client.1.smithi169.stdout: 1 959835 15.80 MB/sec execute 219 sec latency 37.808 ms 2024-04-24T23:42:22.140 INFO:tasks.workunit.client.1.smithi169.stdout: 1 963081 15.81 MB/sec execute 220 sec latency 37.046 ms 2024-04-24T23:42:23.140 INFO:tasks.workunit.client.1.smithi169.stdout: 1 966159 15.81 MB/sec execute 221 sec latency 32.611 ms 2024-04-24T23:42:24.140 INFO:tasks.workunit.client.1.smithi169.stdout: 1 968936 15.79 MB/sec execute 222 sec latency 16.085 ms 2024-04-24T23:42:25.140 INFO:tasks.workunit.client.1.smithi169.stdout: 1 971372 15.80 MB/sec execute 223 sec latency 33.602 ms 2024-04-24T23:42:26.140 INFO:tasks.workunit.client.1.smithi169.stdout: 1 974130 15.79 MB/sec execute 224 sec latency 15.104 ms 2024-04-24T23:42:27.140 INFO:tasks.workunit.client.1.smithi169.stdout: 1 977002 15.80 MB/sec execute 225 sec latency 51.001 ms 2024-04-24T23:42:28.140 INFO:tasks.workunit.client.1.smithi169.stdout: 1 979425 15.76 MB/sec execute 226 sec latency 25.015 ms 2024-04-24T23:42:29.140 INFO:tasks.workunit.client.1.smithi169.stdout: 1 981629 15.76 MB/sec execute 227 sec latency 12.253 ms 2024-04-24T23:42:30.140 INFO:tasks.workunit.client.1.smithi169.stdout: 1 984853 15.76 MB/sec execute 228 sec latency 36.446 ms 2024-04-24T23:42:31.140 INFO:tasks.workunit.client.1.smithi169.stdout: 1 988053 15.78 MB/sec execute 229 sec latency 45.489 ms 2024-04-24T23:42:32.140 INFO:tasks.workunit.client.1.smithi169.stdout: 1 990461 15.76 MB/sec execute 230 sec latency 187.505 ms 2024-04-24T23:42:33.140 INFO:tasks.workunit.client.1.smithi169.stdout: 1 991193 15.72 MB/sec execute 231 sec latency 422.388 ms 2024-04-24T23:42:34.140 INFO:tasks.workunit.client.1.smithi169.stdout: 1 992935 15.68 MB/sec execute 232 sec latency 37.799 ms 2024-04-24T23:42:35.140 INFO:tasks.workunit.client.1.smithi169.stdout: 1 995443 15.67 MB/sec execute 233 sec latency 29.773 ms 2024-04-24T23:42:36.140 INFO:tasks.workunit.client.1.smithi169.stdout: 1 998567 15.68 MB/sec execute 234 sec latency 73.277 ms 2024-04-24T23:42:37.140 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1001659 15.68 MB/sec execute 235 sec latency 34.964 ms 2024-04-24T23:42:38.140 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1004845 15.69 MB/sec execute 236 sec latency 12.314 ms 2024-04-24T23:42:39.140 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1007674 15.67 MB/sec execute 237 sec latency 28.962 ms 2024-04-24T23:42:40.140 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1010410 15.68 MB/sec execute 238 sec latency 32.488 ms 2024-04-24T23:42:41.140 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1013412 15.68 MB/sec execute 239 sec latency 24.805 ms 2024-04-24T23:42:42.140 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1016527 15.69 MB/sec execute 240 sec latency 30.207 ms 2024-04-24T23:42:43.141 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1019732 15.70 MB/sec execute 241 sec latency 19.415 ms 2024-04-24T23:42:44.141 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1022607 15.69 MB/sec execute 242 sec latency 28.523 ms 2024-04-24T23:42:45.141 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1025508 15.68 MB/sec execute 243 sec latency 22.701 ms 2024-04-24T23:42:46.141 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1027757 15.68 MB/sec execute 244 sec latency 20.536 ms 2024-04-24T23:42:47.141 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1030898 15.68 MB/sec execute 245 sec latency 34.198 ms 2024-04-24T23:42:48.141 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1034094 15.70 MB/sec execute 246 sec latency 37.044 ms 2024-04-24T23:42:49.141 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1037128 15.69 MB/sec execute 247 sec latency 40.132 ms 2024-04-24T23:42:50.141 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1039769 15.68 MB/sec execute 248 sec latency 37.393 ms 2024-04-24T23:42:51.141 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1042433 15.68 MB/sec execute 249 sec latency 51.389 ms 2024-04-24T23:42:52.141 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1045328 15.68 MB/sec execute 250 sec latency 22.117 ms 2024-04-24T23:42:53.141 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1048501 15.69 MB/sec execute 251 sec latency 29.250 ms 2024-04-24T23:42:54.141 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1051647 15.70 MB/sec execute 252 sec latency 29.303 ms 2024-04-24T23:42:55.141 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1054695 15.69 MB/sec execute 253 sec latency 33.391 ms 2024-04-24T23:42:56.141 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1057425 15.68 MB/sec execute 254 sec latency 48.226 ms 2024-04-24T23:42:57.141 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1060193 15.69 MB/sec execute 255 sec latency 28.770 ms 2024-04-24T23:42:58.141 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1062974 15.68 MB/sec execute 256 sec latency 32.000 ms 2024-04-24T23:42:59.141 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1066236 15.70 MB/sec execute 257 sec latency 19.941 ms 2024-04-24T23:43:00.141 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1069464 15.71 MB/sec execute 258 sec latency 26.126 ms 2024-04-24T23:43:01.141 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1072156 15.70 MB/sec execute 259 sec latency 29.952 ms 2024-04-24T23:43:02.141 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1074675 15.68 MB/sec execute 260 sec latency 27.560 ms 2024-04-24T23:43:03.141 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1077762 15.69 MB/sec execute 261 sec latency 13.197 ms 2024-04-24T23:43:04.141 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1080555 15.69 MB/sec execute 262 sec latency 31.519 ms 2024-04-24T23:43:05.141 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1083499 15.70 MB/sec execute 263 sec latency 101.444 ms 2024-04-24T23:43:06.141 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1085905 15.67 MB/sec execute 264 sec latency 25.714 ms 2024-04-24T23:43:07.141 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1088212 15.66 MB/sec execute 265 sec latency 38.500 ms 2024-04-24T23:43:08.142 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1091243 15.67 MB/sec execute 266 sec latency 16.922 ms 2024-04-24T23:43:09.141 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1093858 15.66 MB/sec execute 267 sec latency 107.834 ms 2024-04-24T23:43:10.141 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1095694 15.64 MB/sec execute 268 sec latency 29.607 ms 2024-04-24T23:43:11.141 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1098391 15.64 MB/sec execute 269 sec latency 15.423 ms 2024-04-24T23:43:12.141 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1101562 15.65 MB/sec execute 270 sec latency 26.241 ms 2024-04-24T23:43:13.141 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1104811 15.66 MB/sec execute 271 sec latency 27.793 ms 2024-04-24T23:43:14.141 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1106470 15.62 MB/sec execute 272 sec latency 201.593 ms 2024-04-24T23:43:15.141 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1106537 15.57 MB/sec execute 273 sec latency 536.201 ms 2024-04-24T23:43:16.141 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1108315 15.55 MB/sec execute 274 sec latency 469.477 ms 2024-04-24T23:43:17.142 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1108976 15.50 MB/sec execute 275 sec latency 27.591 ms 2024-04-24T23:43:18.142 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1111940 15.51 MB/sec execute 276 sec latency 104.208 ms 2024-04-24T23:43:19.142 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1114762 15.51 MB/sec execute 277 sec latency 24.447 ms 2024-04-24T23:43:20.142 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1117662 15.50 MB/sec execute 278 sec latency 21.721 ms 2024-04-24T23:43:21.142 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1120061 15.49 MB/sec execute 279 sec latency 11.982 ms 2024-04-24T23:43:22.142 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1122923 15.50 MB/sec execute 280 sec latency 25.834 ms 2024-04-24T23:43:23.142 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1126070 15.51 MB/sec execute 281 sec latency 24.998 ms 2024-04-24T23:43:24.142 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1128885 15.50 MB/sec execute 282 sec latency 26.303 ms 2024-04-24T23:43:25.142 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1131980 15.50 MB/sec execute 283 sec latency 22.853 ms 2024-04-24T23:43:26.142 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1134984 15.50 MB/sec execute 284 sec latency 28.155 ms 2024-04-24T23:43:27.142 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1138063 15.51 MB/sec execute 285 sec latency 31.757 ms 2024-04-24T23:43:28.142 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1141181 15.51 MB/sec execute 286 sec latency 27.124 ms 2024-04-24T23:43:29.142 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1144465 15.52 MB/sec execute 287 sec latency 26.458 ms 2024-04-24T23:43:30.142 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1147707 15.53 MB/sec execute 288 sec latency 25.598 ms 2024-04-24T23:43:31.142 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1150969 15.54 MB/sec execute 289 sec latency 18.027 ms 2024-04-24T23:43:32.142 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1153826 15.54 MB/sec execute 290 sec latency 32.158 ms 2024-04-24T23:43:33.142 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1156730 15.53 MB/sec execute 291 sec latency 32.060 ms 2024-04-24T23:43:34.142 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1158929 15.52 MB/sec execute 292 sec latency 34.995 ms 2024-04-24T23:43:35.142 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1162094 15.53 MB/sec execute 293 sec latency 24.303 ms 2024-04-24T23:43:36.142 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1165164 15.54 MB/sec execute 294 sec latency 18.501 ms 2024-04-24T23:43:37.143 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1167454 15.51 MB/sec execute 295 sec latency 40.405 ms 2024-04-24T23:43:38.142 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1170126 15.52 MB/sec execute 296 sec latency 37.060 ms 2024-04-24T23:43:39.142 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1173087 15.52 MB/sec execute 297 sec latency 19.076 ms 2024-04-24T23:43:40.142 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1176368 15.53 MB/sec execute 298 sec latency 43.313 ms 2024-04-24T23:43:41.142 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1179610 15.54 MB/sec execute 299 sec latency 49.146 ms 2024-04-24T23:43:42.142 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1183004 15.55 MB/sec execute 300 sec latency 28.731 ms 2024-04-24T23:43:43.142 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1186283 15.56 MB/sec execute 301 sec latency 15.836 ms 2024-04-24T23:43:44.142 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1189500 15.55 MB/sec execute 302 sec latency 24.126 ms 2024-04-24T23:43:45.142 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1192758 15.56 MB/sec execute 303 sec latency 24.108 ms 2024-04-24T23:43:46.142 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1195835 15.56 MB/sec execute 304 sec latency 21.392 ms 2024-04-24T23:43:47.142 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1199307 15.57 MB/sec execute 305 sec latency 26.217 ms 2024-04-24T23:43:48.142 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1202031 15.57 MB/sec execute 306 sec latency 24.058 ms 2024-04-24T23:43:49.142 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1205059 15.57 MB/sec execute 307 sec latency 24.712 ms 2024-04-24T23:43:49.997 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:43:50.003 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:43:50.142 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1208369 15.58 MB/sec execute 308 sec latency 31.007 ms 2024-04-24T23:43:51.142 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1211206 15.59 MB/sec execute 309 sec latency 71.711 ms 2024-04-24T23:43:52.142 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1214019 15.58 MB/sec execute 310 sec latency 22.625 ms 2024-04-24T23:43:53.142 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1216555 15.57 MB/sec execute 311 sec latency 38.650 ms 2024-04-24T23:43:54.143 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1219134 15.57 MB/sec execute 312 sec latency 21.539 ms 2024-04-24T23:43:55.146 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1222229 15.58 MB/sec execute 313 sec latency 25.649 ms 2024-04-24T23:43:56.144 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1224307 15.55 MB/sec execute 314 sec latency 143.463 ms 2024-04-24T23:43:57.143 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1226427 15.54 MB/sec execute 315 sec latency 28.285 ms 2024-04-24T23:43:58.143 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1229510 15.55 MB/sec execute 316 sec latency 19.349 ms 2024-04-24T23:43:59.143 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1232624 15.56 MB/sec execute 317 sec latency 19.794 ms 2024-04-24T23:44:00.143 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1235413 15.55 MB/sec execute 318 sec latency 17.282 ms 2024-04-24T23:44:01.143 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1236821 15.52 MB/sec execute 319 sec latency 96.380 ms 2024-04-24T23:44:02.143 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1239922 15.53 MB/sec execute 320 sec latency 42.056 ms 2024-04-24T23:44:03.143 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1243288 15.54 MB/sec execute 321 sec latency 30.350 ms 2024-04-24T23:44:04.143 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1246320 15.54 MB/sec execute 322 sec latency 23.954 ms 2024-04-24T23:44:05.143 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1248976 15.53 MB/sec execute 323 sec latency 35.802 ms 2024-04-24T23:44:06.143 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1250380 15.52 MB/sec execute 324 sec latency 450.114 ms 2024-04-24T23:44:07.143 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1250559 15.47 MB/sec execute 325 sec latency 945.849 ms 2024-04-24T23:44:08.143 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1251003 15.43 MB/sec execute 326 sec latency 59.840 ms 2024-04-24T23:44:09.143 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1253625 15.43 MB/sec execute 327 sec latency 36.741 ms 2024-04-24T23:44:10.143 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1256249 15.41 MB/sec execute 328 sec latency 28.519 ms 2024-04-24T23:44:11.143 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1258631 15.41 MB/sec execute 329 sec latency 78.248 ms 2024-04-24T23:44:12.143 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1261556 15.41 MB/sec execute 330 sec latency 29.218 ms 2024-04-24T23:44:13.144 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1264873 15.42 MB/sec execute 331 sec latency 24.169 ms 2024-04-24T23:44:14.143 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1268138 15.43 MB/sec execute 332 sec latency 25.686 ms 2024-04-24T23:44:15.143 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1271407 15.44 MB/sec execute 333 sec latency 29.833 ms 2024-04-24T23:44:16.143 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1274288 15.43 MB/sec execute 334 sec latency 50.745 ms 2024-04-24T23:44:17.146 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1277236 15.43 MB/sec execute 335 sec latency 24.100 ms 2024-04-24T23:44:18.146 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1279889 15.43 MB/sec execute 336 sec latency 35.791 ms 2024-04-24T23:44:19.147 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1282940 15.43 MB/sec execute 337 sec latency 39.916 ms 2024-04-24T23:44:20.149 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1286293 15.44 MB/sec execute 338 sec latency 39.729 ms 2024-04-24T23:44:20.152 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:44:20.155 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:44:21.147 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1289727 15.45 MB/sec execute 339 sec latency 10.658 ms 2024-04-24T23:44:22.147 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1292974 15.46 MB/sec execute 340 sec latency 41.597 ms 2024-04-24T23:44:23.149 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1296309 15.47 MB/sec execute 341 sec latency 17.688 ms 2024-04-24T23:44:24.149 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1299417 15.47 MB/sec execute 342 sec latency 29.545 ms 2024-04-24T23:44:25.149 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1302375 15.46 MB/sec execute 343 sec latency 28.174 ms 2024-04-24T23:44:26.149 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1305279 15.47 MB/sec execute 344 sec latency 38.029 ms 2024-04-24T23:44:27.149 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1308015 15.47 MB/sec execute 345 sec latency 27.251 ms 2024-04-24T23:44:28.149 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1309911 15.45 MB/sec execute 346 sec latency 500.609 ms 2024-04-24T23:44:29.149 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1311985 15.44 MB/sec execute 347 sec latency 216.791 ms 2024-04-24T23:44:30.149 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1313999 15.43 MB/sec execute 348 sec latency 56.438 ms 2024-04-24T23:44:31.149 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1317385 15.43 MB/sec execute 349 sec latency 13.156 ms 2024-04-24T23:44:32.149 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1320550 15.44 MB/sec execute 350 sec latency 19.515 ms 2024-04-24T23:44:33.149 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1323542 15.43 MB/sec execute 351 sec latency 27.089 ms 2024-04-24T23:44:34.149 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1326125 15.44 MB/sec execute 352 sec latency 32.932 ms 2024-04-24T23:44:35.150 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1329103 15.44 MB/sec execute 353 sec latency 25.338 ms 2024-04-24T23:44:36.149 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1332395 15.45 MB/sec execute 354 sec latency 30.643 ms 2024-04-24T23:44:37.149 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1335583 15.46 MB/sec execute 355 sec latency 16.329 ms 2024-04-24T23:44:38.149 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1338465 15.45 MB/sec execute 356 sec latency 38.690 ms 2024-04-24T23:44:39.150 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1341189 15.44 MB/sec execute 357 sec latency 19.435 ms 2024-04-24T23:44:40.149 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1343399 15.44 MB/sec execute 358 sec latency 22.718 ms 2024-04-24T23:44:41.149 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1346453 15.44 MB/sec execute 359 sec latency 24.354 ms 2024-04-24T23:44:42.149 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1349856 15.45 MB/sec execute 360 sec latency 17.943 ms 2024-04-24T23:44:43.150 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1353238 15.46 MB/sec execute 361 sec latency 18.835 ms 2024-04-24T23:44:44.150 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1356509 15.47 MB/sec execute 362 sec latency 25.547 ms 2024-04-24T23:44:45.150 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1359731 15.47 MB/sec execute 363 sec latency 28.193 ms 2024-04-24T23:44:46.150 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1362646 15.46 MB/sec execute 364 sec latency 21.537 ms 2024-04-24T23:44:47.150 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1365243 15.47 MB/sec execute 365 sec latency 24.410 ms 2024-04-24T23:44:48.150 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1367992 15.46 MB/sec execute 366 sec latency 19.298 ms 2024-04-24T23:44:49.150 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1371174 15.47 MB/sec execute 367 sec latency 23.178 ms 2024-04-24T23:44:50.150 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1374416 15.48 MB/sec execute 368 sec latency 29.471 ms 2024-04-24T23:44:50.266 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:44:50.269 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:44:51.150 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1376289 15.50 MB/sec execute 369 sec latency 35.907 ms 2024-04-24T23:44:52.150 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1379535 15.51 MB/sec execute 370 sec latency 17.068 ms 2024-04-24T23:44:53.150 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1382914 15.52 MB/sec execute 371 sec latency 24.541 ms 2024-04-24T23:44:54.150 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1386488 15.53 MB/sec execute 372 sec latency 11.858 ms 2024-04-24T23:44:55.150 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1389788 15.54 MB/sec execute 373 sec latency 32.346 ms 2024-04-24T23:44:56.150 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1393050 15.54 MB/sec execute 374 sec latency 12.233 ms 2024-04-24T23:44:57.150 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1395837 15.54 MB/sec execute 375 sec latency 12.932 ms 2024-04-24T23:44:58.150 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1397994 15.53 MB/sec execute 376 sec latency 43.491 ms 2024-04-24T23:44:59.150 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1400809 15.53 MB/sec execute 377 sec latency 11.605 ms 2024-04-24T23:45:00.150 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1403986 15.53 MB/sec execute 378 sec latency 8.411 ms 2024-04-24T23:45:01.150 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1406806 15.53 MB/sec execute 379 sec latency 74.078 ms 2024-04-24T23:45:02.150 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1409873 15.53 MB/sec execute 380 sec latency 14.065 ms 2024-04-24T23:45:03.150 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1412957 15.53 MB/sec execute 381 sec latency 11.686 ms 2024-04-24T23:45:04.150 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1413518 15.50 MB/sec execute 382 sec latency 693.850 ms 2024-04-24T23:45:05.150 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1414319 15.48 MB/sec execute 383 sec latency 1152.938 ms 2024-04-24T23:45:06.150 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1414829 15.44 MB/sec execute 384 sec latency 55.745 ms 2024-04-24T23:45:07.150 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1416519 15.42 MB/sec execute 385 sec latency 33.680 ms 2024-04-24T23:45:08.150 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1418632 15.41 MB/sec execute 386 sec latency 29.633 ms 2024-04-24T23:45:09.150 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1421734 15.42 MB/sec execute 387 sec latency 26.826 ms 2024-04-24T23:45:10.150 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1424955 15.43 MB/sec execute 388 sec latency 24.396 ms 2024-04-24T23:45:11.150 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1427742 15.42 MB/sec execute 389 sec latency 39.054 ms 2024-04-24T23:45:12.151 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1430635 15.42 MB/sec execute 390 sec latency 32.395 ms 2024-04-24T23:45:13.150 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1433247 15.42 MB/sec execute 391 sec latency 11.735 ms 2024-04-24T23:45:14.151 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1435980 15.42 MB/sec execute 392 sec latency 64.300 ms 2024-04-24T23:45:15.150 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1438911 15.42 MB/sec execute 393 sec latency 30.338 ms 2024-04-24T23:45:16.150 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1441292 15.41 MB/sec execute 394 sec latency 15.329 ms 2024-04-24T23:45:17.150 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1443431 15.40 MB/sec execute 395 sec latency 22.701 ms 2024-04-24T23:45:18.150 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1446321 15.41 MB/sec execute 396 sec latency 29.215 ms 2024-04-24T23:45:19.150 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1449007 15.40 MB/sec execute 397 sec latency 15.021 ms 2024-04-24T23:45:20.151 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1451977 15.39 MB/sec execute 398 sec latency 23.323 ms 2024-04-24T23:45:20.556 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:45:20.559 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:45:20.589 INFO:tasks.ceph.mds.b.smithi169.stderr:2024-04-24T23:45:20.590+0000 7f12ddd9e640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 39575) UID: 0 2024-04-24T23:45:20.590 INFO:tasks.ceph.mds.d.smithi169.stderr:2024-04-24T23:45:20.590+0000 7fcbfaa77640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 39575) UID: 0 2024-04-24T23:45:21.150 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1454282 15.39 MB/sec execute 399 sec latency 27.723 ms 2024-04-24T23:45:22.151 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1457349 15.40 MB/sec execute 400 sec latency 32.983 ms 2024-04-24T23:45:23.151 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1460505 15.40 MB/sec execute 401 sec latency 47.798 ms 2024-04-24T23:45:24.151 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1462472 15.38 MB/sec execute 402 sec latency 235.379 ms 2024-04-24T23:45:25.151 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1463983 15.38 MB/sec execute 403 sec latency 373.292 ms 2024-04-24T23:45:26.151 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1464565 15.34 MB/sec execute 404 sec latency 121.803 ms 2024-04-24T23:45:27.151 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1467540 15.35 MB/sec execute 405 sec latency 33.403 ms 2024-04-24T23:45:28.151 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1470468 15.34 MB/sec execute 406 sec latency 31.231 ms 2024-04-24T23:45:29.151 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1472534 15.33 MB/sec execute 407 sec latency 25.084 ms 2024-04-24T23:45:30.151 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1475116 15.33 MB/sec execute 408 sec latency 34.211 ms 2024-04-24T23:45:31.151 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1477944 15.33 MB/sec execute 409 sec latency 39.463 ms 2024-04-24T23:45:32.151 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1480332 15.32 MB/sec execute 410 sec latency 30.033 ms 2024-04-24T23:45:33.151 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1482290 15.31 MB/sec execute 411 sec latency 32.653 ms 2024-04-24T23:45:34.151 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1485169 15.32 MB/sec execute 412 sec latency 31.040 ms 2024-04-24T23:45:35.151 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1487613 15.30 MB/sec execute 413 sec latency 30.788 ms 2024-04-24T23:45:36.151 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1490105 15.30 MB/sec execute 414 sec latency 17.582 ms 2024-04-24T23:45:37.151 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1492863 15.30 MB/sec execute 415 sec latency 23.917 ms 2024-04-24T23:45:38.151 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1495906 15.31 MB/sec execute 416 sec latency 37.017 ms 2024-04-24T23:45:39.151 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1498255 15.29 MB/sec execute 417 sec latency 13.123 ms 2024-04-24T23:45:40.151 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1500384 15.29 MB/sec execute 418 sec latency 21.477 ms 2024-04-24T23:45:41.151 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1503426 15.29 MB/sec execute 419 sec latency 31.621 ms 2024-04-24T23:45:42.151 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1506522 15.30 MB/sec execute 420 sec latency 36.773 ms 2024-04-24T23:45:43.152 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1509132 15.29 MB/sec execute 421 sec latency 13.262 ms 2024-04-24T23:45:44.151 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1511324 15.29 MB/sec execute 422 sec latency 29.805 ms 2024-04-24T23:45:45.151 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1514122 15.28 MB/sec execute 423 sec latency 21.502 ms 2024-04-24T23:45:46.151 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1517338 15.29 MB/sec execute 424 sec latency 39.679 ms 2024-04-24T23:45:47.151 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1520278 15.29 MB/sec execute 425 sec latency 28.199 ms 2024-04-24T23:45:48.151 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1523088 15.28 MB/sec execute 426 sec latency 22.247 ms 2024-04-24T23:45:49.151 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1525591 15.29 MB/sec execute 427 sec latency 26.232 ms 2024-04-24T23:45:50.151 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1528388 15.28 MB/sec execute 428 sec latency 26.035 ms 2024-04-24T23:45:51.152 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1531688 15.29 MB/sec execute 429 sec latency 30.459 ms 2024-04-24T23:45:52.151 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1535057 15.30 MB/sec execute 430 sec latency 35.410 ms 2024-04-24T23:45:53.151 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1538011 15.30 MB/sec execute 431 sec latency 40.487 ms 2024-04-24T23:45:54.155 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1540773 15.29 MB/sec execute 432 sec latency 17.927 ms 2024-04-24T23:45:55.152 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1543294 15.29 MB/sec execute 433 sec latency 26.426 ms 2024-04-24T23:45:56.151 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1546026 15.29 MB/sec execute 434 sec latency 20.779 ms 2024-04-24T23:45:57.152 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1549294 15.30 MB/sec execute 435 sec latency 18.577 ms 2024-04-24T23:45:58.152 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1552228 15.30 MB/sec execute 436 sec latency 42.413 ms 2024-04-24T23:45:59.152 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1554993 15.29 MB/sec execute 437 sec latency 22.726 ms 2024-04-24T23:46:00.152 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1556564 15.28 MB/sec execute 438 sec latency 237.508 ms 2024-04-24T23:46:01.152 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1556623 15.25 MB/sec execute 439 sec latency 393.654 ms 2024-04-24T23:46:02.152 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1557397 15.22 MB/sec execute 440 sec latency 85.199 ms 2024-04-24T23:46:03.152 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1559577 15.22 MB/sec execute 441 sec latency 64.534 ms 2024-04-24T23:46:04.152 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1562095 15.21 MB/sec execute 442 sec latency 35.676 ms 2024-04-24T23:46:05.152 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1564597 15.21 MB/sec execute 443 sec latency 29.922 ms 2024-04-24T23:46:06.152 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1567557 15.21 MB/sec execute 444 sec latency 19.893 ms 2024-04-24T23:46:07.152 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1570814 15.21 MB/sec execute 445 sec latency 21.277 ms 2024-04-24T23:46:08.152 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1574060 15.22 MB/sec execute 446 sec latency 20.605 ms 2024-04-24T23:46:09.152 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1577288 15.23 MB/sec execute 447 sec latency 24.054 ms 2024-04-24T23:46:10.152 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1579771 15.22 MB/sec execute 448 sec latency 30.310 ms 2024-04-24T23:46:11.152 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1582267 15.22 MB/sec execute 449 sec latency 25.305 ms 2024-04-24T23:46:12.152 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1584977 15.22 MB/sec execute 450 sec latency 30.558 ms 2024-04-24T23:46:13.152 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1588071 15.22 MB/sec execute 451 sec latency 35.587 ms 2024-04-24T23:46:14.152 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1591101 15.22 MB/sec execute 452 sec latency 34.133 ms 2024-04-24T23:46:15.152 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1593933 15.22 MB/sec execute 453 sec latency 31.089 ms 2024-04-24T23:46:16.152 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1596449 15.22 MB/sec execute 454 sec latency 29.428 ms 2024-04-24T23:46:17.152 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1599335 15.22 MB/sec execute 455 sec latency 26.697 ms 2024-04-24T23:46:18.152 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1602376 15.22 MB/sec execute 456 sec latency 48.616 ms 2024-04-24T23:46:19.154 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1605274 15.22 MB/sec execute 457 sec latency 41.530 ms 2024-04-24T23:46:20.152 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1607631 15.21 MB/sec execute 458 sec latency 22.886 ms 2024-04-24T23:46:21.152 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1609555 15.21 MB/sec execute 459 sec latency 346.871 ms 2024-04-24T23:46:22.152 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1609846 15.18 MB/sec execute 460 sec latency 463.855 ms 2024-04-24T23:46:23.153 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1611964 15.17 MB/sec execute 461 sec latency 56.484 ms 2024-04-24T23:46:24.152 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1614432 15.16 MB/sec execute 462 sec latency 18.396 ms 2024-04-24T23:46:25.153 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1617194 15.16 MB/sec execute 463 sec latency 34.222 ms 2024-04-24T23:46:26.152 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1620242 15.17 MB/sec execute 464 sec latency 28.507 ms 2024-04-24T23:46:27.153 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1623181 15.17 MB/sec execute 465 sec latency 43.789 ms 2024-04-24T23:46:28.153 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1625816 15.16 MB/sec execute 466 sec latency 31.352 ms 2024-04-24T23:46:29.153 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1628571 15.16 MB/sec execute 467 sec latency 32.704 ms 2024-04-24T23:46:30.153 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1631376 15.16 MB/sec execute 468 sec latency 19.648 ms 2024-04-24T23:46:31.153 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1634468 15.17 MB/sec execute 469 sec latency 14.659 ms 2024-04-24T23:46:32.153 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1637683 15.18 MB/sec execute 470 sec latency 57.292 ms 2024-04-24T23:46:33.153 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1640443 15.17 MB/sec execute 471 sec latency 26.084 ms 2024-04-24T23:46:34.153 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1642891 15.17 MB/sec execute 472 sec latency 12.260 ms 2024-04-24T23:46:35.153 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1645424 15.16 MB/sec execute 473 sec latency 19.297 ms 2024-04-24T23:46:36.153 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1648447 15.17 MB/sec execute 474 sec latency 66.053 ms 2024-04-24T23:46:37.153 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1651102 15.17 MB/sec execute 475 sec latency 35.409 ms 2024-04-24T23:46:38.153 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1652311 15.15 MB/sec execute 476 sec latency 152.975 ms 2024-04-24T23:46:39.153 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1653213 15.13 MB/sec execute 477 sec latency 255.870 ms 2024-04-24T23:46:40.153 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1655840 15.12 MB/sec execute 478 sec latency 50.144 ms 2024-04-24T23:46:41.153 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1659019 15.13 MB/sec execute 479 sec latency 27.822 ms 2024-04-24T23:46:42.153 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1661807 15.13 MB/sec execute 480 sec latency 25.912 ms 2024-04-24T23:46:43.153 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1664461 15.12 MB/sec execute 481 sec latency 20.306 ms 2024-04-24T23:46:44.153 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1666652 15.12 MB/sec execute 482 sec latency 39.036 ms 2024-04-24T23:46:45.153 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1669679 15.12 MB/sec execute 483 sec latency 39.699 ms 2024-04-24T23:46:46.153 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1671894 15.11 MB/sec execute 484 sec latency 40.884 ms 2024-04-24T23:46:47.153 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1674077 15.11 MB/sec execute 485 sec latency 21.202 ms 2024-04-24T23:46:48.153 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1676942 15.11 MB/sec execute 486 sec latency 39.747 ms 2024-04-24T23:46:49.153 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1679930 15.11 MB/sec execute 487 sec latency 37.407 ms 2024-04-24T23:46:50.153 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1682589 15.10 MB/sec execute 488 sec latency 32.007 ms 2024-04-24T23:46:51.153 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1685539 15.11 MB/sec execute 489 sec latency 13.920 ms 2024-04-24T23:46:52.153 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1688090 15.11 MB/sec execute 490 sec latency 26.810 ms 2024-04-24T23:46:53.153 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1691143 15.11 MB/sec execute 491 sec latency 41.243 ms 2024-04-24T23:46:54.153 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1694037 15.11 MB/sec execute 492 sec latency 112.373 ms 2024-04-24T23:46:55.153 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1694574 15.09 MB/sec execute 493 sec latency 620.882 ms 2024-04-24T23:46:56.154 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1696141 15.07 MB/sec execute 494 sec latency 50.230 ms 2024-04-24T23:46:57.154 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1698632 15.07 MB/sec execute 495 sec latency 42.073 ms 2024-04-24T23:46:58.154 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1701963 15.08 MB/sec execute 496 sec latency 26.944 ms 2024-04-24T23:46:59.154 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1704987 15.08 MB/sec execute 497 sec latency 48.245 ms 2024-04-24T23:47:00.154 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1707342 15.07 MB/sec execute 498 sec latency 36.792 ms 2024-04-24T23:47:01.154 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1709519 15.07 MB/sec execute 499 sec latency 24.735 ms 2024-04-24T23:47:02.154 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1712462 15.07 MB/sec execute 500 sec latency 23.870 ms 2024-04-24T23:47:03.154 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1715541 15.07 MB/sec execute 501 sec latency 32.043 ms 2024-04-24T23:47:04.154 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1717992 15.06 MB/sec execute 502 sec latency 52.364 ms 2024-04-24T23:47:05.154 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1720388 15.06 MB/sec execute 503 sec latency 19.040 ms 2024-04-24T23:47:06.154 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1723382 15.07 MB/sec execute 504 sec latency 37.692 ms 2024-04-24T23:47:07.154 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1726691 15.07 MB/sec execute 505 sec latency 24.771 ms 2024-04-24T23:47:08.154 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1729897 15.08 MB/sec execute 506 sec latency 36.643 ms 2024-04-24T23:47:09.154 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1732495 15.07 MB/sec execute 507 sec latency 42.868 ms 2024-04-24T23:47:10.154 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1734890 15.07 MB/sec execute 508 sec latency 31.397 ms 2024-04-24T23:47:11.154 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1737507 15.07 MB/sec execute 509 sec latency 48.276 ms 2024-04-24T23:47:12.154 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1740644 15.07 MB/sec execute 510 sec latency 24.540 ms 2024-04-24T23:47:13.154 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1743291 15.07 MB/sec execute 511 sec latency 28.985 ms 2024-04-24T23:47:14.154 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1745610 15.06 MB/sec execute 512 sec latency 47.055 ms 2024-04-24T23:47:15.154 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1747252 15.05 MB/sec execute 513 sec latency 641.777 ms 2024-04-24T23:47:16.154 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1748266 15.03 MB/sec execute 514 sec latency 743.837 ms 2024-04-24T23:47:17.155 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1749939 15.02 MB/sec execute 515 sec latency 71.635 ms 2024-04-24T23:47:18.154 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1752459 15.02 MB/sec execute 516 sec latency 35.861 ms 2024-04-24T23:47:19.154 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1755400 15.02 MB/sec execute 517 sec latency 15.918 ms 2024-04-24T23:47:20.155 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1758602 15.03 MB/sec execute 518 sec latency 22.422 ms 2024-04-24T23:47:21.155 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1761832 15.03 MB/sec execute 519 sec latency 17.709 ms 2024-04-24T23:47:22.155 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1764290 15.02 MB/sec execute 520 sec latency 33.062 ms 2024-04-24T23:47:23.155 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1766889 15.02 MB/sec execute 521 sec latency 30.852 ms 2024-04-24T23:47:24.155 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1769538 15.02 MB/sec execute 522 sec latency 31.313 ms 2024-04-24T23:47:25.155 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1772604 15.03 MB/sec execute 523 sec latency 21.673 ms 2024-04-24T23:47:26.155 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1775270 15.02 MB/sec execute 524 sec latency 36.512 ms 2024-04-24T23:47:27.155 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1777717 15.02 MB/sec execute 525 sec latency 30.701 ms 2024-04-24T23:47:28.155 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1780213 15.02 MB/sec execute 526 sec latency 14.723 ms 2024-04-24T23:47:29.155 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1783400 15.02 MB/sec execute 527 sec latency 28.956 ms 2024-04-24T23:47:30.155 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1786516 15.03 MB/sec execute 528 sec latency 38.039 ms 2024-04-24T23:47:31.155 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1789004 15.02 MB/sec execute 529 sec latency 38.050 ms 2024-04-24T23:47:32.155 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1791260 15.01 MB/sec execute 530 sec latency 35.281 ms 2024-04-24T23:47:33.155 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1794209 15.02 MB/sec execute 531 sec latency 32.037 ms 2024-04-24T23:47:34.155 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1797305 15.02 MB/sec execute 532 sec latency 17.311 ms 2024-04-24T23:47:35.155 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1799365 15.01 MB/sec execute 533 sec latency 101.239 ms 2024-04-24T23:47:36.155 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1799665 14.98 MB/sec execute 534 sec latency 642.979 ms 2024-04-24T23:47:37.155 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1800698 14.97 MB/sec execute 535 sec latency 113.782 ms 2024-04-24T23:47:38.155 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1803142 14.96 MB/sec execute 536 sec latency 28.187 ms 2024-04-24T23:47:39.155 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1805707 14.97 MB/sec execute 537 sec latency 47.951 ms 2024-04-24T23:47:40.155 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1808681 14.97 MB/sec execute 538 sec latency 19.196 ms 2024-04-24T23:47:41.155 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1811964 14.97 MB/sec execute 539 sec latency 21.760 ms 2024-04-24T23:47:42.155 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1815209 14.98 MB/sec execute 540 sec latency 18.816 ms 2024-04-24T23:47:43.155 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1818527 14.99 MB/sec execute 541 sec latency 14.684 ms 2024-04-24T23:47:44.155 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1821631 14.99 MB/sec execute 542 sec latency 25.334 ms 2024-04-24T23:47:45.155 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1824416 14.98 MB/sec execute 543 sec latency 37.185 ms 2024-04-24T23:47:46.155 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1827222 14.99 MB/sec execute 544 sec latency 36.228 ms 2024-04-24T23:47:47.155 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1830129 14.99 MB/sec execute 545 sec latency 27.997 ms 2024-04-24T23:47:48.155 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1833306 14.99 MB/sec execute 546 sec latency 30.885 ms 2024-04-24T23:47:49.155 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1836252 15.02 MB/sec execute 547 sec latency 36.438 ms 2024-04-24T23:47:50.155 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1839303 15.02 MB/sec execute 548 sec latency 11.753 ms 2024-04-24T23:47:51.155 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1839955 15.01 MB/sec execute 549 sec latency 373.426 ms 2024-04-24T23:47:52.155 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1841364 14.99 MB/sec execute 550 sec latency 48.027 ms 2024-04-24T23:47:53.155 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1844267 15.00 MB/sec execute 551 sec latency 46.995 ms 2024-04-24T23:47:54.155 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1846507 14.98 MB/sec execute 552 sec latency 56.715 ms 2024-04-24T23:47:55.155 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1848759 14.98 MB/sec execute 553 sec latency 33.708 ms 2024-04-24T23:47:56.156 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1851836 14.99 MB/sec execute 554 sec latency 21.908 ms 2024-04-24T23:47:57.156 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1854997 14.99 MB/sec execute 555 sec latency 27.974 ms 2024-04-24T23:47:58.155 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1857888 14.99 MB/sec execute 556 sec latency 22.612 ms 2024-04-24T23:47:59.156 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1860392 14.98 MB/sec execute 557 sec latency 38.975 ms 2024-04-24T23:48:00.156 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1862615 14.98 MB/sec execute 558 sec latency 25.976 ms 2024-04-24T23:48:01.156 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1865708 14.99 MB/sec execute 559 sec latency 28.844 ms 2024-04-24T23:48:02.156 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1868507 14.99 MB/sec execute 560 sec latency 26.484 ms 2024-04-24T23:48:03.156 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1870727 14.98 MB/sec execute 561 sec latency 41.800 ms 2024-04-24T23:48:04.156 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1873339 14.98 MB/sec execute 562 sec latency 21.067 ms 2024-04-24T23:48:05.156 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1876402 14.98 MB/sec execute 563 sec latency 27.951 ms 2024-04-24T23:48:06.156 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1879526 14.99 MB/sec execute 564 sec latency 30.876 ms 2024-04-24T23:48:07.156 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1882080 14.98 MB/sec execute 565 sec latency 24.252 ms 2024-04-24T23:48:07.308 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:48:07.311 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:48:08.156 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1884491 14.98 MB/sec execute 566 sec latency 32.492 ms 2024-04-24T23:48:09.156 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1887435 14.98 MB/sec execute 567 sec latency 38.702 ms 2024-04-24T23:48:10.156 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1890590 14.99 MB/sec execute 568 sec latency 32.101 ms 2024-04-24T23:48:11.156 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1893817 14.99 MB/sec execute 569 sec latency 24.309 ms 2024-04-24T23:48:12.156 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1896624 14.99 MB/sec execute 570 sec latency 30.535 ms 2024-04-24T23:48:13.156 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1898886 14.98 MB/sec execute 571 sec latency 41.190 ms 2024-04-24T23:48:14.156 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1901597 14.98 MB/sec execute 572 sec latency 35.189 ms 2024-04-24T23:48:15.156 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1904665 14.99 MB/sec execute 573 sec latency 28.257 ms 2024-04-24T23:48:16.156 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1907545 14.99 MB/sec execute 574 sec latency 35.514 ms 2024-04-24T23:48:17.156 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1909625 14.98 MB/sec execute 575 sec latency 37.642 ms 2024-04-24T23:48:18.156 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1912148 14.98 MB/sec execute 576 sec latency 56.482 ms 2024-04-24T23:48:19.156 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1915382 14.99 MB/sec execute 577 sec latency 37.448 ms 2024-04-24T23:48:20.156 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1918628 14.99 MB/sec execute 578 sec latency 23.642 ms 2024-04-24T23:48:21.156 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1921020 14.98 MB/sec execute 579 sec latency 59.914 ms 2024-04-24T23:48:22.157 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1921037 14.95 MB/sec execute 580 sec latency 894.064 ms 2024-04-24T23:48:23.157 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1922858 14.95 MB/sec execute 581 sec latency 168.711 ms 2024-04-24T23:48:24.158 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1923590 14.93 MB/sec execute 582 sec latency 44.688 ms 2024-04-24T23:48:25.158 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1926031 14.93 MB/sec execute 583 sec latency 68.208 ms 2024-04-24T23:48:26.158 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1928994 14.93 MB/sec execute 584 sec latency 25.179 ms 2024-04-24T23:48:27.158 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1931082 14.92 MB/sec execute 585 sec latency 45.397 ms 2024-04-24T23:48:28.158 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1933538 14.92 MB/sec execute 586 sec latency 22.580 ms 2024-04-24T23:48:29.158 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1936644 14.93 MB/sec execute 587 sec latency 21.298 ms 2024-04-24T23:48:30.158 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1939460 14.92 MB/sec execute 588 sec latency 25.954 ms 2024-04-24T23:48:31.158 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1941475 14.92 MB/sec execute 589 sec latency 26.015 ms 2024-04-24T23:48:32.158 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1944166 14.91 MB/sec execute 590 sec latency 43.305 ms 2024-04-24T23:48:33.158 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1947527 14.92 MB/sec execute 591 sec latency 25.564 ms 2024-04-24T23:48:34.158 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1950834 14.93 MB/sec execute 592 sec latency 39.638 ms 2024-04-24T23:48:35.158 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1954143 14.93 MB/sec execute 593 sec latency 16.199 ms 2024-04-24T23:48:36.158 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1957367 14.93 MB/sec execute 594 sec latency 20.652 ms 2024-04-24T23:48:37.163 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1958280 14.92 MB/sec execute 595 sec latency 191.709 ms 2024-04-24T23:48:37.405 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:48:37.408 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:48:38.161 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1959455 14.90 MB/sec execute 596 sec latency 51.143 ms 2024-04-24T23:48:39.161 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1961911 14.90 MB/sec execute 597 sec latency 31.699 ms 2024-04-24T23:48:40.161 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1965020 14.91 MB/sec execute 598 sec latency 32.952 ms 2024-04-24T23:48:41.161 INFO:tasks.workunit.client.1.smithi169.stdout: 1 1967662 14.90 MB/sec execute 599 sec latency 24.855 ms 2024-04-24T23:48:42.161 INFO:tasks.workunit.client.1.smithi169.stdout: 1 cleanup 600 sec 2024-04-24T23:48:42.269 INFO:tasks.workunit.client.1.smithi169.stdout: 0 cleanup 600 sec 2024-04-24T23:48:42.269 INFO:tasks.workunit.client.1.smithi169.stdout: 2024-04-24T23:48:42.269 INFO:tasks.workunit.client.1.smithi169.stdout: Operation Count AvgLat MaxLat 2024-04-24T23:48:42.269 INFO:tasks.workunit.client.1.smithi169.stdout: ---------------------------------------- 2024-04-24T23:48:42.269 INFO:tasks.workunit.client.1.smithi169.stdout: NTCreateX 284423 0.716 945.839 2024-04-24T23:48:42.269 INFO:tasks.workunit.client.1.smithi169.stdout: Close 208952 0.002 0.061 2024-04-24T23:48:42.269 INFO:tasks.workunit.client.1.smithi169.stdout: Rename 12043 3.463 216.783 2024-04-24T23:48:42.270 INFO:tasks.workunit.client.1.smithi169.stdout: Unlink 57429 1.198 152.963 2024-04-24T23:48:42.270 INFO:tasks.workunit.client.1.smithi169.stdout: Deltree 8 61.063 107.268 2024-04-24T23:48:42.270 INFO:tasks.workunit.client.1.smithi169.stdout: Mkdir 4 0.008 0.009 2024-04-24T23:48:42.270 INFO:tasks.workunit.client.1.smithi169.stdout: Qpathinfo 257772 0.560 642.969 2024-04-24T23:48:42.270 INFO:tasks.workunit.client.1.smithi169.stdout: Qfileinfo 45202 0.001 0.021 2024-04-24T23:48:42.270 INFO:tasks.workunit.client.1.smithi169.stdout: Qfsinfo 47266 0.133 104.198 2024-04-24T23:48:42.270 INFO:tasks.workunit.client.1.smithi169.stdout: Sfileinfo 23166 0.014 4.336 2024-04-24T23:48:42.270 INFO:tasks.workunit.client.1.smithi169.stdout: Find 99667 0.059 17.958 2024-04-24T23:48:42.270 INFO:tasks.workunit.client.1.smithi169.stdout: WriteX 141955 0.017 373.284 2024-04-24T23:48:42.270 INFO:tasks.workunit.client.1.smithi169.stdout: ReadX 445793 0.019 411.819 2024-04-24T23:48:42.270 INFO:tasks.workunit.client.1.smithi169.stdout: LockX 926 0.967 219.769 2024-04-24T23:48:42.270 INFO:tasks.workunit.client.1.smithi169.stdout: UnlockX 926 0.775 168.664 2024-04-24T23:48:42.270 INFO:tasks.workunit.client.1.smithi169.stdout: Flush 19934 5.660 1152.931 2024-04-24T23:48:42.270 INFO:tasks.workunit.client.1.smithi169.stdout: 2024-04-24T23:48:42.271 INFO:tasks.workunit.client.1.smithi169.stdout:Throughput 14.9025 MB/sec 1 clients 1 procs max_latency=1152.938 ms 2024-04-24T23:48:42.274 INFO:tasks.workunit.client.1.smithi169.stderr:+ dbench 10 2024-04-24T23:48:42.278 INFO:tasks.workunit.client.1.smithi169.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-04-24T23:48:42.278 INFO:tasks.workunit.client.1.smithi169.stdout: 2024-04-24T23:48:42.278 INFO:tasks.workunit.client.1.smithi169.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-04-24T23:48:42.278 INFO:tasks.workunit.client.1.smithi169.stdout:9 of 10 processes prepared for launch 0 sec 2024-04-24T23:48:42.379 INFO:tasks.workunit.client.1.smithi169.stdout:10 of 10 processes prepared for launch 0 sec 2024-04-24T23:48:42.379 INFO:tasks.workunit.client.1.smithi169.stdout:releasing clients 2024-04-24T23:48:43.379 INFO:tasks.workunit.client.1.smithi169.stdout: 10 495 171.50 MB/sec warmup 1 sec latency 27.648 ms 2024-04-24T23:48:44.379 INFO:tasks.workunit.client.1.smithi169.stdout: 10 992 110.41 MB/sec warmup 2 sec latency 55.206 ms 2024-04-24T23:48:45.379 INFO:tasks.workunit.client.1.smithi169.stdout: 10 1772 87.65 MB/sec warmup 3 sec latency 39.840 ms 2024-04-24T23:48:46.379 INFO:tasks.workunit.client.1.smithi169.stdout: 10 2613 73.22 MB/sec warmup 4 sec latency 34.132 ms 2024-04-24T23:48:47.379 INFO:tasks.workunit.client.1.smithi169.stdout: 10 3396 71.52 MB/sec warmup 5 sec latency 59.587 ms 2024-04-24T23:48:48.379 INFO:tasks.workunit.client.1.smithi169.stdout: 10 3839 66.27 MB/sec warmup 6 sec latency 62.329 ms 2024-04-24T23:48:49.379 INFO:tasks.workunit.client.1.smithi169.stdout: 10 4069 57.25 MB/sec warmup 7 sec latency 115.930 ms 2024-04-24T23:48:50.379 INFO:tasks.workunit.client.1.smithi169.stdout: 10 4300 50.50 MB/sec warmup 8 sec latency 138.570 ms 2024-04-24T23:48:51.379 INFO:tasks.workunit.client.1.smithi169.stdout: 10 4748 46.83 MB/sec warmup 9 sec latency 98.395 ms 2024-04-24T23:48:52.379 INFO:tasks.workunit.client.1.smithi169.stdout: 10 5621 45.88 MB/sec warmup 10 sec latency 44.888 ms 2024-04-24T23:48:53.379 INFO:tasks.workunit.client.1.smithi169.stdout: 10 6381 46.53 MB/sec warmup 11 sec latency 69.282 ms 2024-04-24T23:48:54.379 INFO:tasks.workunit.client.1.smithi169.stdout: 10 7052 47.22 MB/sec warmup 12 sec latency 91.569 ms 2024-04-24T23:48:55.380 INFO:tasks.workunit.client.1.smithi169.stdout: 10 7405 45.32 MB/sec warmup 13 sec latency 86.290 ms 2024-04-24T23:48:56.379 INFO:tasks.workunit.client.1.smithi169.stdout: 10 7622 42.29 MB/sec warmup 14 sec latency 73.238 ms 2024-04-24T23:48:57.379 INFO:tasks.workunit.client.1.smithi169.stdout: 10 7835 39.69 MB/sec warmup 15 sec latency 95.597 ms 2024-04-24T23:48:58.379 INFO:tasks.workunit.client.1.smithi169.stdout: 10 8017 37.48 MB/sec warmup 16 sec latency 433.918 ms 2024-04-24T23:48:59.379 INFO:tasks.workunit.client.1.smithi169.stdout: 10 8252 36.02 MB/sec warmup 17 sec latency 779.242 ms 2024-04-24T23:49:00.379 INFO:tasks.workunit.client.1.smithi169.stdout: 10 8766 35.80 MB/sec warmup 18 sec latency 75.755 ms 2024-04-24T23:49:01.379 INFO:tasks.workunit.client.1.smithi169.stdout: 10 9570 35.05 MB/sec warmup 19 sec latency 40.547 ms 2024-04-24T23:49:02.379 INFO:tasks.workunit.client.1.smithi169.stdout: 10 10329 36.37 MB/sec warmup 20 sec latency 31.258 ms 2024-04-24T23:49:03.379 INFO:tasks.workunit.client.1.smithi169.stdout: 10 10865 37.00 MB/sec warmup 21 sec latency 54.055 ms 2024-04-24T23:49:04.380 INFO:tasks.workunit.client.1.smithi169.stdout: 10 11070 35.59 MB/sec warmup 22 sec latency 75.571 ms 2024-04-24T23:49:05.380 INFO:tasks.workunit.client.1.smithi169.stdout: 10 11293 34.11 MB/sec warmup 23 sec latency 87.997 ms 2024-04-24T23:49:06.380 INFO:tasks.workunit.client.1.smithi169.stdout: 10 11599 33.07 MB/sec warmup 24 sec latency 163.510 ms 2024-04-24T23:49:07.379 INFO:tasks.workunit.client.1.smithi169.stdout: 10 12214 33.39 MB/sec warmup 25 sec latency 77.605 ms 2024-04-24T23:49:07.486 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:49:07.488 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:49:08.380 INFO:tasks.workunit.client.1.smithi169.stdout: 10 13016 32.78 MB/sec warmup 26 sec latency 122.538 ms 2024-04-24T23:49:09.380 INFO:tasks.workunit.client.1.smithi169.stdout: 10 13671 33.65 MB/sec warmup 27 sec latency 85.190 ms 2024-04-24T23:49:10.380 INFO:tasks.workunit.client.1.smithi169.stdout: 10 14274 34.28 MB/sec warmup 28 sec latency 58.604 ms 2024-04-24T23:49:11.380 INFO:tasks.workunit.client.1.smithi169.stdout: 10 14530 33.54 MB/sec warmup 29 sec latency 70.204 ms 2024-04-24T23:49:12.380 INFO:tasks.workunit.client.1.smithi169.stdout: 10 14719 32.48 MB/sec warmup 30 sec latency 80.398 ms 2024-04-24T23:49:13.380 INFO:tasks.workunit.client.1.smithi169.stdout: 10 14915 31.53 MB/sec warmup 31 sec latency 126.429 ms 2024-04-24T23:49:14.380 INFO:tasks.workunit.client.1.smithi169.stdout: 10 14950 30.56 MB/sec warmup 32 sec latency 1076.512 ms 2024-04-24T23:49:15.380 INFO:tasks.workunit.client.1.smithi169.stdout: 10 14963 29.64 MB/sec warmup 33 sec latency 1644.884 ms 2024-04-24T23:49:16.380 INFO:tasks.workunit.client.1.smithi169.stdout: 10 15051 28.83 MB/sec warmup 34 sec latency 1877.111 ms 2024-04-24T23:49:17.380 INFO:tasks.workunit.client.1.smithi169.stdout: 10 15322 28.38 MB/sec warmup 35 sec latency 87.636 ms 2024-04-24T23:49:18.380 INFO:tasks.workunit.client.1.smithi169.stdout: 10 15626 28.16 MB/sec warmup 36 sec latency 153.200 ms 2024-04-24T23:49:19.380 INFO:tasks.workunit.client.1.smithi169.stdout: 10 16096 27.87 MB/sec warmup 37 sec latency 98.596 ms 2024-04-24T23:49:20.380 INFO:tasks.workunit.client.1.smithi169.stdout: 10 16779 27.96 MB/sec warmup 38 sec latency 111.658 ms 2024-04-24T23:49:21.381 INFO:tasks.workunit.client.1.smithi169.stdout: 10 17516 28.89 MB/sec warmup 39 sec latency 43.415 ms 2024-04-24T23:49:22.380 INFO:tasks.workunit.client.1.smithi169.stdout: 10 17950 28.96 MB/sec warmup 40 sec latency 73.124 ms 2024-04-24T23:49:23.380 INFO:tasks.workunit.client.1.smithi169.stdout: 10 18161 28.43 MB/sec warmup 41 sec latency 80.006 ms 2024-04-24T23:49:24.380 INFO:tasks.workunit.client.1.smithi169.stdout: 10 18340 27.78 MB/sec warmup 42 sec latency 112.749 ms 2024-04-24T23:49:25.382 INFO:tasks.workunit.client.1.smithi169.stdout: 10 18611 27.29 MB/sec warmup 43 sec latency 129.942 ms 2024-04-24T23:49:26.380 INFO:tasks.workunit.client.1.smithi169.stdout: 10 19151 27.38 MB/sec warmup 44 sec latency 69.623 ms 2024-04-24T23:49:27.380 INFO:tasks.workunit.client.1.smithi169.stdout: 10 20039 27.40 MB/sec warmup 45 sec latency 52.337 ms 2024-04-24T23:49:28.380 INFO:tasks.workunit.client.1.smithi169.stdout: 10 20663 27.83 MB/sec warmup 46 sec latency 113.927 ms 2024-04-24T23:49:29.380 INFO:tasks.workunit.client.1.smithi169.stdout: 10 21294 28.31 MB/sec warmup 47 sec latency 65.963 ms 2024-04-24T23:49:30.380 INFO:tasks.workunit.client.1.smithi169.stdout: 10 21609 28.20 MB/sec warmup 48 sec latency 77.120 ms 2024-04-24T23:49:31.380 INFO:tasks.workunit.client.1.smithi169.stdout: 10 21809 27.69 MB/sec warmup 49 sec latency 81.119 ms 2024-04-24T23:49:32.381 INFO:tasks.workunit.client.1.smithi169.stdout: 10 22026 27.21 MB/sec warmup 50 sec latency 156.866 ms 2024-04-24T23:49:33.381 INFO:tasks.workunit.client.1.smithi169.stdout: 10 22426 26.98 MB/sec warmup 51 sec latency 93.304 ms 2024-04-24T23:49:34.383 INFO:tasks.workunit.client.1.smithi169.stdout: 10 22670 26.80 MB/sec warmup 52 sec latency 586.230 ms 2024-04-24T23:49:35.383 INFO:tasks.workunit.client.1.smithi169.stdout: 10 22766 26.36 MB/sec warmup 53 sec latency 805.365 ms 2024-04-24T23:49:36.383 INFO:tasks.workunit.client.1.smithi169.stdout: 10 23254 26.14 MB/sec warmup 54 sec latency 100.450 ms 2024-04-24T23:49:37.383 INFO:tasks.workunit.client.1.smithi169.stdout: 10 23548 25.89 MB/sec warmup 55 sec latency 115.360 ms 2024-04-24T23:49:37.859 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:49:37.861 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:49:37.895 INFO:tasks.ceph.mds.b.smithi169.stderr:2024-04-24T23:49:37.894+0000 7f12ddd9e640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 39670) UID: 0 2024-04-24T23:49:37.896 INFO:tasks.ceph.mds.d.smithi169.stderr:2024-04-24T23:49:37.894+0000 7fcbfaa77640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 39670) UID: 0 2024-04-24T23:49:38.383 INFO:tasks.workunit.client.1.smithi169.stdout: 10 24120 26.16 MB/sec warmup 56 sec latency 59.620 ms 2024-04-24T23:49:39.383 INFO:tasks.workunit.client.1.smithi169.stdout: 10 24768 26.67 MB/sec warmup 57 sec latency 37.175 ms 2024-04-24T23:49:40.383 INFO:tasks.workunit.client.1.smithi169.stdout: 10 25083 26.60 MB/sec warmup 58 sec latency 66.593 ms 2024-04-24T23:49:41.383 INFO:tasks.workunit.client.1.smithi169.stdout: 10 25290 26.23 MB/sec warmup 59 sec latency 91.384 ms 2024-04-24T23:49:42.383 INFO:tasks.workunit.client.1.smithi169.stdout: 10 25468 25.82 MB/sec warmup 60 sec latency 141.596 ms 2024-04-24T23:49:43.383 INFO:tasks.workunit.client.1.smithi169.stdout: 10 25706 25.48 MB/sec warmup 61 sec latency 172.047 ms 2024-04-24T23:49:44.384 INFO:tasks.workunit.client.1.smithi169.stdout: 10 26174 25.55 MB/sec warmup 62 sec latency 63.886 ms 2024-04-24T23:49:45.383 INFO:tasks.workunit.client.1.smithi169.stdout: 10 26906 25.46 MB/sec warmup 63 sec latency 80.727 ms 2024-04-24T23:49:46.384 INFO:tasks.workunit.client.1.smithi169.stdout: 10 27535 25.77 MB/sec warmup 64 sec latency 43.019 ms 2024-04-24T23:49:47.384 INFO:tasks.workunit.client.1.smithi169.stdout: 10 28196 26.13 MB/sec warmup 65 sec latency 38.488 ms 2024-04-24T23:49:48.383 INFO:tasks.workunit.client.1.smithi169.stdout: 10 28517 26.10 MB/sec warmup 66 sec latency 112.577 ms 2024-04-24T23:49:49.383 INFO:tasks.workunit.client.1.smithi169.stdout: 10 28806 25.92 MB/sec warmup 67 sec latency 91.185 ms 2024-04-24T23:49:50.384 INFO:tasks.workunit.client.1.smithi169.stdout: 10 28980 25.60 MB/sec warmup 68 sec latency 242.263 ms 2024-04-24T23:49:51.384 INFO:tasks.workunit.client.1.smithi169.stdout: 10 29272 25.36 MB/sec warmup 69 sec latency 102.812 ms 2024-04-24T23:49:52.384 INFO:tasks.workunit.client.1.smithi169.stdout: 10 29743 25.32 MB/sec warmup 70 sec latency 73.261 ms 2024-04-24T23:49:53.384 INFO:tasks.workunit.client.1.smithi169.stdout: 10 30442 25.29 MB/sec warmup 71 sec latency 70.630 ms 2024-04-24T23:49:54.384 INFO:tasks.workunit.client.1.smithi169.stdout: 10 31177 25.64 MB/sec warmup 72 sec latency 35.048 ms 2024-04-24T23:49:55.384 INFO:tasks.workunit.client.1.smithi169.stdout: 10 31820 25.96 MB/sec warmup 73 sec latency 50.009 ms 2024-04-24T23:49:56.384 INFO:tasks.workunit.client.1.smithi169.stdout: 10 32130 25.91 MB/sec warmup 74 sec latency 81.624 ms 2024-04-24T23:49:57.384 INFO:tasks.workunit.client.1.smithi169.stdout: 10 32390 25.71 MB/sec warmup 75 sec latency 114.683 ms 2024-04-24T23:49:58.384 INFO:tasks.workunit.client.1.smithi169.stdout: 10 32576 25.43 MB/sec warmup 76 sec latency 123.767 ms 2024-04-24T23:49:59.384 INFO:tasks.workunit.client.1.smithi169.stdout: 10 32911 25.24 MB/sec warmup 77 sec latency 102.150 ms 2024-04-24T23:50:00.384 INFO:tasks.workunit.client.1.smithi169.stdout: 10 33456 25.30 MB/sec warmup 78 sec latency 50.575 ms 2024-04-24T23:50:01.384 INFO:tasks.workunit.client.1.smithi169.stdout: 10 34115 25.19 MB/sec warmup 79 sec latency 87.928 ms 2024-04-24T23:50:02.384 INFO:tasks.workunit.client.1.smithi169.stdout: 10 34736 25.52 MB/sec warmup 80 sec latency 38.493 ms 2024-04-24T23:50:03.384 INFO:tasks.workunit.client.1.smithi169.stdout: 10 35331 25.70 MB/sec warmup 81 sec latency 40.147 ms 2024-04-24T23:50:04.384 INFO:tasks.workunit.client.1.smithi169.stdout: 10 35619 25.67 MB/sec warmup 82 sec latency 110.351 ms 2024-04-24T23:50:05.384 INFO:tasks.workunit.client.1.smithi169.stdout: 10 35877 25.53 MB/sec warmup 83 sec latency 98.278 ms 2024-04-24T23:50:06.384 INFO:tasks.workunit.client.1.smithi169.stdout: 10 36064 25.27 MB/sec warmup 84 sec latency 127.272 ms 2024-04-24T23:50:07.384 INFO:tasks.workunit.client.1.smithi169.stdout: 10 36319 25.05 MB/sec warmup 85 sec latency 115.242 ms 2024-04-24T23:50:08.384 INFO:tasks.workunit.client.1.smithi169.stdout: 10 36783 25.08 MB/sec warmup 86 sec latency 59.295 ms 2024-04-24T23:50:09.386 INFO:tasks.workunit.client.1.smithi169.stdout: 10 37448 25.00 MB/sec warmup 87 sec latency 82.422 ms 2024-04-24T23:50:10.384 INFO:tasks.workunit.client.1.smithi169.stdout: 10 37653 24.81 MB/sec warmup 88 sec latency 723.419 ms 2024-04-24T23:50:11.384 INFO:tasks.workunit.client.1.smithi169.stdout: 10 37662 24.54 MB/sec warmup 89 sec latency 1639.347 ms 2024-04-24T23:50:12.385 INFO:tasks.workunit.client.1.smithi169.stdout: 10 37670 24.26 MB/sec warmup 90 sec latency 2320.440 ms 2024-04-24T23:50:13.385 INFO:tasks.workunit.client.1.smithi169.stdout: 10 37757 24.08 MB/sec warmup 91 sec latency 2704.486 ms 2024-04-24T23:50:14.385 INFO:tasks.workunit.client.1.smithi169.stdout: 10 38031 24.08 MB/sec warmup 92 sec latency 103.960 ms 2024-04-24T23:50:15.386 INFO:tasks.workunit.client.1.smithi169.stdout: 10 38350 24.03 MB/sec warmup 93 sec latency 119.694 ms 2024-04-24T23:50:16.385 INFO:tasks.workunit.client.1.smithi169.stdout: 10 38663 24.00 MB/sec warmup 94 sec latency 118.561 ms 2024-04-24T23:50:17.386 INFO:tasks.workunit.client.1.smithi169.stdout: 10 39065 24.12 MB/sec warmup 95 sec latency 83.879 ms 2024-04-24T23:50:18.385 INFO:tasks.workunit.client.1.smithi169.stdout: 10 39300 23.99 MB/sec warmup 96 sec latency 102.328 ms 2024-04-24T23:50:19.386 INFO:tasks.workunit.client.1.smithi169.stdout: 10 39533 23.85 MB/sec warmup 97 sec latency 168.339 ms 2024-04-24T23:50:20.386 INFO:tasks.workunit.client.1.smithi169.stdout: 10 39766 23.66 MB/sec warmup 98 sec latency 169.924 ms 2024-04-24T23:50:21.387 INFO:tasks.workunit.client.1.smithi169.stdout: 10 40121 23.60 MB/sec warmup 99 sec latency 108.882 ms 2024-04-24T23:50:22.386 INFO:tasks.workunit.client.1.smithi169.stdout: 10 40659 23.57 MB/sec warmup 100 sec latency 101.958 ms 2024-04-24T23:50:23.386 INFO:tasks.workunit.client.1.smithi169.stdout: 10 41384 23.68 MB/sec warmup 101 sec latency 56.273 ms 2024-04-24T23:50:24.386 INFO:tasks.workunit.client.1.smithi169.stdout: 10 42145 23.96 MB/sec warmup 102 sec latency 44.237 ms 2024-04-24T23:50:25.386 INFO:tasks.workunit.client.1.smithi169.stdout: 10 42632 24.14 MB/sec warmup 103 sec latency 77.991 ms 2024-04-24T23:50:26.386 INFO:tasks.workunit.client.1.smithi169.stdout: 10 42936 24.04 MB/sec warmup 104 sec latency 91.334 ms 2024-04-24T23:50:27.386 INFO:tasks.workunit.client.1.smithi169.stdout: 10 43125 23.85 MB/sec warmup 105 sec latency 134.855 ms 2024-04-24T23:50:28.386 INFO:tasks.workunit.client.1.smithi169.stdout: 10 43368 23.69 MB/sec warmup 106 sec latency 139.261 ms 2024-04-24T23:50:29.386 INFO:tasks.workunit.client.1.smithi169.stdout: 10 43792 23.69 MB/sec warmup 107 sec latency 84.621 ms 2024-04-24T23:50:30.386 INFO:tasks.workunit.client.1.smithi169.stdout: 10 44200 23.58 MB/sec warmup 108 sec latency 282.352 ms 2024-04-24T23:50:31.388 INFO:tasks.workunit.client.1.smithi169.stdout: 10 44246 23.38 MB/sec warmup 109 sec latency 1111.168 ms 2024-04-24T23:50:32.386 INFO:tasks.workunit.client.1.smithi169.stdout: 10 44547 23.27 MB/sec warmup 110 sec latency 577.941 ms 2024-04-24T23:50:33.387 INFO:tasks.workunit.client.1.smithi169.stdout: 10 44726 23.11 MB/sec warmup 111 sec latency 158.898 ms 2024-04-24T23:50:34.386 INFO:tasks.workunit.client.1.smithi169.stdout: 10 45335 23.29 MB/sec warmup 112 sec latency 83.048 ms 2024-04-24T23:50:35.386 INFO:tasks.workunit.client.1.smithi169.stdout: 10 45933 23.50 MB/sec warmup 113 sec latency 90.346 ms 2024-04-24T23:50:36.386 INFO:tasks.workunit.client.1.smithi169.stdout: 10 46279 23.51 MB/sec warmup 114 sec latency 100.076 ms 2024-04-24T23:50:37.386 INFO:tasks.workunit.client.1.smithi169.stdout: 10 46528 23.41 MB/sec warmup 115 sec latency 134.711 ms 2024-04-24T23:50:38.386 INFO:tasks.workunit.client.1.smithi169.stdout: 10 46739 23.25 MB/sec warmup 116 sec latency 202.320 ms 2024-04-24T23:50:39.386 INFO:tasks.workunit.client.1.smithi169.stdout: 10 47004 23.11 MB/sec warmup 117 sec latency 222.884 ms 2024-04-24T23:50:40.386 INFO:tasks.workunit.client.1.smithi169.stdout: 10 47452 23.16 MB/sec warmup 118 sec latency 112.963 ms 2024-04-24T23:50:41.386 INFO:tasks.workunit.client.1.smithi169.stdout: 10 48108 23.09 MB/sec warmup 119 sec latency 47.211 ms 2024-04-24T23:50:43.387 INFO:tasks.workunit.client.1.smithi169.stdout: 10 49303 35.84 MB/sec execute 1 sec latency 85.969 ms 2024-04-24T23:50:44.386 INFO:tasks.workunit.client.1.smithi169.stdout: 10 49694 34.14 MB/sec execute 2 sec latency 81.353 ms 2024-04-24T23:50:45.388 INFO:tasks.workunit.client.1.smithi169.stdout: 10 49961 26.98 MB/sec execute 3 sec latency 115.550 ms 2024-04-24T23:50:46.388 INFO:tasks.workunit.client.1.smithi169.stdout: 10 50022 21.01 MB/sec execute 4 sec latency 926.477 ms 2024-04-24T23:50:47.388 INFO:tasks.workunit.client.1.smithi169.stdout: 10 50106 17.80 MB/sec execute 5 sec latency 932.001 ms 2024-04-24T23:50:48.388 INFO:tasks.workunit.client.1.smithi169.stdout: 10 50202 15.01 MB/sec execute 6 sec latency 272.826 ms 2024-04-24T23:50:49.388 INFO:tasks.workunit.client.1.smithi169.stdout: 10 50376 13.58 MB/sec execute 7 sec latency 206.303 ms 2024-04-24T23:50:50.389 INFO:tasks.workunit.client.1.smithi169.stdout: 10 50825 15.18 MB/sec execute 8 sec latency 71.843 ms 2024-04-24T23:50:51.389 INFO:tasks.workunit.client.1.smithi169.stdout: 10 51284 14.82 MB/sec execute 9 sec latency 94.556 ms 2024-04-24T23:50:52.389 INFO:tasks.workunit.client.1.smithi169.stdout: 10 51825 15.86 MB/sec execute 10 sec latency 147.887 ms 2024-04-24T23:50:53.389 INFO:tasks.workunit.client.1.smithi169.stdout: 10 52516 18.94 MB/sec execute 11 sec latency 35.940 ms 2024-04-24T23:50:54.389 INFO:tasks.workunit.client.1.smithi169.stdout: 10 53032 20.25 MB/sec execute 12 sec latency 86.691 ms 2024-04-24T23:50:55.389 INFO:tasks.workunit.client.1.smithi169.stdout: 10 53355 20.33 MB/sec execute 13 sec latency 91.021 ms 2024-04-24T23:50:56.389 INFO:tasks.workunit.client.1.smithi169.stdout: 10 53669 19.97 MB/sec execute 14 sec latency 86.869 ms 2024-04-24T23:50:57.389 INFO:tasks.workunit.client.1.smithi169.stdout: 10 53902 19.05 MB/sec execute 15 sec latency 114.147 ms 2024-04-24T23:50:58.389 INFO:tasks.workunit.client.1.smithi169.stdout: 10 54300 18.97 MB/sec execute 16 sec latency 76.377 ms 2024-04-24T23:50:59.389 INFO:tasks.workunit.client.1.smithi169.stdout: 10 54731 18.93 MB/sec execute 17 sec latency 144.849 ms 2024-04-24T23:51:00.389 INFO:tasks.workunit.client.1.smithi169.stdout: 10 55358 19.13 MB/sec execute 18 sec latency 68.315 ms 2024-04-24T23:51:01.389 INFO:tasks.workunit.client.1.smithi169.stdout: 10 55965 20.57 MB/sec execute 19 sec latency 115.276 ms 2024-04-24T23:51:02.389 INFO:tasks.workunit.client.1.smithi169.stdout: 10 56515 21.45 MB/sec execute 20 sec latency 88.553 ms 2024-04-24T23:51:03.389 INFO:tasks.workunit.client.1.smithi169.stdout: 10 56910 21.71 MB/sec execute 21 sec latency 71.506 ms 2024-04-24T23:51:04.390 INFO:tasks.workunit.client.1.smithi169.stdout: 10 57187 21.36 MB/sec execute 22 sec latency 121.556 ms 2024-04-24T23:51:05.390 INFO:tasks.workunit.client.1.smithi169.stdout: 10 57396 20.69 MB/sec execute 23 sec latency 143.736 ms 2024-04-24T23:51:06.389 INFO:tasks.workunit.client.1.smithi169.stdout: 10 57827 20.76 MB/sec execute 24 sec latency 59.761 ms 2024-04-24T23:51:07.389 INFO:tasks.workunit.client.1.smithi169.stdout: 10 58327 20.54 MB/sec execute 25 sec latency 102.400 ms 2024-04-24T23:51:08.389 INFO:tasks.workunit.client.1.smithi169.stdout: 10 58914 20.73 MB/sec execute 26 sec latency 77.576 ms 2024-04-24T23:51:09.389 INFO:tasks.workunit.client.1.smithi169.stdout: 10 59648 21.92 MB/sec execute 27 sec latency 45.090 ms 2024-04-24T23:51:10.389 INFO:tasks.workunit.client.1.smithi169.stdout: 10 60146 22.43 MB/sec execute 28 sec latency 96.786 ms 2024-04-24T23:51:11.390 INFO:tasks.workunit.client.1.smithi169.stdout: 10 60393 22.23 MB/sec execute 29 sec latency 337.004 ms 2024-04-24T23:51:12.390 INFO:tasks.workunit.client.1.smithi169.stdout: 10 60399 21.51 MB/sec execute 30 sec latency 1279.507 ms 2024-04-24T23:51:13.389 INFO:tasks.workunit.client.1.smithi169.stdout: 10 60401 20.81 MB/sec execute 31 sec latency 930.506 ms 2024-04-24T23:51:14.390 INFO:tasks.workunit.client.1.smithi169.stdout: 10 60482 20.26 MB/sec execute 32 sec latency 922.631 ms 2024-04-24T23:51:15.389 INFO:tasks.workunit.client.1.smithi169.stdout: 10 60643 19.92 MB/sec execute 33 sec latency 124.470 ms 2024-04-24T23:51:16.390 INFO:tasks.workunit.client.1.smithi169.stdout: 10 60813 19.52 MB/sec execute 34 sec latency 107.304 ms 2024-04-24T23:51:17.390 INFO:tasks.workunit.client.1.smithi169.stdout: 10 61071 19.23 MB/sec execute 35 sec latency 154.072 ms 2024-04-24T23:51:18.390 INFO:tasks.workunit.client.1.smithi169.stdout: 10 61372 19.14 MB/sec execute 36 sec latency 114.458 ms 2024-04-24T23:51:19.390 INFO:tasks.workunit.client.1.smithi169.stdout: 10 61886 18.98 MB/sec execute 37 sec latency 132.639 ms 2024-04-24T23:51:20.390 INFO:tasks.workunit.client.1.smithi169.stdout: 10 62392 19.31 MB/sec execute 38 sec latency 118.169 ms 2024-04-24T23:51:21.390 INFO:tasks.workunit.client.1.smithi169.stdout: 10 63097 19.87 MB/sec execute 39 sec latency 45.134 ms 2024-04-24T23:51:22.390 INFO:tasks.workunit.client.1.smithi169.stdout: 10 63643 20.33 MB/sec execute 40 sec latency 60.868 ms 2024-04-24T23:51:23.390 INFO:tasks.workunit.client.1.smithi169.stdout: 10 63952 20.38 MB/sec execute 41 sec latency 100.959 ms 2024-04-24T23:51:24.390 INFO:tasks.workunit.client.1.smithi169.stdout: 10 64251 20.29 MB/sec execute 42 sec latency 91.570 ms 2024-04-24T23:51:25.390 INFO:tasks.workunit.client.1.smithi169.stdout: 10 64466 19.97 MB/sec execute 43 sec latency 139.205 ms 2024-04-24T23:51:26.390 INFO:tasks.workunit.client.1.smithi169.stdout: 10 64893 19.99 MB/sec execute 44 sec latency 58.735 ms 2024-04-24T23:51:27.390 INFO:tasks.workunit.client.1.smithi169.stdout: 10 65425 19.85 MB/sec execute 45 sec latency 109.969 ms 2024-04-24T23:51:28.390 INFO:tasks.workunit.client.1.smithi169.stdout: 10 65945 20.13 MB/sec execute 46 sec latency 105.266 ms 2024-04-24T23:51:29.390 INFO:tasks.workunit.client.1.smithi169.stdout: 10 66671 20.70 MB/sec execute 47 sec latency 59.535 ms 2024-04-24T23:51:30.390 INFO:tasks.workunit.client.1.smithi169.stdout: 10 67175 20.90 MB/sec execute 48 sec latency 102.503 ms 2024-04-24T23:51:31.391 INFO:tasks.workunit.client.1.smithi169.stdout: 10 67538 21.02 MB/sec execute 49 sec latency 116.003 ms 2024-04-24T23:51:32.392 INFO:tasks.workunit.client.1.smithi169.stdout: 10 67619 20.63 MB/sec execute 50 sec latency 850.826 ms 2024-04-24T23:51:33.392 INFO:tasks.workunit.client.1.smithi169.stdout: 10 67647 20.27 MB/sec execute 51 sec latency 1324.897 ms 2024-04-24T23:51:34.392 INFO:tasks.workunit.client.1.smithi169.stdout: 10 67823 20.03 MB/sec execute 52 sec latency 201.287 ms 2024-04-24T23:51:35.391 INFO:tasks.workunit.client.1.smithi169.stdout: 10 67980 19.77 MB/sec execute 53 sec latency 116.106 ms 2024-04-24T23:51:36.391 INFO:tasks.workunit.client.1.smithi169.stdout: 10 68326 19.75 MB/sec execute 54 sec latency 104.835 ms 2024-04-24T23:51:37.392 INFO:tasks.workunit.client.1.smithi169.stdout: 10 68659 19.57 MB/sec execute 55 sec latency 125.592 ms 2024-04-24T23:51:38.392 INFO:tasks.workunit.client.1.smithi169.stdout: 10 68745 19.25 MB/sec execute 56 sec latency 736.610 ms 2024-04-24T23:51:39.392 INFO:tasks.workunit.client.1.smithi169.stdout: 10 69246 19.29 MB/sec execute 57 sec latency 624.121 ms 2024-04-24T23:51:40.392 INFO:tasks.workunit.client.1.smithi169.stdout: 10 69853 19.59 MB/sec execute 58 sec latency 83.139 ms 2024-04-24T23:51:41.392 INFO:tasks.workunit.client.1.smithi169.stdout: 10 70407 19.87 MB/sec execute 59 sec latency 77.606 ms 2024-04-24T23:51:42.392 INFO:tasks.workunit.client.1.smithi169.stdout: 10 70812 20.00 MB/sec execute 60 sec latency 112.150 ms 2024-04-24T23:51:43.392 INFO:tasks.workunit.client.1.smithi169.stdout: 10 71136 20.03 MB/sec execute 61 sec latency 107.230 ms 2024-04-24T23:51:44.392 INFO:tasks.workunit.client.1.smithi169.stdout: 10 71398 19.89 MB/sec execute 62 sec latency 105.754 ms 2024-04-24T23:51:45.392 INFO:tasks.workunit.client.1.smithi169.stdout: 10 71712 19.80 MB/sec execute 63 sec latency 83.278 ms 2024-04-24T23:51:46.392 INFO:tasks.workunit.client.1.smithi169.stdout: 10 72099 19.78 MB/sec execute 64 sec latency 90.154 ms 2024-04-24T23:51:47.392 INFO:tasks.workunit.client.1.smithi169.stdout: 10 72283 19.53 MB/sec execute 65 sec latency 498.800 ms 2024-04-24T23:51:48.392 INFO:tasks.workunit.client.1.smithi169.stdout: 10 72468 19.30 MB/sec execute 66 sec latency 1068.098 ms 2024-04-24T23:51:49.392 INFO:tasks.workunit.client.1.smithi169.stdout: 10 72749 19.19 MB/sec execute 67 sec latency 123.884 ms 2024-04-24T23:51:50.392 INFO:tasks.workunit.client.1.smithi169.stdout: 10 73282 19.46 MB/sec execute 68 sec latency 73.232 ms 2024-04-24T23:51:51.392 INFO:tasks.workunit.client.1.smithi169.stdout: 10 73950 19.79 MB/sec execute 69 sec latency 48.795 ms 2024-04-24T23:51:52.392 INFO:tasks.workunit.client.1.smithi169.stdout: 10 74441 19.97 MB/sec execute 70 sec latency 105.071 ms 2024-04-24T23:51:53.392 INFO:tasks.workunit.client.1.smithi169.stdout: 10 74782 19.99 MB/sec execute 71 sec latency 80.428 ms 2024-04-24T23:51:54.392 INFO:tasks.workunit.client.1.smithi169.stdout: 10 75027 19.82 MB/sec execute 72 sec latency 71.939 ms 2024-04-24T23:51:55.392 INFO:tasks.workunit.client.1.smithi169.stdout: 10 75330 19.70 MB/sec execute 73 sec latency 90.734 ms 2024-04-24T23:51:56.395 INFO:tasks.workunit.client.1.smithi169.stdout: 10 75674 19.71 MB/sec execute 74 sec latency 112.899 ms 2024-04-24T23:51:57.392 INFO:tasks.workunit.client.1.smithi169.stdout: 10 76265 19.70 MB/sec execute 75 sec latency 107.631 ms 2024-04-24T23:51:58.392 INFO:tasks.workunit.client.1.smithi169.stdout: 10 76746 19.80 MB/sec execute 76 sec latency 79.790 ms 2024-04-24T23:51:59.392 INFO:tasks.workunit.client.1.smithi169.stdout: 10 77374 20.03 MB/sec execute 77 sec latency 72.026 ms 2024-04-24T23:52:00.392 INFO:tasks.workunit.client.1.smithi169.stdout: 10 77877 20.28 MB/sec execute 78 sec latency 101.443 ms 2024-04-24T23:52:01.392 INFO:tasks.workunit.client.1.smithi169.stdout: 10 78226 20.33 MB/sec execute 79 sec latency 92.426 ms 2024-04-24T23:52:02.392 INFO:tasks.workunit.client.1.smithi169.stdout: 10 78453 20.19 MB/sec execute 80 sec latency 111.917 ms 2024-04-24T23:52:03.392 INFO:tasks.workunit.client.1.smithi169.stdout: 10 78690 20.04 MB/sec execute 81 sec latency 134.895 ms 2024-04-24T23:52:04.393 INFO:tasks.workunit.client.1.smithi169.stdout: 10 79011 19.95 MB/sec execute 82 sec latency 119.228 ms 2024-04-24T23:52:05.393 INFO:tasks.workunit.client.1.smithi169.stdout: 10 79486 19.93 MB/sec execute 83 sec latency 99.832 ms 2024-04-24T23:52:06.393 INFO:tasks.workunit.client.1.smithi169.stdout: 10 80136 20.03 MB/sec execute 84 sec latency 47.376 ms 2024-04-24T23:52:07.392 INFO:tasks.workunit.client.1.smithi169.stdout: 10 80869 20.34 MB/sec execute 85 sec latency 56.948 ms 2024-04-24T23:52:08.393 INFO:tasks.workunit.client.1.smithi169.stdout: 10 81409 20.60 MB/sec execute 86 sec latency 51.470 ms 2024-04-24T23:52:09.393 INFO:tasks.workunit.client.1.smithi169.stdout: 10 81770 20.66 MB/sec execute 87 sec latency 84.225 ms 2024-04-24T23:52:10.393 INFO:tasks.workunit.client.1.smithi169.stdout: 10 82001 20.53 MB/sec execute 88 sec latency 79.070 ms 2024-04-24T23:52:11.393 INFO:tasks.workunit.client.1.smithi169.stdout: 10 82255 20.39 MB/sec execute 89 sec latency 116.073 ms 2024-04-24T23:52:12.393 INFO:tasks.workunit.client.1.smithi169.stdout: 10 82543 20.27 MB/sec execute 90 sec latency 139.380 ms 2024-04-24T23:52:13.393 INFO:tasks.workunit.client.1.smithi169.stdout: 10 82622 20.12 MB/sec execute 91 sec latency 871.092 ms 2024-04-24T23:52:14.395 INFO:tasks.workunit.client.1.smithi169.stdout: 10 82635 19.90 MB/sec execute 92 sec latency 1752.770 ms 2024-04-24T23:52:15.393 INFO:tasks.workunit.client.1.smithi169.stdout: 10 82704 19.73 MB/sec execute 93 sec latency 1996.256 ms 2024-04-24T23:52:16.393 INFO:tasks.workunit.client.1.smithi169.stdout: 10 82952 19.63 MB/sec execute 94 sec latency 137.643 ms 2024-04-24T23:52:17.393 INFO:tasks.workunit.client.1.smithi169.stdout: 10 83321 19.54 MB/sec execute 95 sec latency 104.743 ms 2024-04-24T23:52:18.393 INFO:tasks.workunit.client.1.smithi169.stdout: 10 83713 19.53 MB/sec execute 96 sec latency 96.660 ms 2024-04-24T23:52:19.393 INFO:tasks.workunit.client.1.smithi169.stdout: 10 84423 19.82 MB/sec execute 97 sec latency 110.542 ms 2024-04-24T23:52:20.393 INFO:tasks.workunit.client.1.smithi169.stdout: 10 85036 20.12 MB/sec execute 98 sec latency 91.936 ms 2024-04-24T23:52:21.393 INFO:tasks.workunit.client.1.smithi169.stdout: 10 85368 20.13 MB/sec execute 99 sec latency 92.358 ms 2024-04-24T23:52:22.393 INFO:tasks.workunit.client.1.smithi169.stdout: 10 85603 19.99 MB/sec execute 100 sec latency 104.692 ms 2024-04-24T23:52:23.393 INFO:tasks.workunit.client.1.smithi169.stdout: 10 85848 19.88 MB/sec execute 101 sec latency 99.691 ms 2024-04-24T23:52:24.393 INFO:tasks.workunit.client.1.smithi169.stdout: 10 86227 19.85 MB/sec execute 102 sec latency 101.772 ms 2024-04-24T23:52:25.393 INFO:tasks.workunit.client.1.smithi169.stdout: 10 86832 19.88 MB/sec execute 103 sec latency 69.924 ms 2024-04-24T23:52:26.393 INFO:tasks.workunit.client.1.smithi169.stdout: 10 87488 20.00 MB/sec execute 104 sec latency 72.586 ms 2024-04-24T23:52:27.393 INFO:tasks.workunit.client.1.smithi169.stdout: 10 88248 20.31 MB/sec execute 105 sec latency 45.049 ms 2024-04-24T23:52:28.393 INFO:tasks.workunit.client.1.smithi169.stdout: 10 88707 20.47 MB/sec execute 106 sec latency 95.234 ms 2024-04-24T23:52:29.393 INFO:tasks.workunit.client.1.smithi169.stdout: 10 89015 20.44 MB/sec execute 107 sec latency 83.405 ms 2024-04-24T23:52:30.393 INFO:tasks.workunit.client.1.smithi169.stdout: 10 89268 20.32 MB/sec execute 108 sec latency 115.718 ms 2024-04-24T23:52:31.393 INFO:tasks.workunit.client.1.smithi169.stdout: 10 89564 20.21 MB/sec execute 109 sec latency 90.113 ms 2024-04-24T23:52:32.393 INFO:tasks.workunit.client.1.smithi169.stdout: 10 90055 20.30 MB/sec execute 110 sec latency 110.033 ms 2024-04-24T23:52:33.393 INFO:tasks.workunit.client.1.smithi169.stdout: 10 90434 20.18 MB/sec execute 111 sec latency 375.243 ms 2024-04-24T23:52:34.393 INFO:tasks.workunit.client.1.smithi169.stdout: 10 90460 20.00 MB/sec execute 112 sec latency 1177.337 ms 2024-04-24T23:52:35.393 INFO:tasks.workunit.client.1.smithi169.stdout: 10 90804 19.98 MB/sec execute 113 sec latency 1214.820 ms 2024-04-24T23:52:36.393 INFO:tasks.workunit.client.1.smithi169.stdout: 10 91011 19.93 MB/sec execute 114 sec latency 100.095 ms 2024-04-24T23:52:37.393 INFO:tasks.workunit.client.1.smithi169.stdout: 10 91507 20.08 MB/sec execute 115 sec latency 116.107 ms 2024-04-24T23:52:38.394 INFO:tasks.workunit.client.1.smithi169.stdout: 10 92034 20.23 MB/sec execute 116 sec latency 85.506 ms 2024-04-24T23:52:39.394 INFO:tasks.workunit.client.1.smithi169.stdout: 10 92407 20.26 MB/sec execute 117 sec latency 77.478 ms 2024-04-24T23:52:40.394 INFO:tasks.workunit.client.1.smithi169.stdout: 10 92648 20.19 MB/sec execute 118 sec latency 116.518 ms 2024-04-24T23:52:41.393 INFO:tasks.workunit.client.1.smithi169.stdout: 10 92893 20.09 MB/sec execute 119 sec latency 144.017 ms 2024-04-24T23:52:42.394 INFO:tasks.workunit.client.1.smithi169.stdout: 10 93172 20.01 MB/sec execute 120 sec latency 180.045 ms 2024-04-24T23:52:43.394 INFO:tasks.workunit.client.1.smithi169.stdout: 10 93688 20.04 MB/sec execute 121 sec latency 85.300 ms 2024-04-24T23:52:44.394 INFO:tasks.workunit.client.1.smithi169.stdout: 10 94413 20.14 MB/sec execute 122 sec latency 57.923 ms 2024-04-24T23:52:45.394 INFO:tasks.workunit.client.1.smithi169.stdout: 10 95110 20.36 MB/sec execute 123 sec latency 37.461 ms 2024-04-24T23:52:46.394 INFO:tasks.workunit.client.1.smithi169.stdout: 10 95662 20.51 MB/sec execute 124 sec latency 83.968 ms 2024-04-24T23:52:47.394 INFO:tasks.workunit.client.1.smithi169.stdout: 10 96026 20.54 MB/sec execute 125 sec latency 62.675 ms 2024-04-24T23:52:47.417 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:52:47.419 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:52:47.453 INFO:tasks.ceph.mds.b.smithi169.stderr:2024-04-24T23:52:47.453+0000 7f12ddd9e640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 39701) UID: 0 2024-04-24T23:52:47.453 INFO:tasks.ceph.mds.d.smithi169.stderr:2024-04-24T23:52:47.453+0000 7fcbfaa77640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 39701) UID: 0 2024-04-24T23:52:48.394 INFO:tasks.workunit.client.1.smithi169.stdout: 10 96258 20.45 MB/sec execute 126 sec latency 126.856 ms 2024-04-24T23:52:49.394 INFO:tasks.workunit.client.1.smithi169.stdout: 10 96302 20.29 MB/sec execute 127 sec latency 962.735 ms 2024-04-24T23:52:50.394 INFO:tasks.workunit.client.1.smithi169.stdout: 10 96333 20.13 MB/sec execute 128 sec latency 1161.286 ms 2024-04-24T23:52:51.394 INFO:tasks.workunit.client.1.smithi169.stdout: 10 96434 20.00 MB/sec execute 129 sec latency 303.813 ms 2024-04-24T23:52:52.394 INFO:tasks.workunit.client.1.smithi169.stdout: 10 96543 19.86 MB/sec execute 130 sec latency 185.931 ms 2024-04-24T23:52:53.394 INFO:tasks.workunit.client.1.smithi169.stdout: 10 96947 19.91 MB/sec execute 131 sec latency 68.295 ms 2024-04-24T23:52:54.394 INFO:tasks.workunit.client.1.smithi169.stdout: 10 97511 19.86 MB/sec execute 132 sec latency 84.987 ms 2024-04-24T23:52:55.394 INFO:tasks.workunit.client.1.smithi169.stdout: 10 98114 19.99 MB/sec execute 133 sec latency 53.134 ms 2024-04-24T23:52:56.394 INFO:tasks.workunit.client.1.smithi169.stdout: 10 98841 20.19 MB/sec execute 134 sec latency 95.567 ms 2024-04-24T23:52:57.394 INFO:tasks.workunit.client.1.smithi169.stdout: 10 99260 20.28 MB/sec execute 135 sec latency 91.164 ms 2024-04-24T23:52:58.394 INFO:tasks.workunit.client.1.smithi169.stdout: 10 99560 20.27 MB/sec execute 136 sec latency 97.232 ms 2024-04-24T23:52:59.394 INFO:tasks.workunit.client.1.smithi169.stdout: 10 99774 20.17 MB/sec execute 137 sec latency 191.243 ms 2024-04-24T23:53:00.394 INFO:tasks.workunit.client.1.smithi169.stdout: 10 100017 20.08 MB/sec execute 138 sec latency 126.301 ms 2024-04-24T23:53:01.394 INFO:tasks.workunit.client.1.smithi169.stdout: 10 100441 20.11 MB/sec execute 139 sec latency 52.851 ms 2024-04-24T23:53:02.394 INFO:tasks.workunit.client.1.smithi169.stdout: 10 100896 20.07 MB/sec execute 140 sec latency 107.101 ms 2024-04-24T23:53:03.394 INFO:tasks.workunit.client.1.smithi169.stdout: 10 101523 20.13 MB/sec execute 141 sec latency 54.882 ms 2024-04-24T23:53:04.394 INFO:tasks.workunit.client.1.smithi169.stdout: 10 102195 20.32 MB/sec execute 142 sec latency 44.512 ms 2024-04-24T23:53:05.394 INFO:tasks.workunit.client.1.smithi169.stdout: 10 102672 20.40 MB/sec execute 143 sec latency 95.815 ms 2024-04-24T23:53:06.394 INFO:tasks.workunit.client.1.smithi169.stdout: 10 103026 20.43 MB/sec execute 144 sec latency 81.757 ms 2024-04-24T23:53:07.395 INFO:tasks.workunit.client.1.smithi169.stdout: 10 103295 20.39 MB/sec execute 145 sec latency 172.451 ms 2024-04-24T23:53:08.394 INFO:tasks.workunit.client.1.smithi169.stdout: 10 103495 20.29 MB/sec execute 146 sec latency 142.198 ms 2024-04-24T23:53:09.395 INFO:tasks.workunit.client.1.smithi169.stdout: 10 103785 20.21 MB/sec execute 147 sec latency 105.625 ms 2024-04-24T23:53:10.395 INFO:tasks.workunit.client.1.smithi169.stdout: 10 104271 20.25 MB/sec execute 148 sec latency 94.528 ms 2024-04-24T23:53:11.395 INFO:tasks.workunit.client.1.smithi169.stdout: 10 104940 20.24 MB/sec execute 149 sec latency 50.894 ms 2024-04-24T23:53:12.395 INFO:tasks.workunit.client.1.smithi169.stdout: 10 105575 20.45 MB/sec execute 150 sec latency 79.221 ms 2024-04-24T23:53:13.395 INFO:tasks.workunit.client.1.smithi169.stdout: 10 106145 20.55 MB/sec execute 151 sec latency 83.370 ms 2024-04-24T23:53:14.395 INFO:tasks.workunit.client.1.smithi169.stdout: 10 106380 20.52 MB/sec execute 152 sec latency 430.431 ms 2024-04-24T23:53:15.395 INFO:tasks.workunit.client.1.smithi169.stdout: 10 106416 20.41 MB/sec execute 153 sec latency 1430.470 ms 2024-04-24T23:53:16.395 INFO:tasks.workunit.client.1.smithi169.stdout: 10 106442 20.30 MB/sec execute 154 sec latency 1661.078 ms 2024-04-24T23:53:17.395 INFO:tasks.workunit.client.1.smithi169.stdout: 10 106563 20.21 MB/sec execute 155 sec latency 553.901 ms 2024-04-24T23:53:18.395 INFO:tasks.workunit.client.1.smithi169.stdout: 10 106720 20.13 MB/sec execute 156 sec latency 115.570 ms 2024-04-24T23:53:19.395 INFO:tasks.workunit.client.1.smithi169.stdout: 10 106868 20.05 MB/sec execute 157 sec latency 104.890 ms 2024-04-24T23:53:20.395 INFO:tasks.workunit.client.1.smithi169.stdout: 10 107069 19.96 MB/sec execute 158 sec latency 122.741 ms 2024-04-24T23:53:21.397 INFO:tasks.workunit.client.1.smithi169.stdout: 10 107385 19.93 MB/sec execute 159 sec latency 182.926 ms 2024-04-24T23:53:22.395 INFO:tasks.workunit.client.1.smithi169.stdout: 10 107850 19.93 MB/sec execute 160 sec latency 93.442 ms 2024-04-24T23:53:23.395 INFO:tasks.workunit.client.1.smithi169.stdout: 10 108338 19.89 MB/sec execute 161 sec latency 117.133 ms 2024-04-24T23:53:24.395 INFO:tasks.workunit.client.1.smithi169.stdout: 10 109028 20.10 MB/sec execute 162 sec latency 34.835 ms 2024-04-24T23:53:25.395 INFO:tasks.workunit.client.1.smithi169.stdout: 10 109652 20.19 MB/sec execute 163 sec latency 63.366 ms 2024-04-24T23:53:26.396 INFO:tasks.workunit.client.1.smithi169.stdout: 10 110002 20.22 MB/sec execute 164 sec latency 88.333 ms 2024-04-24T23:53:27.395 INFO:tasks.workunit.client.1.smithi169.stdout: 10 110311 20.19 MB/sec execute 165 sec latency 82.719 ms 2024-04-24T23:53:28.395 INFO:tasks.workunit.client.1.smithi169.stdout: 10 110506 20.13 MB/sec execute 166 sec latency 161.741 ms 2024-04-24T23:53:29.395 INFO:tasks.workunit.client.1.smithi169.stdout: 10 110749 20.06 MB/sec execute 167 sec latency 195.455 ms 2024-04-24T23:53:30.395 INFO:tasks.workunit.client.1.smithi169.stdout: 10 111195 20.11 MB/sec execute 168 sec latency 112.657 ms 2024-04-24T23:53:31.395 INFO:tasks.workunit.client.1.smithi169.stdout: 10 111697 20.04 MB/sec execute 169 sec latency 114.948 ms 2024-04-24T23:53:32.395 INFO:tasks.workunit.client.1.smithi169.stdout: 10 112259 20.12 MB/sec execute 170 sec latency 94.649 ms 2024-04-24T23:53:33.395 INFO:tasks.workunit.client.1.smithi169.stdout: 10 112990 20.30 MB/sec execute 171 sec latency 56.449 ms 2024-04-24T23:53:34.395 INFO:tasks.workunit.client.1.smithi169.stdout: 10 113454 20.36 MB/sec execute 172 sec latency 72.192 ms 2024-04-24T23:53:35.396 INFO:tasks.workunit.client.1.smithi169.stdout: 10 113486 20.25 MB/sec execute 173 sec latency 654.516 ms 2024-04-24T23:53:36.396 INFO:tasks.workunit.client.1.smithi169.stdout: 10 113520 20.15 MB/sec execute 174 sec latency 1313.999 ms 2024-04-24T23:53:37.396 INFO:tasks.workunit.client.1.smithi169.stdout: 10 113663 20.08 MB/sec execute 175 sec latency 174.059 ms 2024-04-24T23:53:38.396 INFO:tasks.workunit.client.1.smithi169.stdout: 10 113820 20.01 MB/sec execute 176 sec latency 121.785 ms 2024-04-24T23:53:39.396 INFO:tasks.workunit.client.1.smithi169.stdout: 10 114125 19.97 MB/sec execute 177 sec latency 176.352 ms 2024-04-24T23:53:40.396 INFO:tasks.workunit.client.1.smithi169.stdout: 10 114313 19.91 MB/sec execute 178 sec latency 128.382 ms 2024-04-24T23:53:41.396 INFO:tasks.workunit.client.1.smithi169.stdout: 10 114795 19.92 MB/sec execute 179 sec latency 104.227 ms 2024-04-24T23:53:42.396 INFO:tasks.workunit.client.1.smithi169.stdout: 10 115398 19.92 MB/sec execute 180 sec latency 63.271 ms 2024-04-24T23:53:43.396 INFO:tasks.workunit.client.1.smithi169.stdout: 10 115813 19.94 MB/sec execute 181 sec latency 111.063 ms 2024-04-24T23:53:44.396 INFO:tasks.workunit.client.1.smithi169.stdout: 10 116476 20.09 MB/sec execute 182 sec latency 53.063 ms 2024-04-24T23:53:45.399 INFO:tasks.workunit.client.1.smithi169.stdout: 10 116906 20.14 MB/sec execute 183 sec latency 103.250 ms 2024-04-24T23:53:46.396 INFO:tasks.workunit.client.1.smithi169.stdout: 10 117227 20.14 MB/sec execute 184 sec latency 91.785 ms 2024-04-24T23:53:47.396 INFO:tasks.workunit.client.1.smithi169.stdout: 10 117502 20.10 MB/sec execute 185 sec latency 91.142 ms 2024-04-24T23:53:48.396 INFO:tasks.workunit.client.1.smithi169.stdout: 10 117696 20.04 MB/sec execute 186 sec latency 152.602 ms 2024-04-24T23:53:49.396 INFO:tasks.workunit.client.1.smithi169.stdout: 10 117994 20.01 MB/sec execute 187 sec latency 193.833 ms 2024-04-24T23:53:50.396 INFO:tasks.workunit.client.1.smithi169.stdout: 10 118478 20.02 MB/sec execute 188 sec latency 91.280 ms 2024-04-24T23:53:51.399 INFO:tasks.workunit.client.1.smithi169.stdout: 10 118960 19.98 MB/sec execute 189 sec latency 116.680 ms 2024-04-24T23:53:52.399 INFO:tasks.workunit.client.1.smithi169.stdout: 10 119574 20.10 MB/sec execute 190 sec latency 67.629 ms 2024-04-24T23:53:53.399 INFO:tasks.workunit.client.1.smithi169.stdout: 10 120227 20.22 MB/sec execute 191 sec latency 58.592 ms 2024-04-24T23:53:54.399 INFO:tasks.workunit.client.1.smithi169.stdout: 10 120600 20.25 MB/sec execute 192 sec latency 94.657 ms 2024-04-24T23:53:55.399 INFO:tasks.workunit.client.1.smithi169.stdout: 10 120769 20.19 MB/sec execute 193 sec latency 466.831 ms 2024-04-24T23:53:56.399 INFO:tasks.workunit.client.1.smithi169.stdout: 10 120783 20.09 MB/sec execute 194 sec latency 1237.573 ms 2024-04-24T23:53:57.399 INFO:tasks.workunit.client.1.smithi169.stdout: 10 120848 20.00 MB/sec execute 195 sec latency 880.603 ms 2024-04-24T23:53:58.399 INFO:tasks.workunit.client.1.smithi169.stdout: 10 121027 19.94 MB/sec execute 196 sec latency 171.848 ms 2024-04-24T23:53:59.399 INFO:tasks.workunit.client.1.smithi169.stdout: 10 121180 19.87 MB/sec execute 197 sec latency 193.787 ms 2024-04-24T23:54:00.399 INFO:tasks.workunit.client.1.smithi169.stdout: 10 121488 19.84 MB/sec execute 198 sec latency 99.311 ms 2024-04-24T23:54:01.399 INFO:tasks.workunit.client.1.smithi169.stdout: 10 121971 19.87 MB/sec execute 199 sec latency 85.706 ms 2024-04-24T23:54:02.399 INFO:tasks.workunit.client.1.smithi169.stdout: 10 122535 19.85 MB/sec execute 200 sec latency 103.278 ms 2024-04-24T23:54:03.399 INFO:tasks.workunit.client.1.smithi169.stdout: 10 123088 19.95 MB/sec execute 201 sec latency 71.439 ms 2024-04-24T23:54:04.399 INFO:tasks.workunit.client.1.smithi169.stdout: 10 123749 20.07 MB/sec execute 202 sec latency 78.204 ms 2024-04-24T23:54:05.399 INFO:tasks.workunit.client.1.smithi169.stdout: 10 124068 20.07 MB/sec execute 203 sec latency 186.720 ms 2024-04-24T23:54:06.399 INFO:tasks.workunit.client.1.smithi169.stdout: 10 124351 20.03 MB/sec execute 204 sec latency 121.508 ms 2024-04-24T23:54:07.399 INFO:tasks.workunit.client.1.smithi169.stdout: 10 124588 20.00 MB/sec execute 205 sec latency 162.524 ms 2024-04-24T23:54:08.399 INFO:tasks.workunit.client.1.smithi169.stdout: 10 124873 19.97 MB/sec execute 206 sec latency 136.997 ms 2024-04-24T23:54:09.399 INFO:tasks.workunit.client.1.smithi169.stdout: 10 125207 19.93 MB/sec execute 207 sec latency 125.511 ms 2024-04-24T23:54:10.399 INFO:tasks.workunit.client.1.smithi169.stdout: 10 125671 19.95 MB/sec execute 208 sec latency 66.059 ms 2024-04-24T23:54:11.399 INFO:tasks.workunit.client.1.smithi169.stdout: 10 126167 19.92 MB/sec execute 209 sec latency 146.637 ms 2024-04-24T23:54:12.399 INFO:tasks.workunit.client.1.smithi169.stdout: 10 126695 20.03 MB/sec execute 210 sec latency 80.683 ms 2024-04-24T23:54:13.399 INFO:tasks.workunit.client.1.smithi169.stdout: 10 127215 20.10 MB/sec execute 211 sec latency 102.119 ms 2024-04-24T23:54:14.400 INFO:tasks.workunit.client.1.smithi169.stdout: 10 127590 20.10 MB/sec execute 212 sec latency 134.563 ms 2024-04-24T23:54:15.400 INFO:tasks.workunit.client.1.smithi169.stdout: 10 127864 20.07 MB/sec execute 213 sec latency 145.912 ms 2024-04-24T23:54:16.399 INFO:tasks.workunit.client.1.smithi169.stdout: 10 127883 19.98 MB/sec execute 214 sec latency 1028.673 ms 2024-04-24T23:54:17.400 INFO:tasks.workunit.client.1.smithi169.stdout: 10 127917 19.91 MB/sec execute 215 sec latency 1784.717 ms 2024-04-24T23:54:18.400 INFO:tasks.workunit.client.1.smithi169.stdout: 10 128032 19.85 MB/sec execute 216 sec latency 420.286 ms 2024-04-24T23:54:19.400 INFO:tasks.workunit.client.1.smithi169.stdout: 10 128146 19.77 MB/sec execute 217 sec latency 324.821 ms 2024-04-24T23:54:20.400 INFO:tasks.workunit.client.1.smithi169.stdout: 10 128332 19.72 MB/sec execute 218 sec latency 134.810 ms 2024-04-24T23:54:21.400 INFO:tasks.workunit.client.1.smithi169.stdout: 10 128733 19.72 MB/sec execute 219 sec latency 85.038 ms 2024-04-24T23:54:22.400 INFO:tasks.workunit.client.1.smithi169.stdout: 10 129206 19.71 MB/sec execute 220 sec latency 148.714 ms 2024-04-24T23:54:23.400 INFO:tasks.workunit.client.1.smithi169.stdout: 10 129637 19.71 MB/sec execute 221 sec latency 116.607 ms 2024-04-24T23:54:24.400 INFO:tasks.workunit.client.1.smithi169.stdout: 10 130217 19.79 MB/sec execute 222 sec latency 93.988 ms 2024-04-24T23:54:25.400 INFO:tasks.workunit.client.1.smithi169.stdout: 10 130774 19.87 MB/sec execute 223 sec latency 104.118 ms 2024-04-24T23:54:26.400 INFO:tasks.workunit.client.1.smithi169.stdout: 10 131176 19.89 MB/sec execute 224 sec latency 60.216 ms 2024-04-24T23:54:27.400 INFO:tasks.workunit.client.1.smithi169.stdout: 10 131451 19.87 MB/sec execute 225 sec latency 106.294 ms 2024-04-24T23:54:28.403 INFO:tasks.workunit.client.1.smithi169.stdout: 10 131719 19.84 MB/sec execute 226 sec latency 140.417 ms 2024-04-24T23:54:29.403 INFO:tasks.workunit.client.1.smithi169.stdout: 10 131975 19.80 MB/sec execute 227 sec latency 145.157 ms 2024-04-24T23:54:30.403 INFO:tasks.workunit.client.1.smithi169.stdout: 10 132309 19.79 MB/sec execute 228 sec latency 119.674 ms 2024-04-24T23:54:31.403 INFO:tasks.workunit.client.1.smithi169.stdout: 10 132841 19.79 MB/sec execute 229 sec latency 55.167 ms 2024-04-24T23:54:32.403 INFO:tasks.workunit.client.1.smithi169.stdout: 10 133206 19.76 MB/sec execute 230 sec latency 126.797 ms 2024-04-24T23:54:33.403 INFO:tasks.workunit.client.1.smithi169.stdout: 10 133768 19.86 MB/sec execute 231 sec latency 92.682 ms 2024-04-24T23:54:34.403 INFO:tasks.workunit.client.1.smithi169.stdout: 10 134438 19.95 MB/sec execute 232 sec latency 50.594 ms 2024-04-24T23:54:35.403 INFO:tasks.workunit.client.1.smithi169.stdout: 10 134766 19.96 MB/sec execute 233 sec latency 120.464 ms 2024-04-24T23:54:36.403 INFO:tasks.workunit.client.1.smithi169.stdout: 10 134953 19.92 MB/sec execute 234 sec latency 415.383 ms 2024-04-24T23:54:37.403 INFO:tasks.workunit.client.1.smithi169.stdout: 10 134959 19.83 MB/sec execute 235 sec latency 1415.420 ms 2024-04-24T23:54:38.403 INFO:tasks.workunit.client.1.smithi169.stdout: 10 135043 19.76 MB/sec execute 236 sec latency 1422.396 ms 2024-04-24T23:54:39.403 INFO:tasks.workunit.client.1.smithi169.stdout: 10 135186 19.70 MB/sec execute 237 sec latency 296.648 ms 2024-04-24T23:54:40.409 INFO:tasks.workunit.client.1.smithi169.stdout: 10 135284 19.65 MB/sec execute 238 sec latency 275.273 ms 2024-04-24T23:54:41.404 INFO:tasks.workunit.client.1.smithi169.stdout: 10 135544 19.62 MB/sec execute 239 sec latency 158.335 ms 2024-04-24T23:54:42.403 INFO:tasks.workunit.client.1.smithi169.stdout: 10 135892 19.62 MB/sec execute 240 sec latency 109.116 ms 2024-04-24T23:54:43.403 INFO:tasks.workunit.client.1.smithi169.stdout: 10 136458 19.60 MB/sec execute 241 sec latency 77.780 ms 2024-04-24T23:54:44.403 INFO:tasks.workunit.client.1.smithi169.stdout: 10 136821 19.62 MB/sec execute 242 sec latency 118.963 ms 2024-04-24T23:54:45.403 INFO:tasks.workunit.client.1.smithi169.stdout: 10 137429 19.69 MB/sec execute 243 sec latency 70.122 ms 2024-04-24T23:54:46.404 INFO:tasks.workunit.client.1.smithi169.stdout: 10 137948 19.75 MB/sec execute 244 sec latency 85.481 ms 2024-04-24T23:54:47.403 INFO:tasks.workunit.client.1.smithi169.stdout: 10 138346 19.76 MB/sec execute 245 sec latency 39.567 ms 2024-04-24T23:54:48.404 INFO:tasks.workunit.client.1.smithi169.stdout: 10 138578 19.73 MB/sec execute 246 sec latency 127.813 ms 2024-04-24T23:54:49.404 INFO:tasks.workunit.client.1.smithi169.stdout: 10 138947 19.73 MB/sec execute 247 sec latency 80.802 ms 2024-04-24T23:54:50.404 INFO:tasks.workunit.client.1.smithi169.stdout: 10 139332 19.74 MB/sec execute 248 sec latency 99.935 ms 2024-04-24T23:54:51.404 INFO:tasks.workunit.client.1.smithi169.stdout: 10 139926 19.72 MB/sec execute 249 sec latency 44.908 ms 2024-04-24T23:54:52.404 INFO:tasks.workunit.client.1.smithi169.stdout: 10 140107 19.68 MB/sec execute 250 sec latency 682.215 ms 2024-04-24T23:54:53.404 INFO:tasks.workunit.client.1.smithi169.stdout: 10 140135 19.61 MB/sec execute 251 sec latency 942.045 ms 2024-04-24T23:54:54.404 INFO:tasks.workunit.client.1.smithi169.stdout: 10 140460 19.63 MB/sec execute 252 sec latency 135.894 ms 2024-04-24T23:54:55.404 INFO:tasks.workunit.client.1.smithi169.stdout: 10 140754 19.60 MB/sec execute 253 sec latency 149.469 ms 2024-04-24T23:54:56.404 INFO:tasks.workunit.client.1.smithi169.stdout: 10 141272 19.69 MB/sec execute 254 sec latency 86.602 ms 2024-04-24T23:54:57.404 INFO:tasks.workunit.client.1.smithi169.stdout: 10 141719 19.69 MB/sec execute 255 sec latency 86.030 ms 2024-04-24T23:54:58.404 INFO:tasks.workunit.client.1.smithi169.stdout: 10 142007 19.69 MB/sec execute 256 sec latency 131.972 ms 2024-04-24T23:54:59.404 INFO:tasks.workunit.client.1.smithi169.stdout: 10 142236 19.64 MB/sec execute 257 sec latency 173.055 ms 2024-04-24T23:55:00.404 INFO:tasks.workunit.client.1.smithi169.stdout: 10 142624 19.66 MB/sec execute 258 sec latency 130.889 ms 2024-04-24T23:55:01.404 INFO:tasks.workunit.client.1.smithi169.stdout: 10 142993 19.65 MB/sec execute 259 sec latency 105.751 ms 2024-04-24T23:55:02.404 INFO:tasks.workunit.client.1.smithi169.stdout: 10 143560 19.64 MB/sec execute 260 sec latency 76.735 ms 2024-04-24T23:55:03.404 INFO:tasks.workunit.client.1.smithi169.stdout: 10 144055 19.69 MB/sec execute 261 sec latency 105.051 ms 2024-04-24T23:55:04.404 INFO:tasks.workunit.client.1.smithi169.stdout: 10 144602 19.77 MB/sec execute 262 sec latency 57.912 ms 2024-04-24T23:55:05.404 INFO:tasks.workunit.client.1.smithi169.stdout: 10 145018 19.78 MB/sec execute 263 sec latency 76.965 ms 2024-04-24T23:55:06.405 INFO:tasks.workunit.client.1.smithi169.stdout: 10 145409 19.77 MB/sec execute 264 sec latency 96.110 ms 2024-04-24T23:55:07.404 INFO:tasks.workunit.client.1.smithi169.stdout: 10 145707 19.76 MB/sec execute 265 sec latency 130.418 ms 2024-04-24T23:55:08.404 INFO:tasks.workunit.client.1.smithi169.stdout: 10 146037 19.73 MB/sec execute 266 sec latency 140.096 ms 2024-04-24T23:55:09.404 INFO:tasks.workunit.client.1.smithi169.stdout: 10 146452 19.78 MB/sec execute 267 sec latency 88.218 ms 2024-04-24T23:55:10.404 INFO:tasks.workunit.client.1.smithi169.stdout: 10 147040 19.76 MB/sec execute 268 sec latency 83.482 ms 2024-04-24T23:55:11.404 INFO:tasks.workunit.client.1.smithi169.stdout: 10 147432 19.76 MB/sec execute 269 sec latency 110.095 ms 2024-04-24T23:55:12.405 INFO:tasks.workunit.client.1.smithi169.stdout: 10 147862 19.80 MB/sec execute 270 sec latency 205.483 ms 2024-04-24T23:55:13.405 INFO:tasks.workunit.client.1.smithi169.stdout: 10 147908 19.73 MB/sec execute 271 sec latency 989.917 ms 2024-04-24T23:55:14.405 INFO:tasks.workunit.client.1.smithi169.stdout: 10 147920 19.66 MB/sec execute 272 sec latency 1617.171 ms 2024-04-24T23:55:15.404 INFO:tasks.workunit.client.1.smithi169.stdout: 10 148144 19.67 MB/sec execute 273 sec latency 802.520 ms 2024-04-24T23:55:16.405 INFO:tasks.workunit.client.1.smithi169.stdout: 10 148383 19.65 MB/sec execute 274 sec latency 112.046 ms 2024-04-24T23:55:17.405 INFO:tasks.workunit.client.1.smithi169.stdout: 10 148786 19.65 MB/sec execute 275 sec latency 100.193 ms 2024-04-24T23:55:18.405 INFO:tasks.workunit.client.1.smithi169.stdout: 10 149046 19.62 MB/sec execute 276 sec latency 110.076 ms 2024-04-24T23:55:19.405 INFO:tasks.workunit.client.1.smithi169.stdout: 10 149323 19.60 MB/sec execute 277 sec latency 116.837 ms 2024-04-24T23:55:20.405 INFO:tasks.workunit.client.1.smithi169.stdout: 10 149684 19.61 MB/sec execute 278 sec latency 148.545 ms 2024-04-24T23:55:21.405 INFO:tasks.workunit.client.1.smithi169.stdout: 10 150062 19.62 MB/sec execute 279 sec latency 104.503 ms 2024-04-24T23:55:22.405 INFO:tasks.workunit.client.1.smithi169.stdout: 10 150634 19.60 MB/sec execute 280 sec latency 40.468 ms 2024-04-24T23:55:23.405 INFO:tasks.workunit.client.1.smithi169.stdout: 10 150972 19.60 MB/sec execute 281 sec latency 130.566 ms 2024-04-24T23:55:24.405 INFO:tasks.workunit.client.1.smithi169.stdout: 10 151512 19.66 MB/sec execute 282 sec latency 82.053 ms 2024-04-24T23:55:25.405 INFO:tasks.workunit.client.1.smithi169.stdout: 10 151961 19.71 MB/sec execute 283 sec latency 162.986 ms 2024-04-24T23:55:26.405 INFO:tasks.workunit.client.1.smithi169.stdout: 10 152381 19.71 MB/sec execute 284 sec latency 100.894 ms 2024-04-24T23:55:27.405 INFO:tasks.workunit.client.1.smithi169.stdout: 10 152649 19.69 MB/sec execute 285 sec latency 123.480 ms 2024-04-24T23:55:28.405 INFO:tasks.workunit.client.1.smithi169.stdout: 10 152700 19.62 MB/sec execute 286 sec latency 839.706 ms 2024-04-24T23:55:29.409 INFO:tasks.workunit.client.1.smithi169.stdout: 10 152749 19.57 MB/sec execute 287 sec latency 1348.514 ms 2024-04-24T23:55:30.409 INFO:tasks.workunit.client.1.smithi169.stdout: 10 152927 19.52 MB/sec execute 288 sec latency 217.921 ms 2024-04-24T23:55:31.409 INFO:tasks.workunit.client.1.smithi169.stdout: 10 153257 19.53 MB/sec execute 289 sec latency 164.521 ms 2024-04-24T23:55:32.409 INFO:tasks.workunit.client.1.smithi169.stdout: 10 153651 19.53 MB/sec execute 290 sec latency 96.920 ms 2024-04-24T23:55:33.409 INFO:tasks.workunit.client.1.smithi169.stdout: 10 154233 19.53 MB/sec execute 291 sec latency 38.834 ms 2024-04-24T23:55:34.409 INFO:tasks.workunit.client.1.smithi169.stdout: 10 154712 19.57 MB/sec execute 292 sec latency 81.745 ms 2024-04-24T23:55:35.409 INFO:tasks.workunit.client.1.smithi169.stdout: 10 155184 19.61 MB/sec execute 293 sec latency 132.069 ms 2024-04-24T23:55:36.409 INFO:tasks.workunit.client.1.smithi169.stdout: 10 155639 19.65 MB/sec execute 294 sec latency 84.814 ms 2024-04-24T23:55:37.409 INFO:tasks.workunit.client.1.smithi169.stdout: 10 156032 19.63 MB/sec execute 295 sec latency 113.517 ms 2024-04-24T23:55:38.409 INFO:tasks.workunit.client.1.smithi169.stdout: 10 156319 19.62 MB/sec execute 296 sec latency 123.088 ms 2024-04-24T23:55:39.409 INFO:tasks.workunit.client.1.smithi169.stdout: 10 156704 19.61 MB/sec execute 297 sec latency 163.613 ms 2024-04-24T23:55:40.409 INFO:tasks.workunit.client.1.smithi169.stdout: 10 157162 19.65 MB/sec execute 298 sec latency 121.365 ms 2024-04-24T23:55:41.409 INFO:tasks.workunit.client.1.smithi169.stdout: 10 157743 19.64 MB/sec execute 299 sec latency 60.345 ms 2024-04-24T23:55:42.409 INFO:tasks.workunit.client.1.smithi169.stdout: 10 158227 19.68 MB/sec execute 300 sec latency 100.974 ms 2024-04-24T23:55:43.409 INFO:tasks.workunit.client.1.smithi169.stdout: 10 158786 19.75 MB/sec execute 301 sec latency 86.476 ms 2024-04-24T23:55:44.409 INFO:tasks.workunit.client.1.smithi169.stdout: 10 159225 19.77 MB/sec execute 302 sec latency 79.018 ms 2024-04-24T23:55:45.410 INFO:tasks.workunit.client.1.smithi169.stdout: 10 159634 19.74 MB/sec execute 303 sec latency 140.553 ms 2024-04-24T23:55:46.409 INFO:tasks.workunit.client.1.smithi169.stdout: 10 159991 19.75 MB/sec execute 304 sec latency 101.133 ms 2024-04-24T23:55:47.409 INFO:tasks.workunit.client.1.smithi169.stdout: 10 160477 19.79 MB/sec execute 305 sec latency 87.168 ms 2024-04-24T23:55:48.409 INFO:tasks.workunit.client.1.smithi169.stdout: 10 161003 19.78 MB/sec execute 306 sec latency 82.501 ms 2024-04-24T23:55:49.409 INFO:tasks.workunit.client.1.smithi169.stdout: 10 161550 19.81 MB/sec execute 307 sec latency 108.035 ms 2024-04-24T23:55:50.411 INFO:tasks.workunit.client.1.smithi169.stdout: 10 161998 19.84 MB/sec execute 308 sec latency 175.301 ms 2024-04-24T23:55:51.412 INFO:tasks.workunit.client.1.smithi169.stdout: 10 162500 19.89 MB/sec execute 309 sec latency 79.445 ms 2024-04-24T23:55:52.414 INFO:tasks.workunit.client.1.smithi169.stdout: 10 162925 19.89 MB/sec execute 310 sec latency 97.858 ms 2024-04-24T23:55:53.414 INFO:tasks.workunit.client.1.smithi169.stdout: 10 163243 19.87 MB/sec execute 311 sec latency 97.184 ms 2024-04-24T23:55:54.415 INFO:tasks.workunit.client.1.smithi169.stdout: 10 163611 19.87 MB/sec execute 312 sec latency 95.594 ms 2024-04-24T23:55:55.414 INFO:tasks.workunit.client.1.smithi169.stdout: 10 164081 19.89 MB/sec execute 313 sec latency 94.981 ms 2024-04-24T23:55:56.415 INFO:tasks.workunit.client.1.smithi169.stdout: 10 164704 19.91 MB/sec execute 314 sec latency 40.788 ms 2024-04-24T23:55:57.415 INFO:tasks.workunit.client.1.smithi169.stdout: 10 165217 19.93 MB/sec execute 315 sec latency 100.412 ms 2024-04-24T23:55:58.415 INFO:tasks.workunit.client.1.smithi169.stdout: 10 165649 19.96 MB/sec execute 316 sec latency 122.326 ms 2024-04-24T23:55:58.679 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:55:58.682 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:55:58.746 INFO:tasks.ceph.mds.b.smithi169.stderr:2024-04-24T23:55:58.745+0000 7f12ddd9e640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 39740) UID: 0 2024-04-24T23:55:58.746 INFO:tasks.ceph.mds.d.smithi169.stderr:2024-04-24T23:55:58.746+0000 7fcbfaa77640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 39740) UID: 0 2024-04-24T23:55:59.415 INFO:tasks.workunit.client.1.smithi169.stdout: 10 165691 19.91 MB/sec execute 317 sec latency 1042.015 ms 2024-04-24T23:56:00.415 INFO:tasks.workunit.client.1.smithi169.stdout: 10 165711 19.84 MB/sec execute 318 sec latency 2042.053 ms 2024-04-24T23:56:01.415 INFO:tasks.workunit.client.1.smithi169.stdout: 10 165723 19.79 MB/sec execute 319 sec latency 2138.611 ms 2024-04-24T23:56:02.415 INFO:tasks.workunit.client.1.smithi169.stdout: 10 165868 19.76 MB/sec execute 320 sec latency 923.267 ms 2024-04-24T23:56:03.415 INFO:tasks.workunit.client.1.smithi169.stdout: 10 166067 19.75 MB/sec execute 321 sec latency 93.760 ms 2024-04-24T23:56:04.415 INFO:tasks.workunit.client.1.smithi169.stdout: 10 166291 19.72 MB/sec execute 322 sec latency 155.183 ms 2024-04-24T23:56:05.415 INFO:tasks.workunit.client.1.smithi169.stdout: 10 166550 19.69 MB/sec execute 323 sec latency 150.151 ms 2024-04-24T23:56:06.415 INFO:tasks.workunit.client.1.smithi169.stdout: 10 166727 19.65 MB/sec execute 324 sec latency 121.100 ms 2024-04-24T23:56:07.415 INFO:tasks.workunit.client.1.smithi169.stdout: 10 167031 19.64 MB/sec execute 325 sec latency 112.557 ms 2024-04-24T23:56:08.415 INFO:tasks.workunit.client.1.smithi169.stdout: 10 167381 19.63 MB/sec execute 326 sec latency 154.682 ms 2024-04-24T23:56:09.415 INFO:tasks.workunit.client.1.smithi169.stdout: 10 167818 19.65 MB/sec execute 327 sec latency 177.742 ms 2024-04-24T23:56:10.415 INFO:tasks.workunit.client.1.smithi169.stdout: 10 168377 19.66 MB/sec execute 328 sec latency 127.340 ms 2024-04-24T23:56:11.415 INFO:tasks.workunit.client.1.smithi169.stdout: 10 168887 19.69 MB/sec execute 329 sec latency 61.265 ms 2024-04-24T23:56:12.415 INFO:tasks.workunit.client.1.smithi169.stdout: 10 169182 19.69 MB/sec execute 330 sec latency 158.101 ms 2024-04-24T23:56:13.416 INFO:tasks.workunit.client.1.smithi169.stdout: 10 169645 19.74 MB/sec execute 331 sec latency 56.348 ms 2024-04-24T23:56:14.416 INFO:tasks.workunit.client.1.smithi169.stdout: 10 169965 19.73 MB/sec execute 332 sec latency 136.825 ms 2024-04-24T23:56:15.416 INFO:tasks.workunit.client.1.smithi169.stdout: 10 170290 19.69 MB/sec execute 333 sec latency 161.626 ms 2024-04-24T23:56:16.423 INFO:tasks.workunit.client.1.smithi169.stdout: 10 170647 19.70 MB/sec execute 334 sec latency 82.447 ms 2024-04-24T23:56:17.423 INFO:tasks.workunit.client.1.smithi169.stdout: 10 171002 19.69 MB/sec execute 335 sec latency 157.722 ms 2024-04-24T23:56:18.423 INFO:tasks.workunit.client.1.smithi169.stdout: 10 171446 19.70 MB/sec execute 336 sec latency 108.577 ms 2024-04-24T23:56:19.423 INFO:tasks.workunit.client.1.smithi169.stdout: 10 171906 19.70 MB/sec execute 337 sec latency 187.744 ms 2024-04-24T23:56:20.423 INFO:tasks.workunit.client.1.smithi169.stdout: 10 172382 19.72 MB/sec execute 338 sec latency 64.684 ms 2024-04-24T23:56:21.423 INFO:tasks.workunit.client.1.smithi169.stdout: 10 172762 19.75 MB/sec execute 339 sec latency 142.995 ms 2024-04-24T23:56:22.430 INFO:tasks.workunit.client.1.smithi169.stdout: 10 173200 19.78 MB/sec execute 340 sec latency 136.519 ms 2024-04-24T23:56:23.430 INFO:tasks.workunit.client.1.smithi169.stdout: 10 173573 19.77 MB/sec execute 341 sec latency 96.442 ms 2024-04-24T23:56:24.430 INFO:tasks.workunit.client.1.smithi169.stdout: 10 173901 19.75 MB/sec execute 342 sec latency 117.122 ms 2024-04-24T23:56:25.430 INFO:tasks.workunit.client.1.smithi169.stdout: 10 174238 19.75 MB/sec execute 343 sec latency 111.921 ms 2024-04-24T23:56:26.434 INFO:tasks.workunit.client.1.smithi169.stdout: 10 174624 19.74 MB/sec execute 344 sec latency 98.440 ms 2024-04-24T23:56:27.434 INFO:tasks.workunit.client.1.smithi169.stdout: 10 175083 19.75 MB/sec execute 345 sec latency 126.170 ms 2024-04-24T23:56:28.434 INFO:tasks.workunit.client.1.smithi169.stdout: 10 175611 19.77 MB/sec execute 346 sec latency 75.287 ms 2024-04-24T23:56:29.434 INFO:tasks.workunit.client.1.smithi169.stdout: 10 176095 19.78 MB/sec execute 347 sec latency 69.617 ms 2024-04-24T23:56:30.434 INFO:tasks.workunit.client.1.smithi169.stdout: 10 176412 19.79 MB/sec execute 348 sec latency 136.765 ms 2024-04-24T23:56:31.434 INFO:tasks.workunit.client.1.smithi169.stdout: 10 176882 19.84 MB/sec execute 349 sec latency 103.739 ms 2024-04-24T23:56:32.434 INFO:tasks.workunit.client.1.smithi169.stdout: 10 177248 19.82 MB/sec execute 350 sec latency 93.366 ms 2024-04-24T23:56:33.434 INFO:tasks.workunit.client.1.smithi169.stdout: 10 177611 19.81 MB/sec execute 351 sec latency 102.317 ms 2024-04-24T23:56:34.434 INFO:tasks.workunit.client.1.smithi169.stdout: 10 177899 19.80 MB/sec execute 352 sec latency 313.274 ms 2024-04-24T23:56:35.434 INFO:tasks.workunit.client.1.smithi169.stdout: 10 177911 19.75 MB/sec execute 353 sec latency 1285.658 ms 2024-04-24T23:56:36.434 INFO:tasks.workunit.client.1.smithi169.stdout: 10 177915 19.69 MB/sec execute 354 sec latency 2050.099 ms 2024-04-24T23:56:37.434 INFO:tasks.workunit.client.1.smithi169.stdout: 10 177928 19.64 MB/sec execute 355 sec latency 2273.375 ms 2024-04-24T23:56:38.434 INFO:tasks.workunit.client.1.smithi169.stdout: 10 178075 19.60 MB/sec execute 356 sec latency 2008.301 ms 2024-04-24T23:56:39.434 INFO:tasks.workunit.client.1.smithi169.stdout: 10 178275 19.58 MB/sec execute 357 sec latency 260.305 ms 2024-04-24T23:56:40.435 INFO:tasks.workunit.client.1.smithi169.stdout: 10 178475 19.55 MB/sec execute 358 sec latency 134.627 ms 2024-04-24T23:56:41.434 INFO:tasks.workunit.client.1.smithi169.stdout: 10 178713 19.53 MB/sec execute 359 sec latency 109.201 ms 2024-04-24T23:56:42.434 INFO:tasks.workunit.client.1.smithi169.stdout: 10 179046 19.52 MB/sec execute 360 sec latency 98.230 ms 2024-04-24T23:56:43.434 INFO:tasks.workunit.client.1.smithi169.stdout: 10 179536 19.53 MB/sec execute 361 sec latency 86.632 ms 2024-04-24T23:56:44.434 INFO:tasks.workunit.client.1.smithi169.stdout: 10 179906 19.56 MB/sec execute 362 sec latency 94.856 ms 2024-04-24T23:56:45.434 INFO:tasks.workunit.client.1.smithi169.stdout: 10 180407 19.58 MB/sec execute 363 sec latency 68.141 ms 2024-04-24T23:56:46.434 INFO:tasks.workunit.client.1.smithi169.stdout: 10 180804 19.58 MB/sec execute 364 sec latency 77.120 ms 2024-04-24T23:56:47.435 INFO:tasks.workunit.client.1.smithi169.stdout: 10 181116 19.56 MB/sec execute 365 sec latency 79.349 ms 2024-04-24T23:56:48.435 INFO:tasks.workunit.client.1.smithi169.stdout: 10 181479 19.57 MB/sec execute 366 sec latency 110.064 ms 2024-04-24T23:56:49.436 INFO:tasks.workunit.client.1.smithi169.stdout: 10 181831 19.57 MB/sec execute 367 sec latency 133.097 ms 2024-04-24T23:56:50.435 INFO:tasks.workunit.client.1.smithi169.stdout: 10 182359 19.59 MB/sec execute 368 sec latency 77.607 ms 2024-04-24T23:56:51.435 INFO:tasks.workunit.client.1.smithi169.stdout: 10 182971 19.61 MB/sec execute 369 sec latency 65.125 ms 2024-04-24T23:56:52.435 INFO:tasks.workunit.client.1.smithi169.stdout: 10 183360 19.63 MB/sec execute 370 sec latency 104.192 ms 2024-04-24T23:56:53.435 INFO:tasks.workunit.client.1.smithi169.stdout: 10 183810 19.65 MB/sec execute 371 sec latency 95.131 ms 2024-04-24T23:56:54.435 INFO:tasks.workunit.client.1.smithi169.stdout: 10 184240 19.66 MB/sec execute 372 sec latency 72.960 ms 2024-04-24T23:56:55.435 INFO:tasks.workunit.client.1.smithi169.stdout: 10 184353 19.63 MB/sec execute 373 sec latency 797.067 ms 2024-04-24T23:56:56.435 INFO:tasks.workunit.client.1.smithi169.stdout: 10 184372 19.58 MB/sec execute 374 sec latency 1516.699 ms 2024-04-24T23:56:57.435 INFO:tasks.workunit.client.1.smithi169.stdout: 10 184488 19.54 MB/sec execute 375 sec latency 843.394 ms 2024-04-24T23:56:58.435 INFO:tasks.workunit.client.1.smithi169.stdout: 10 184624 19.50 MB/sec execute 376 sec latency 148.770 ms 2024-04-24T23:56:59.435 INFO:tasks.workunit.client.1.smithi169.stdout: 10 184939 19.50 MB/sec execute 377 sec latency 151.908 ms 2024-04-24T23:57:00.435 INFO:tasks.workunit.client.1.smithi169.stdout: 10 185313 19.49 MB/sec execute 378 sec latency 139.771 ms 2024-04-24T23:57:01.435 INFO:tasks.workunit.client.1.smithi169.stdout: 10 185765 19.50 MB/sec execute 379 sec latency 101.206 ms 2024-04-24T23:57:02.436 INFO:tasks.workunit.client.1.smithi169.stdout: 10 186320 19.51 MB/sec execute 380 sec latency 50.909 ms 2024-04-24T23:57:03.436 INFO:tasks.workunit.client.1.smithi169.stdout: 10 186778 19.53 MB/sec execute 381 sec latency 86.110 ms 2024-04-24T23:57:04.435 INFO:tasks.workunit.client.1.smithi169.stdout: 10 187204 19.56 MB/sec execute 382 sec latency 131.391 ms 2024-04-24T23:57:05.435 INFO:tasks.workunit.client.1.smithi169.stdout: 10 187717 19.59 MB/sec execute 383 sec latency 115.250 ms 2024-04-24T23:57:06.435 INFO:tasks.workunit.client.1.smithi169.stdout: 10 188069 19.58 MB/sec execute 384 sec latency 122.826 ms 2024-04-24T23:57:07.435 INFO:tasks.workunit.client.1.smithi169.stdout: 10 188468 19.59 MB/sec execute 385 sec latency 120.102 ms 2024-04-24T23:57:08.436 INFO:tasks.workunit.client.1.smithi169.stdout: 10 188792 19.58 MB/sec execute 386 sec latency 122.895 ms 2024-04-24T23:57:09.435 INFO:tasks.workunit.client.1.smithi169.stdout: 10 189271 19.58 MB/sec execute 387 sec latency 71.287 ms 2024-04-24T23:57:10.436 INFO:tasks.workunit.client.1.smithi169.stdout: 10 189766 19.60 MB/sec execute 388 sec latency 110.685 ms 2024-04-24T23:57:11.435 INFO:tasks.workunit.client.1.smithi169.stdout: 10 190267 19.61 MB/sec execute 389 sec latency 96.683 ms 2024-04-24T23:57:12.436 INFO:tasks.workunit.client.1.smithi169.stdout: 10 190725 19.65 MB/sec execute 390 sec latency 104.696 ms 2024-04-24T23:57:13.436 INFO:tasks.workunit.client.1.smithi169.stdout: 10 191217 19.68 MB/sec execute 391 sec latency 100.408 ms 2024-04-24T23:57:14.436 INFO:tasks.workunit.client.1.smithi169.stdout: 10 191538 19.66 MB/sec execute 392 sec latency 76.539 ms 2024-04-24T23:57:15.439 INFO:tasks.workunit.client.1.smithi169.stdout: 10 191783 19.64 MB/sec execute 393 sec latency 270.541 ms 2024-04-24T23:57:16.439 INFO:tasks.workunit.client.1.smithi169.stdout: 10 191811 19.59 MB/sec execute 394 sec latency 1270.579 ms 2024-04-24T23:57:17.440 INFO:tasks.workunit.client.1.smithi169.stdout: 10 191866 19.55 MB/sec execute 395 sec latency 1435.438 ms 2024-04-24T23:57:18.440 INFO:tasks.workunit.client.1.smithi169.stdout: 10 192027 19.53 MB/sec execute 396 sec latency 361.244 ms 2024-04-24T23:57:19.440 INFO:tasks.workunit.client.1.smithi169.stdout: 10 192136 19.49 MB/sec execute 397 sec latency 229.001 ms 2024-04-24T23:57:20.440 INFO:tasks.workunit.client.1.smithi169.stdout: 10 192347 19.47 MB/sec execute 398 sec latency 222.487 ms 2024-04-24T23:57:21.440 INFO:tasks.workunit.client.1.smithi169.stdout: 10 192823 19.48 MB/sec execute 399 sec latency 94.997 ms 2024-04-24T23:57:22.440 INFO:tasks.workunit.client.1.smithi169.stdout: 10 193391 19.49 MB/sec execute 400 sec latency 83.937 ms 2024-04-24T23:57:23.440 INFO:tasks.workunit.client.1.smithi169.stdout: 10 193768 19.49 MB/sec execute 401 sec latency 121.786 ms 2024-04-24T23:57:24.440 INFO:tasks.workunit.client.1.smithi169.stdout: 10 194279 19.54 MB/sec execute 402 sec latency 125.096 ms 2024-04-24T23:57:25.440 INFO:tasks.workunit.client.1.smithi169.stdout: 10 194738 19.57 MB/sec execute 403 sec latency 138.851 ms 2024-04-24T23:57:26.440 INFO:tasks.workunit.client.1.smithi169.stdout: 10 195045 19.54 MB/sec execute 404 sec latency 115.668 ms 2024-04-24T23:57:27.440 INFO:tasks.workunit.client.1.smithi169.stdout: 10 195372 19.53 MB/sec execute 405 sec latency 94.907 ms 2024-04-24T23:57:28.440 INFO:tasks.workunit.client.1.smithi169.stdout: 10 195687 19.53 MB/sec execute 406 sec latency 106.836 ms 2024-04-24T23:57:29.440 INFO:tasks.workunit.client.1.smithi169.stdout: 10 196108 19.53 MB/sec execute 407 sec latency 133.156 ms 2024-04-24T23:57:30.440 INFO:tasks.workunit.client.1.smithi169.stdout: 10 196653 19.56 MB/sec execute 408 sec latency 42.692 ms 2024-04-24T23:57:31.440 INFO:tasks.workunit.client.1.smithi169.stdout: 10 197128 19.55 MB/sec execute 409 sec latency 111.173 ms 2024-04-24T23:57:32.440 INFO:tasks.workunit.client.1.smithi169.stdout: 10 197594 19.59 MB/sec execute 410 sec latency 90.592 ms 2024-04-24T23:57:33.440 INFO:tasks.workunit.client.1.smithi169.stdout: 10 198070 19.61 MB/sec execute 411 sec latency 96.106 ms 2024-04-24T23:57:34.440 INFO:tasks.workunit.client.1.smithi169.stdout: 10 198442 19.62 MB/sec execute 412 sec latency 68.606 ms 2024-04-24T23:57:35.440 INFO:tasks.workunit.client.1.smithi169.stdout: 10 198727 19.60 MB/sec execute 413 sec latency 110.020 ms 2024-04-24T23:57:36.440 INFO:tasks.workunit.client.1.smithi169.stdout: 10 198791 19.56 MB/sec execute 414 sec latency 858.123 ms 2024-04-24T23:57:37.440 INFO:tasks.workunit.client.1.smithi169.stdout: 10 198795 19.51 MB/sec execute 415 sec latency 1796.654 ms 2024-04-24T23:57:38.440 INFO:tasks.workunit.client.1.smithi169.stdout: 10 198941 19.48 MB/sec execute 416 sec latency 1921.496 ms 2024-04-24T23:57:39.440 INFO:tasks.workunit.client.1.smithi169.stdout: 10 199052 19.44 MB/sec execute 417 sec latency 322.423 ms 2024-04-24T23:57:40.440 INFO:tasks.workunit.client.1.smithi169.stdout: 10 199217 19.42 MB/sec execute 418 sec latency 155.341 ms 2024-04-24T23:57:41.440 INFO:tasks.workunit.client.1.smithi169.stdout: 10 199529 19.41 MB/sec execute 419 sec latency 166.210 ms 2024-04-24T23:57:42.440 INFO:tasks.workunit.client.1.smithi169.stdout: 10 200060 19.44 MB/sec execute 420 sec latency 91.187 ms 2024-04-24T23:57:43.440 INFO:tasks.workunit.client.1.smithi169.stdout: 10 200565 19.43 MB/sec execute 421 sec latency 118.435 ms 2024-04-24T23:57:44.440 INFO:tasks.workunit.client.1.smithi169.stdout: 10 200928 19.44 MB/sec execute 422 sec latency 113.323 ms 2024-04-24T23:57:45.440 INFO:tasks.workunit.client.1.smithi169.stdout: 10 201476 19.49 MB/sec execute 423 sec latency 58.982 ms 2024-04-24T23:57:46.441 INFO:tasks.workunit.client.1.smithi169.stdout: 10 201872 19.50 MB/sec execute 424 sec latency 109.808 ms 2024-04-24T23:57:47.441 INFO:tasks.workunit.client.1.smithi169.stdout: 10 202193 19.49 MB/sec execute 425 sec latency 120.649 ms 2024-04-24T23:57:48.440 INFO:tasks.workunit.client.1.smithi169.stdout: 10 202544 19.48 MB/sec execute 426 sec latency 94.198 ms 2024-04-24T23:57:49.441 INFO:tasks.workunit.client.1.smithi169.stdout: 10 202906 19.47 MB/sec execute 427 sec latency 114.386 ms 2024-04-24T23:57:50.441 INFO:tasks.workunit.client.1.smithi169.stdout: 10 203449 19.50 MB/sec execute 428 sec latency 81.565 ms 2024-04-24T23:57:51.441 INFO:tasks.workunit.client.1.smithi169.stdout: 10 204005 19.51 MB/sec execute 429 sec latency 64.669 ms 2024-04-24T23:57:52.441 INFO:tasks.workunit.client.1.smithi169.stdout: 10 204505 19.54 MB/sec execute 430 sec latency 66.685 ms 2024-04-24T23:57:53.441 INFO:tasks.workunit.client.1.smithi169.stdout: 10 204951 19.57 MB/sec execute 431 sec latency 106.750 ms 2024-04-24T23:57:54.441 INFO:tasks.workunit.client.1.smithi169.stdout: 10 205385 19.58 MB/sec execute 432 sec latency 64.047 ms 2024-04-24T23:57:55.441 INFO:tasks.workunit.client.1.smithi169.stdout: 10 205683 19.56 MB/sec execute 433 sec latency 129.782 ms 2024-04-24T23:57:56.441 INFO:tasks.workunit.client.1.smithi169.stdout: 10 205973 19.54 MB/sec execute 434 sec latency 400.496 ms 2024-04-24T23:57:57.441 INFO:tasks.workunit.client.1.smithi169.stdout: 10 205983 19.50 MB/sec execute 435 sec latency 1341.393 ms 2024-04-24T23:57:58.441 INFO:tasks.workunit.client.1.smithi169.stdout: 10 206017 19.46 MB/sec execute 436 sec latency 1484.679 ms 2024-04-24T23:57:59.441 INFO:tasks.workunit.client.1.smithi169.stdout: 10 206167 19.43 MB/sec execute 437 sec latency 332.675 ms 2024-04-24T23:58:00.441 INFO:tasks.workunit.client.1.smithi169.stdout: 10 206391 19.42 MB/sec execute 438 sec latency 174.426 ms 2024-04-24T23:58:01.441 INFO:tasks.workunit.client.1.smithi169.stdout: 10 206719 19.41 MB/sec execute 439 sec latency 118.762 ms 2024-04-24T23:58:02.441 INFO:tasks.workunit.client.1.smithi169.stdout: 10 207229 19.43 MB/sec execute 440 sec latency 94.795 ms 2024-04-24T23:58:03.441 INFO:tasks.workunit.client.1.smithi169.stdout: 10 207738 19.43 MB/sec execute 441 sec latency 118.785 ms 2024-04-24T23:58:04.441 INFO:tasks.workunit.client.1.smithi169.stdout: 10 208071 19.44 MB/sec execute 442 sec latency 188.057 ms 2024-04-24T23:58:05.441 INFO:tasks.workunit.client.1.smithi169.stdout: 10 208430 19.45 MB/sec execute 443 sec latency 139.485 ms 2024-04-24T23:58:06.441 INFO:tasks.workunit.client.1.smithi169.stdout: 10 208819 19.47 MB/sec execute 444 sec latency 128.013 ms 2024-04-24T23:58:07.441 INFO:tasks.workunit.client.1.smithi169.stdout: 10 209173 19.45 MB/sec execute 445 sec latency 122.724 ms 2024-04-24T23:58:08.441 INFO:tasks.workunit.client.1.smithi169.stdout: 10 209522 19.45 MB/sec execute 446 sec latency 101.186 ms 2024-04-24T23:58:09.441 INFO:tasks.workunit.client.1.smithi169.stdout: 10 209870 19.44 MB/sec execute 447 sec latency 138.916 ms 2024-04-24T23:58:10.441 INFO:tasks.workunit.client.1.smithi169.stdout: 10 210357 19.46 MB/sec execute 448 sec latency 78.473 ms 2024-04-24T23:58:11.441 INFO:tasks.workunit.client.1.smithi169.stdout: 10 210964 19.49 MB/sec execute 449 sec latency 44.409 ms 2024-04-24T23:58:12.441 INFO:tasks.workunit.client.1.smithi169.stdout: 10 211435 19.49 MB/sec execute 450 sec latency 91.756 ms 2024-04-24T23:58:13.441 INFO:tasks.workunit.client.1.smithi169.stdout: 10 211772 19.50 MB/sec execute 451 sec latency 131.837 ms 2024-04-24T23:58:14.441 INFO:tasks.workunit.client.1.smithi169.stdout: 10 212277 19.54 MB/sec execute 452 sec latency 45.740 ms 2024-04-24T23:58:15.441 INFO:tasks.workunit.client.1.smithi169.stdout: 10 212678 19.53 MB/sec execute 453 sec latency 113.967 ms 2024-04-24T23:58:16.441 INFO:tasks.workunit.client.1.smithi169.stdout: 10 212982 19.52 MB/sec execute 454 sec latency 129.812 ms 2024-04-24T23:58:17.441 INFO:tasks.workunit.client.1.smithi169.stdout: 10 213024 19.48 MB/sec execute 455 sec latency 922.946 ms 2024-04-24T23:58:18.442 INFO:tasks.workunit.client.1.smithi169.stdout: 10 213040 19.44 MB/sec execute 456 sec latency 1755.041 ms 2024-04-24T23:58:19.442 INFO:tasks.workunit.client.1.smithi169.stdout: 10 213268 19.41 MB/sec execute 457 sec latency 735.493 ms 2024-04-24T23:58:20.442 INFO:tasks.workunit.client.1.smithi169.stdout: 10 213453 19.40 MB/sec execute 458 sec latency 167.983 ms 2024-04-24T23:58:21.442 INFO:tasks.workunit.client.1.smithi169.stdout: 10 213703 19.38 MB/sec execute 459 sec latency 171.881 ms 2024-04-24T23:58:22.442 INFO:tasks.workunit.client.1.smithi169.stdout: 10 213951 19.37 MB/sec execute 460 sec latency 118.997 ms 2024-04-24T23:58:23.442 INFO:tasks.workunit.client.1.smithi169.stdout: 10 214499 19.39 MB/sec execute 461 sec latency 66.229 ms 2024-04-24T23:58:24.442 INFO:tasks.workunit.client.1.smithi169.stdout: 10 215001 19.40 MB/sec execute 462 sec latency 168.611 ms 2024-04-24T23:58:25.442 INFO:tasks.workunit.client.1.smithi169.stdout: 10 215307 19.39 MB/sec execute 463 sec latency 160.223 ms 2024-04-24T23:58:26.442 INFO:tasks.workunit.client.1.smithi169.stdout: 10 215768 19.43 MB/sec execute 464 sec latency 99.791 ms 2024-04-24T23:58:27.442 INFO:tasks.workunit.client.1.smithi169.stdout: 10 216181 19.43 MB/sec execute 465 sec latency 127.947 ms 2024-04-24T23:58:28.443 INFO:tasks.workunit.client.1.smithi169.stdout: 10 216487 19.41 MB/sec execute 466 sec latency 85.718 ms 2024-04-24T23:58:29.442 INFO:tasks.workunit.client.1.smithi169.stdout: 10 216825 19.41 MB/sec execute 467 sec latency 139.945 ms 2024-04-24T23:58:30.447 INFO:tasks.workunit.client.1.smithi169.stdout: 10 217244 19.42 MB/sec execute 468 sec latency 103.912 ms 2024-04-24T23:58:31.446 INFO:tasks.workunit.client.1.smithi169.stdout: 10 217742 19.42 MB/sec execute 469 sec latency 115.185 ms 2024-04-24T23:58:32.446 INFO:tasks.workunit.client.1.smithi169.stdout: 10 218204 19.44 MB/sec execute 470 sec latency 94.468 ms 2024-04-24T23:58:33.447 INFO:tasks.workunit.client.1.smithi169.stdout: 10 218723 19.45 MB/sec execute 471 sec latency 91.790 ms 2024-04-24T23:58:34.447 INFO:tasks.workunit.client.1.smithi169.stdout: 10 219072 19.47 MB/sec execute 472 sec latency 155.990 ms 2024-04-24T23:58:35.447 INFO:tasks.workunit.client.1.smithi169.stdout: 10 219528 19.49 MB/sec execute 473 sec latency 72.736 ms 2024-04-24T23:58:36.447 INFO:tasks.workunit.client.1.smithi169.stdout: 10 219839 19.48 MB/sec execute 474 sec latency 107.494 ms 2024-04-24T23:58:37.447 INFO:tasks.workunit.client.1.smithi169.stdout: 10 220061 19.45 MB/sec execute 475 sec latency 293.822 ms 2024-04-24T23:58:38.447 INFO:tasks.workunit.client.1.smithi169.stdout: 10 220073 19.41 MB/sec execute 476 sec latency 1146.123 ms 2024-04-24T23:58:39.447 INFO:tasks.workunit.client.1.smithi169.stdout: 10 220113 19.37 MB/sec execute 477 sec latency 1511.780 ms 2024-04-24T23:58:40.447 INFO:tasks.workunit.client.1.smithi169.stdout: 10 220326 19.36 MB/sec execute 478 sec latency 140.350 ms 2024-04-24T23:58:41.448 INFO:tasks.workunit.client.1.smithi169.stdout: 10 220574 19.35 MB/sec execute 479 sec latency 95.968 ms 2024-04-24T23:58:42.447 INFO:tasks.workunit.client.1.smithi169.stdout: 10 220847 19.35 MB/sec execute 480 sec latency 113.508 ms 2024-04-24T23:58:43.447 INFO:tasks.workunit.client.1.smithi169.stdout: 10 221333 19.34 MB/sec execute 481 sec latency 124.625 ms 2024-04-24T23:58:44.454 INFO:tasks.workunit.client.1.smithi169.stdout: 10 221790 19.36 MB/sec execute 482 sec latency 59.015 ms 2024-04-24T23:58:45.449 INFO:tasks.workunit.client.1.smithi169.stdout: 10 222186 19.35 MB/sec execute 483 sec latency 170.637 ms 2024-04-24T23:58:46.449 INFO:tasks.workunit.client.1.smithi169.stdout: 10 222589 19.38 MB/sec execute 484 sec latency 129.528 ms 2024-04-24T23:58:47.449 INFO:tasks.workunit.client.1.smithi169.stdout: 10 223006 19.40 MB/sec execute 485 sec latency 139.630 ms 2024-04-24T23:58:48.450 INFO:tasks.workunit.client.1.smithi169.stdout: 10 223373 19.39 MB/sec execute 486 sec latency 111.822 ms 2024-04-24T23:58:49.450 INFO:tasks.workunit.client.1.smithi169.stdout: 10 223591 19.36 MB/sec execute 487 sec latency 340.262 ms 2024-04-24T23:58:50.450 INFO:tasks.workunit.client.1.smithi169.stdout: 10 223980 19.37 MB/sec execute 488 sec latency 109.238 ms 2024-04-24T23:58:51.450 INFO:tasks.workunit.client.1.smithi169.stdout: 10 224420 19.38 MB/sec execute 489 sec latency 86.614 ms 2024-04-24T23:58:52.450 INFO:tasks.workunit.client.1.smithi169.stdout: 10 224883 19.37 MB/sec execute 490 sec latency 100.380 ms 2024-04-24T23:58:53.450 INFO:tasks.workunit.client.1.smithi169.stdout: 10 224912 19.34 MB/sec execute 491 sec latency 1037.084 ms 2024-04-24T23:58:54.450 INFO:tasks.workunit.client.1.smithi169.stdout: 10 225154 19.33 MB/sec execute 492 sec latency 1232.014 ms 2024-04-24T23:58:55.450 INFO:tasks.workunit.client.1.smithi169.stdout: 10 225486 19.33 MB/sec execute 493 sec latency 115.007 ms 2024-04-24T23:58:56.450 INFO:tasks.workunit.client.1.smithi169.stdout: 10 225852 19.33 MB/sec execute 494 sec latency 183.372 ms 2024-04-24T23:58:57.450 INFO:tasks.workunit.client.1.smithi169.stdout: 10 226279 19.35 MB/sec execute 495 sec latency 114.768 ms 2024-04-24T23:58:58.450 INFO:tasks.workunit.client.1.smithi169.stdout: 10 226749 19.36 MB/sec execute 496 sec latency 107.899 ms 2024-04-24T23:58:59.450 INFO:tasks.workunit.client.1.smithi169.stdout: 10 227095 19.36 MB/sec execute 497 sec latency 200.755 ms 2024-04-24T23:59:00.450 INFO:tasks.workunit.client.1.smithi169.stdout: 10 227460 19.36 MB/sec execute 498 sec latency 85.849 ms 2024-04-24T23:59:01.450 INFO:tasks.workunit.client.1.smithi169.stdout: 10 227863 19.36 MB/sec execute 499 sec latency 97.077 ms 2024-04-24T23:59:02.452 INFO:tasks.workunit.client.1.smithi169.stdout: 10 228390 19.37 MB/sec execute 500 sec latency 100.215 ms 2024-04-24T23:59:03.452 INFO:tasks.workunit.client.1.smithi169.stdout: 10 228954 19.39 MB/sec execute 501 sec latency 40.031 ms 2024-04-24T23:59:04.452 INFO:tasks.workunit.client.1.smithi169.stdout: 10 229429 19.41 MB/sec execute 502 sec latency 114.747 ms 2024-04-24T23:59:05.452 INFO:tasks.workunit.client.1.smithi169.stdout: 10 229994 19.46 MB/sec execute 503 sec latency 77.879 ms 2024-04-24T23:59:06.452 INFO:tasks.workunit.client.1.smithi169.stdout: 10 230467 19.45 MB/sec execute 504 sec latency 74.206 ms 2024-04-24T23:59:07.453 INFO:tasks.workunit.client.1.smithi169.stdout: 10 230784 19.45 MB/sec execute 505 sec latency 71.883 ms 2024-04-24T23:59:08.453 INFO:tasks.workunit.client.1.smithi169.stdout: 10 230855 19.42 MB/sec execute 506 sec latency 726.356 ms 2024-04-24T23:59:09.455 INFO:tasks.workunit.client.1.smithi169.stdout: 10 230882 19.38 MB/sec execute 507 sec latency 1480.135 ms 2024-04-24T23:59:10.455 INFO:tasks.workunit.client.1.smithi169.stdout: 10 231049 19.35 MB/sec execute 508 sec latency 201.777 ms 2024-04-24T23:59:11.455 INFO:tasks.workunit.client.1.smithi169.stdout: 10 231196 19.34 MB/sec execute 509 sec latency 223.011 ms 2024-04-24T23:59:12.455 INFO:tasks.workunit.client.1.smithi169.stdout: 10 231566 19.33 MB/sec execute 510 sec latency 184.543 ms 2024-04-24T23:59:13.455 INFO:tasks.workunit.client.1.smithi169.stdout: 10 232091 19.34 MB/sec execute 511 sec latency 124.572 ms 2024-04-24T23:59:14.456 INFO:tasks.workunit.client.1.smithi169.stdout: 10 232705 19.36 MB/sec execute 512 sec latency 40.006 ms 2024-04-24T23:59:15.456 INFO:tasks.workunit.client.1.smithi169.stdout: 10 233155 19.38 MB/sec execute 513 sec latency 114.700 ms 2024-04-24T23:59:16.456 INFO:tasks.workunit.client.1.smithi169.stdout: 10 233658 19.42 MB/sec execute 514 sec latency 64.093 ms 2024-04-24T23:59:17.456 INFO:tasks.workunit.client.1.smithi169.stdout: 10 234098 19.42 MB/sec execute 515 sec latency 90.936 ms 2024-04-24T23:59:18.456 INFO:tasks.workunit.client.1.smithi169.stdout: 10 234401 19.41 MB/sec execute 516 sec latency 87.537 ms 2024-04-24T23:59:19.195 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:59:19.198 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:59:19.231 INFO:tasks.ceph.mds.b.smithi169.stderr:2024-04-24T23:59:19.231+0000 7f12ddd9e640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 39771) UID: 0 2024-04-24T23:59:19.232 INFO:tasks.ceph.mds.d.smithi169.stderr:2024-04-24T23:59:19.231+0000 7fcbfaa77640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 39771) UID: 0 2024-04-24T23:59:19.456 INFO:tasks.workunit.client.1.smithi169.stdout: 10 234731 19.40 MB/sec execute 517 sec latency 103.686 ms 2024-04-24T23:59:20.456 INFO:tasks.workunit.client.1.smithi169.stdout: 10 235191 19.41 MB/sec execute 518 sec latency 91.906 ms 2024-04-24T23:59:21.456 INFO:tasks.workunit.client.1.smithi169.stdout: 10 235815 19.42 MB/sec execute 519 sec latency 63.214 ms 2024-04-24T23:59:22.456 INFO:tasks.workunit.client.1.smithi169.stdout: 10 236276 19.43 MB/sec execute 520 sec latency 75.927 ms 2024-04-24T23:59:23.456 INFO:tasks.workunit.client.1.smithi169.stdout: 10 236741 19.46 MB/sec execute 521 sec latency 145.612 ms 2024-04-24T23:59:24.456 INFO:tasks.workunit.client.1.smithi169.stdout: 10 237205 19.49 MB/sec execute 522 sec latency 112.686 ms 2024-04-24T23:59:25.456 INFO:tasks.workunit.client.1.smithi169.stdout: 10 237573 19.48 MB/sec execute 523 sec latency 119.294 ms 2024-04-24T23:59:26.456 INFO:tasks.workunit.client.1.smithi169.stdout: 10 237802 19.46 MB/sec execute 524 sec latency 150.820 ms 2024-04-24T23:59:27.456 INFO:tasks.workunit.client.1.smithi169.stdout: 10 238109 19.45 MB/sec execute 525 sec latency 157.017 ms 2024-04-24T23:59:28.456 INFO:tasks.workunit.client.1.smithi169.stdout: 10 238351 19.44 MB/sec execute 526 sec latency 260.449 ms 2024-04-24T23:59:29.456 INFO:tasks.workunit.client.1.smithi169.stdout: 10 238359 19.41 MB/sec execute 527 sec latency 1216.448 ms 2024-04-24T23:59:30.456 INFO:tasks.workunit.client.1.smithi169.stdout: 10 238387 19.38 MB/sec execute 528 sec latency 1678.922 ms 2024-04-24T23:59:31.456 INFO:tasks.workunit.client.1.smithi169.stdout: 10 238671 19.37 MB/sec execute 529 sec latency 198.396 ms 2024-04-24T23:59:32.456 INFO:tasks.workunit.client.1.smithi169.stdout: 10 238944 19.35 MB/sec execute 530 sec latency 102.820 ms 2024-04-24T23:59:33.457 INFO:tasks.workunit.client.1.smithi169.stdout: 10 239186 19.34 MB/sec execute 531 sec latency 127.063 ms 2024-04-24T23:59:34.456 INFO:tasks.workunit.client.1.smithi169.stdout: 10 239658 19.34 MB/sec execute 532 sec latency 116.734 ms 2024-04-24T23:59:35.456 INFO:tasks.workunit.client.1.smithi169.stdout: 10 240053 19.34 MB/sec execute 533 sec latency 125.995 ms 2024-04-24T23:59:36.456 INFO:tasks.workunit.client.1.smithi169.stdout: 10 240506 19.38 MB/sec execute 534 sec latency 136.988 ms 2024-04-24T23:59:37.458 INFO:tasks.workunit.client.1.smithi169.stdout: 10 240954 19.39 MB/sec execute 535 sec latency 93.241 ms 2024-04-24T23:59:38.457 INFO:tasks.workunit.client.1.smithi169.stdout: 10 241311 19.38 MB/sec execute 536 sec latency 95.186 ms 2024-04-24T23:59:39.456 INFO:tasks.workunit.client.1.smithi169.stdout: 10 241590 19.37 MB/sec execute 537 sec latency 125.453 ms 2024-04-24T23:59:40.456 INFO:tasks.workunit.client.1.smithi169.stdout: 10 241951 19.37 MB/sec execute 538 sec latency 103.229 ms 2024-04-24T23:59:41.456 INFO:tasks.workunit.client.1.smithi169.stdout: 10 242386 19.37 MB/sec execute 539 sec latency 93.517 ms 2024-04-24T23:59:42.456 INFO:tasks.workunit.client.1.smithi169.stdout: 10 242952 19.38 MB/sec execute 540 sec latency 71.948 ms 2024-04-24T23:59:43.456 INFO:tasks.workunit.client.1.smithi169.stdout: 10 243463 19.39 MB/sec execute 541 sec latency 82.443 ms 2024-04-24T23:59:44.457 INFO:tasks.workunit.client.1.smithi169.stdout: 10 243555 19.37 MB/sec execute 542 sec latency 877.503 ms 2024-04-24T23:59:45.457 INFO:tasks.workunit.client.1.smithi169.stdout: 10 243638 19.35 MB/sec execute 543 sec latency 1310.465 ms 2024-04-24T23:59:46.459 INFO:tasks.workunit.client.1.smithi169.stdout: 10 243906 19.35 MB/sec execute 544 sec latency 153.667 ms 2024-04-24T23:59:47.459 INFO:tasks.workunit.client.1.smithi169.stdout: 10 244145 19.35 MB/sec execute 545 sec latency 139.064 ms 2024-04-24T23:59:48.459 INFO:tasks.workunit.client.1.smithi169.stdout: 10 244606 19.36 MB/sec execute 546 sec latency 103.928 ms 2024-04-24T23:59:49.459 INFO:tasks.workunit.client.1.smithi169.stdout: 10 244840 19.34 MB/sec execute 547 sec latency 115.631 ms 2024-04-24T23:59:50.459 INFO:tasks.workunit.client.1.smithi169.stdout: 10 245092 19.33 MB/sec execute 548 sec latency 133.655 ms 2024-04-24T23:59:51.459 INFO:tasks.workunit.client.1.smithi169.stdout: 10 245391 19.33 MB/sec execute 549 sec latency 141.399 ms 2024-04-24T23:59:52.459 INFO:tasks.workunit.client.1.smithi169.stdout: 10 245730 19.32 MB/sec execute 550 sec latency 98.036 ms 2024-04-24T23:59:53.459 INFO:tasks.workunit.client.1.smithi169.stdout: 10 246299 19.33 MB/sec execute 551 sec latency 70.931 ms 2024-04-24T23:59:54.459 INFO:tasks.workunit.client.1.smithi169.stdout: 10 246791 19.33 MB/sec execute 552 sec latency 102.184 ms 2024-04-24T23:59:55.459 INFO:tasks.workunit.client.1.smithi169.stdout: 10 247305 19.37 MB/sec execute 553 sec latency 81.380 ms 2024-04-24T23:59:56.459 INFO:tasks.workunit.client.1.smithi169.stdout: 10 247850 19.39 MB/sec execute 554 sec latency 91.654 ms 2024-04-24T23:59:57.460 INFO:tasks.workunit.client.1.smithi169.stdout: 10 248246 19.40 MB/sec execute 555 sec latency 89.273 ms 2024-04-24T23:59:58.460 INFO:tasks.workunit.client.1.smithi169.stdout: 10 248520 19.39 MB/sec execute 556 sec latency 143.480 ms 2024-04-24T23:59:59.460 INFO:tasks.workunit.client.1.smithi169.stdout: 10 248654 19.36 MB/sec execute 557 sec latency 549.780 ms 2024-04-25T00:00:00.460 INFO:tasks.workunit.client.1.smithi169.stdout: 10 248672 19.33 MB/sec execute 558 sec latency 1455.501 ms 2024-04-25T00:00:01.460 INFO:tasks.workunit.client.1.smithi169.stdout: 10 248821 19.30 MB/sec execute 559 sec latency 1497.143 ms 2024-04-25T00:00:02.462 INFO:tasks.workunit.client.1.smithi169.stdout: 10 248984 19.29 MB/sec execute 560 sec latency 233.034 ms 2024-04-25T00:00:03.450 INFO:tasks.ceph.osd.0.smithi157.stderr:2024-04-25T00:00:03.450+0000 7f9af40bf640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 44592) UID: 0 2024-04-25T00:00:03.451 INFO:tasks.ceph.osd.1.smithi157.stderr:2024-04-25T00:00:03.450+0000 7f33b57c8640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 44592) UID: 0 2024-04-25T00:00:03.451 INFO:tasks.ceph.osd.2.smithi157.stderr:2024-04-25T00:00:03.450+0000 7fbb527f4640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 44592) UID: 0 2024-04-25T00:00:03.452 INFO:tasks.ceph.osd.3.smithi157.stderr:2024-04-25T00:00:03.450+0000 7fc5e948c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 44592) UID: 0 2024-04-25T00:00:03.461 INFO:tasks.workunit.client.1.smithi169.stdout: 10 249272 19.28 MB/sec execute 561 sec latency 239.673 ms 2024-04-25T00:00:04.462 INFO:tasks.workunit.client.1.smithi169.stdout: 10 249827 19.29 MB/sec execute 562 sec latency 41.057 ms 2024-04-25T00:00:05.462 INFO:tasks.workunit.client.1.smithi169.stdout: 10 250324 19.29 MB/sec execute 563 sec latency 121.520 ms 2024-04-25T00:00:06.462 INFO:tasks.workunit.client.1.smithi169.stdout: 10 250809 19.32 MB/sec execute 564 sec latency 93.548 ms 2024-04-25T00:00:07.462 INFO:tasks.workunit.client.1.smithi169.stdout: 10 251330 19.34 MB/sec execute 565 sec latency 103.876 ms 2024-04-25T00:00:08.462 INFO:tasks.workunit.client.1.smithi169.stdout: 10 251707 19.35 MB/sec execute 566 sec latency 68.201 ms 2024-04-25T00:00:09.462 INFO:tasks.workunit.client.1.smithi169.stdout: 10 251994 19.34 MB/sec execute 567 sec latency 120.897 ms 2024-04-25T00:00:10.462 INFO:tasks.workunit.client.1.smithi169.stdout: 10 252240 19.33 MB/sec execute 568 sec latency 172.433 ms 2024-04-25T00:00:11.462 INFO:tasks.workunit.client.1.smithi169.stdout: 10 252505 19.32 MB/sec execute 569 sec latency 115.617 ms 2024-04-25T00:00:12.462 INFO:tasks.workunit.client.1.smithi169.stdout: 10 252879 19.32 MB/sec execute 570 sec latency 131.437 ms 2024-04-25T00:00:13.462 INFO:tasks.workunit.client.1.smithi169.stdout: 10 253427 19.32 MB/sec execute 571 sec latency 88.490 ms 2024-04-25T00:00:14.462 INFO:tasks.workunit.client.1.smithi169.stdout: 10 253988 19.34 MB/sec execute 572 sec latency 81.831 ms 2024-04-25T00:00:15.462 INFO:tasks.workunit.client.1.smithi169.stdout: 10 254525 19.37 MB/sec execute 573 sec latency 89.964 ms 2024-04-25T00:00:16.462 INFO:tasks.workunit.client.1.smithi169.stdout: 10 255001 19.39 MB/sec execute 574 sec latency 68.540 ms 2024-04-25T00:00:17.462 INFO:tasks.workunit.client.1.smithi169.stdout: 10 255335 19.39 MB/sec execute 575 sec latency 96.084 ms 2024-04-25T00:00:18.462 INFO:tasks.workunit.client.1.smithi169.stdout: 10 255612 19.37 MB/sec execute 576 sec latency 102.027 ms 2024-04-25T00:00:19.462 INFO:tasks.workunit.client.1.smithi169.stdout: 10 255852 19.36 MB/sec execute 577 sec latency 150.358 ms 2024-04-25T00:00:20.463 INFO:tasks.workunit.client.1.smithi169.stdout: 10 255868 19.33 MB/sec execute 578 sec latency 1043.976 ms 2024-04-25T00:00:21.462 INFO:tasks.workunit.client.1.smithi169.stdout: 10 255895 19.30 MB/sec execute 579 sec latency 1894.570 ms 2024-04-25T00:00:22.462 INFO:tasks.workunit.client.1.smithi169.stdout: 10 256136 19.29 MB/sec execute 580 sec latency 1037.465 ms 2024-04-25T00:00:23.466 INFO:tasks.workunit.client.1.smithi169.stdout: 10 256298 19.27 MB/sec execute 581 sec latency 169.433 ms 2024-04-25T00:00:24.467 INFO:tasks.workunit.client.1.smithi169.stdout: 10 256507 19.25 MB/sec execute 582 sec latency 186.333 ms 2024-04-25T00:00:25.467 INFO:tasks.workunit.client.1.smithi169.stdout: 10 256778 19.24 MB/sec execute 583 sec latency 154.704 ms 2024-04-25T00:00:26.467 INFO:tasks.workunit.client.1.smithi169.stdout: 10 257190 19.24 MB/sec execute 584 sec latency 132.532 ms 2024-04-25T00:00:27.470 INFO:tasks.workunit.client.1.smithi169.stdout: 10 257646 19.24 MB/sec execute 585 sec latency 123.241 ms 2024-04-25T00:00:28.467 INFO:tasks.workunit.client.1.smithi169.stdout: 10 258028 19.26 MB/sec execute 586 sec latency 200.408 ms 2024-04-25T00:00:29.467 INFO:tasks.workunit.client.1.smithi169.stdout: 10 258471 19.28 MB/sec execute 587 sec latency 115.143 ms 2024-04-25T00:00:30.467 INFO:tasks.workunit.client.1.smithi169.stdout: 10 258856 19.28 MB/sec execute 588 sec latency 125.153 ms 2024-04-25T00:00:31.467 INFO:tasks.workunit.client.1.smithi169.stdout: 10 259118 19.27 MB/sec execute 589 sec latency 129.134 ms 2024-04-25T00:00:32.467 INFO:tasks.workunit.client.1.smithi169.stdout: 10 259417 19.26 MB/sec execute 590 sec latency 108.084 ms 2024-04-25T00:00:33.467 INFO:tasks.workunit.client.1.smithi169.stdout: 10 259789 19.27 MB/sec execute 591 sec latency 88.502 ms 2024-04-25T00:00:34.467 INFO:tasks.workunit.client.1.smithi169.stdout: 10 260218 19.26 MB/sec execute 592 sec latency 83.205 ms 2024-04-25T00:00:35.467 INFO:tasks.workunit.client.1.smithi169.stdout: 10 260696 19.26 MB/sec execute 593 sec latency 115.245 ms 2024-04-25T00:00:36.467 INFO:tasks.workunit.client.1.smithi169.stdout: 10 261225 19.28 MB/sec execute 594 sec latency 106.174 ms 2024-04-25T00:00:37.468 INFO:tasks.workunit.client.1.smithi169.stdout: 10 261718 19.31 MB/sec execute 595 sec latency 94.148 ms 2024-04-25T00:00:38.467 INFO:tasks.workunit.client.1.smithi169.stdout: 10 262147 19.32 MB/sec execute 596 sec latency 87.544 ms 2024-04-25T00:00:39.468 INFO:tasks.workunit.client.1.smithi169.stdout: 10 262500 19.32 MB/sec execute 597 sec latency 168.408 ms 2024-04-25T00:00:40.468 INFO:tasks.workunit.client.1.smithi169.stdout: 10 262811 19.31 MB/sec execute 598 sec latency 136.969 ms 2024-04-25T00:00:41.468 INFO:tasks.workunit.client.1.smithi169.stdout: 10 263130 19.31 MB/sec execute 599 sec latency 155.386 ms 2024-04-25T00:00:42.468 INFO:tasks.workunit.client.1.smithi169.stdout: 10 cleanup 600 sec 2024-04-25T00:00:43.468 INFO:tasks.workunit.client.1.smithi169.stdout: 6 cleanup 601 sec 2024-04-25T00:00:43.647 INFO:tasks.workunit.client.1.smithi169.stdout: 0 cleanup 601 sec 2024-04-25T00:00:43.647 INFO:tasks.workunit.client.1.smithi169.stdout: 2024-04-25T00:00:43.647 INFO:tasks.workunit.client.1.smithi169.stdout: Operation Count AvgLat MaxLat 2024-04-25T00:00:43.647 INFO:tasks.workunit.client.1.smithi169.stdout: ---------------------------------------- 2024-04-25T00:00:43.647 INFO:tasks.workunit.client.1.smithi169.stdout: NTCreateX 371500 6.470 1784.708 2024-04-25T00:00:43.647 INFO:tasks.workunit.client.1.smithi169.stdout: Close 272682 0.003 0.174 2024-04-25T00:00:43.647 INFO:tasks.workunit.client.1.smithi169.stdout: Rename 15748 26.000 1782.258 2024-04-25T00:00:43.648 INFO:tasks.workunit.client.1.smithi169.stdout: Unlink 75214 9.130 922.621 2024-04-25T00:00:43.648 INFO:tasks.workunit.client.1.smithi169.stdout: Qpathinfo 336772 4.801 1757.658 2024-04-25T00:00:43.648 INFO:tasks.workunit.client.1.smithi169.stdout: Qfileinfo 58671 0.001 0.024 2024-04-25T00:00:43.648 INFO:tasks.workunit.client.1.smithi169.stdout: Qfsinfo 61893 0.248 16.695 2024-04-25T00:00:43.648 INFO:tasks.workunit.client.1.smithi169.stdout: Sfileinfo 30224 1.046 537.353 2024-04-25T00:00:43.648 INFO:tasks.workunit.client.1.smithi169.stdout: Find 130286 0.531 386.676 2024-04-25T00:00:43.648 INFO:tasks.workunit.client.1.smithi169.stdout: WriteX 183533 0.391 1996.248 2024-04-25T00:00:43.648 INFO:tasks.workunit.client.1.smithi169.stdout: ReadX 583053 0.074 1755.032 2024-04-25T00:00:43.648 INFO:tasks.workunit.client.1.smithi169.stdout: LockX 1210 8.769 316.313 2024-04-25T00:00:43.648 INFO:tasks.workunit.client.1.smithi169.stdout: UnlockX 1210 7.762 257.062 2024-04-25T00:00:43.648 INFO:tasks.workunit.client.1.smithi169.stdout: Flush 26049 24.110 2273.365 2024-04-25T00:00:43.648 INFO:tasks.workunit.client.1.smithi169.stdout: 2024-04-25T00:00:43.648 INFO:tasks.workunit.client.1.smithi169.stdout:Throughput 19.31 MB/sec 10 clients 10 procs max_latency=2273.375 ms 2024-04-25T00:00:43.649 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-25T00:00:43.649 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-04-25T00:00:45.146 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.1... 2024-04-25T00:00:45.147 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-04-25T00:00:45.701 DEBUG:teuthology.parallel:result is None 2024-04-25T00:00:45.702 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-04-25T00:00:46.053 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-04-25T00:00:46.053 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T00:00:46.053 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T00:00:46.054 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:00:46.182 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T00:00:46.182 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:00:46.255 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:00:47.506 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.505+0000 7f60cbf90640 1 Processor -- start 2024-04-25T00:00:47.509 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.506+0000 7f60cbf90640 1 -- start start 2024-04-25T00:00:47.509 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.507+0000 7f60cbf90640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f60c40c1f80 0x7f60c4159720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:47.509 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.507+0000 7f60cbf90640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60c406c400 0x7f60c40c2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:47.510 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.507+0000 7f60cbf90640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60c40c1ce0 con 0x7f60c415cd10 2024-04-25T00:00:47.510 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.507+0000 7f60cbf90640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f60c406d680 con 0x7f60c40c1f80 2024-04-25T00:00:47.510 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.507+0000 7f60cbf90640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f60c406ca70 con 0x7f60c406c400 2024-04-25T00:00:47.510 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.507+0000 7f60c952b640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60c406c400 0x7f60c40c2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:47.510 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.507+0000 7f60c952b640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60c406c400 0x7f60c40c2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.157:45906/0 (socket says 172.21.15.157:45906) 2024-04-25T00:00:47.510 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.507+0000 7f60c952b640 1 -- 172.21.15.157:0/2841425366 learned_addr learned my addr 172.21.15.157:0/2841425366 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:00:47.510 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.507+0000 7f60c9d2c640 1 --2- 172.21.15.157:0/2841425366 >> v2:172.21.15.157:3300/0 conn(0x7f60c40c1f80 0x7f60c4159720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:47.510 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.508+0000 7f60c952b640 1 -- 172.21.15.157:0/2841425366 >> v2:172.21.15.157:3300/0 conn(0x7f60c40c1f80 msgr2=0x7f60c4159720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:47.511 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.508+0000 7f60c952b640 1 --2- 172.21.15.157:0/2841425366 >> v2:172.21.15.157:3300/0 conn(0x7f60c40c1f80 0x7f60c4159720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:47.511 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.508+0000 7f60c952b640 1 -- 172.21.15.157:0/2841425366 >> v1:172.21.15.169:6789/0 conn(0x7f60c415cd10 legacy=0x7f60c415d1a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:00:47.511 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.508+0000 7f60c952b640 1 -- 172.21.15.157:0/2841425366 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60c406cd50 con 0x7f60c406c400 2024-04-25T00:00:47.520 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.519+0000 7f60c952b640 1 --2- 172.21.15.157:0/2841425366 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60c406c400 0x7f60c40c2360 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f60bc009e50 tx=0x7f60bc02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e010a7dbcab98feb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:47.523 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.522+0000 7f60c8d2a640 1 -- 172.21.15.157:0/2841425366 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60bc036070 con 0x7f60c406c400 2024-04-25T00:00:47.523 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.522+0000 7f60c8d2a640 1 -- 172.21.15.157:0/2841425366 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60bc031600 con 0x7f60c406c400 2024-04-25T00:00:47.523 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.522+0000 7f60c8d2a640 1 -- 172.21.15.157:0/2841425366 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60bc0319b0 con 0x7f60c406c400 2024-04-25T00:00:47.524 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.523+0000 7f60cbf90640 1 -- 172.21.15.157:0/2841425366 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60c406c400 msgr2=0x7f60c40c2360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:47.524 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.523+0000 7f60cbf90640 1 --2- 172.21.15.157:0/2841425366 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60c406c400 0x7f60c40c2360 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f60bc009e50 tx=0x7f60bc02a660 comp rx=0 tx=0).stop 2024-04-25T00:00:47.527 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.526+0000 7f60cbf90640 1 -- 172.21.15.157:0/2841425366 shutdown_connections 2024-04-25T00:00:47.527 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.526+0000 7f60cbf90640 1 --2- 172.21.15.157:0/2841425366 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60c406c400 0x7f60c40c2360 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:47.527 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.526+0000 7f60cbf90640 1 --2- 172.21.15.157:0/2841425366 >> v2:172.21.15.157:3300/0 conn(0x7f60c40c1f80 0x7f60c4159720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:47.528 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.526+0000 7f60cbf90640 1 -- 172.21.15.157:0/2841425366 >> 172.21.15.157:0/2841425366 conn(0x7f60c40a8720 msgr2=0x7f60c40a8b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:47.528 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.526+0000 7f60cbf90640 1 -- 172.21.15.157:0/2841425366 shutdown_connections 2024-04-25T00:00:47.530 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.530+0000 7f60cbf90640 1 -- 172.21.15.157:0/2841425366 wait complete. 2024-04-25T00:00:47.533 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.531+0000 7f60cbf90640 1 Processor -- start 2024-04-25T00:00:47.539 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.532+0000 7f60cbf90640 1 -- start start 2024-04-25T00:00:47.539 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.533+0000 7f60cbf90640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60c406c400 0x7f60c40ac340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:47.540 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.533+0000 7f60cbf90640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60c40c1f80 0x7f60c41522d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:47.540 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.533+0000 7f60cbf90640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60c415cd10 0x7f60c4152810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:47.540 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.533+0000 7f60cbf90640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f60c4153a10 con 0x7f60c406c400 2024-04-25T00:00:47.540 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.533+0000 7f60cbf90640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f60c406d310 con 0x7f60c40c1f80 2024-04-25T00:00:47.540 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.533+0000 7f60cbf90640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f60c4153b90 con 0x7f60c415cd10 2024-04-25T00:00:47.540 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.533+0000 7f60c952b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60c40c1f80 0x7f60c41522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:47.541 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.533+0000 7f60ca52d640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60c415cd10 0x7f60c4152810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:47.541 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.533+0000 7f60ca52d640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60c415cd10 0x7f60c4152810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.157:45914/0 (socket says 172.21.15.157:45914) 2024-04-25T00:00:47.541 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.533+0000 7f60ca52d640 1 -- 172.21.15.157:0/1842351285 learned_addr learned my addr 172.21.15.157:0/1842351285 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:00:47.542 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.534+0000 7f60ca52d640 1 -- 172.21.15.157:0/1842351285 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60c40c1f80 msgr2=0x7f60c41522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:47.542 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.534+0000 7f60ca52d640 1 --2- 172.21.15.157:0/1842351285 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60c40c1f80 0x7f60c41522d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:47.542 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.534+0000 7f60ca52d640 1 -- 172.21.15.157:0/1842351285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60c406c400 msgr2=0x7f60c40ac340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:47.542 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.534+0000 7f60c9d2c640 1 --2- 172.21.15.157:0/1842351285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60c406c400 0x7f60c40ac340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:47.542 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.534+0000 7f60ca52d640 1 --2- 172.21.15.157:0/1842351285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60c406c400 0x7f60c40ac340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:47.542 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.534+0000 7f60ca52d640 1 -- 172.21.15.157:0/1842351285 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60c4151230 con 0x7f60c415cd10 2024-04-25T00:00:47.543 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.534+0000 7f60c9d2c640 1 --2- 172.21.15.157:0/1842351285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60c406c400 0x7f60c40ac340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:00:47.543 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.535+0000 7f60ca52d640 1 --2- 172.21.15.157:0/1842351285 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60c415cd10 0x7f60c4152810 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f60b400b490 tx=0x7f60b400b950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:47.544 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.535+0000 7f60ba7fc640 1 -- 172.21.15.157:0/1842351285 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b4015020 con 0x7f60c415cd10 2024-04-25T00:00:47.544 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.535+0000 7f60ba7fc640 1 -- 172.21.15.157:0/1842351285 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60b4011850 con 0x7f60c415cd10 2024-04-25T00:00:47.544 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.535+0000 7f60cbf90640 1 -- 172.21.15.157:0/1842351285 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60c4151510 con 0x7f60c415cd10 2024-04-25T00:00:47.544 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.535+0000 7f60cbf90640 1 -- 172.21.15.157:0/1842351285 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f60c4151ad0 con 0x7f60c415cd10 2024-04-25T00:00:47.544 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.537+0000 7f60ba7fc640 1 -- 172.21.15.157:0/1842351285 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b4011c00 con 0x7f60c415cd10 2024-04-25T00:00:47.544 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.538+0000 7f60cbf90640 1 -- 172.21.15.157:0/1842351285 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60ac005290 con 0x7f60c415cd10 2024-04-25T00:00:47.546 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.544+0000 7f60ba7fc640 1 -- 172.21.15.157:0/1842351285 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f60b4018020 con 0x7f60c415cd10 2024-04-25T00:00:47.546 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.545+0000 7f60ba7fc640 1 --2- 172.21.15.157:0/1842351285 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f60a806be60 0x7f60a806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:47.547 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.545+0000 7f60ba7fc640 1 -- 172.21.15.157:0/1842351285 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f60b4057690 con 0x7f60c415cd10 2024-04-25T00:00:47.547 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.547+0000 7f60c9d2c640 1 --2- 172.21.15.157:0/1842351285 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f60a806be60 0x7f60a806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:47.550 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.550+0000 7f60ba7fc640 1 -- 172.21.15.157:0/1842351285 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f60b401f020 con 0x7f60c415cd10 2024-04-25T00:00:47.557 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.557+0000 7f60c9d2c640 1 --2- 172.21.15.157:0/1842351285 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f60a806be60 0x7f60a806e320 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f60c0009ac0 tx=0x7f60c0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:47.937 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.934+0000 7f60cbf90640 1 -- 172.21.15.157:0/1842351285 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f60ac005850 con 0x7f60c415cd10 2024-04-25T00:00:47.938 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.937+0000 7f60ba7fc640 1 -- 172.21.15.157:0/1842351285 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18928 (secure 0 0 0) 0x7f60b401f020 con 0x7f60c415cd10 2024-04-25T00:00:47.941 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-25T00:00:47.941 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":27,"fsid":"223685fb-5b64-46a5-9857-3791c9c07df2","created":"2024-04-24T23:34:37.504202+0000","modified":"2024-04-24T23:35:43.517316+0000","last_up_change":"2024-04-24T23:34:47.584963+0000","last_in_change":"2024-04-24T23:34:39.632954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:34:47.796655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:34:52.177208+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:34:52.667022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:34:56.395517+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"95d14ac8-fccf-4261-be8e-edafa746f629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6809","nonce":4003822160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6811","nonce":4003822160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6815","nonce":4003822160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6813","nonce":4003822160}]},"public_addr":"172.21.15.157:6809/4003822160","cluster_addr":"172.21.15.157:6811/4003822160","heartbeat_back_addr":"172.21.15.157:6815/4003822160","heartbeat_front_addr":"172.21.15.157:6813/4003822160","state":["exists","up"]},{"osd":1,"uuid":"7be79da9-8eec-4990-9178-71d602e39d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6817","nonce":1557575576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6819","nonce":1557575576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6823","nonce":1557575576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6821","nonce":1557575576}]},"public_addr":"172.21.15.157:6817/1557575576","cluster_addr":"172.21.15.157:6819/1557575576","heartbeat_back_addr":"172.21.15.157:6823/1557575576","heartbeat_front_addr":"172.21.15.157:6821/1557575576","state":["exists","up"]},{"osd":2,"uuid":"0127ed00-b7c3-466d-bfb0-01b0fd315f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6825","nonce":2793948092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6827","nonce":2793948092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6831","nonce":2793948092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6829","nonce":2793948092}]},"public_addr":"172.21.15.157:6825/2793948092","cluster_addr":"172.21.15.157:6827/2793948092","heartbeat_back_addr":"172.21.15.157:6831/2793948092","heartbeat_front_addr":"172.21.15.157:6829/2793948092","state":["exists","up"]},{"osd":3,"uuid":"f2331acc-aa37-433e-868e-3561b6039562","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6801","nonce":422758549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6803","nonce":422758549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6807","nonce":422758549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6805","nonce":422758549}]},"public_addr":"172.21.15.157:6801/422758549","cluster_addr":"172.21.15.157:6803/422758549","heartbeat_back_addr":"172.21.15.157:6807/422758549","heartbeat_front_addr":"172.21.15.157:6805/422758549","state":["exists","up"]},{"osd":4,"uuid":"1998854b-7398-43f5-ab7d-79fa0debcf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6801","nonce":321910099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6803","nonce":321910099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6807","nonce":321910099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6805","nonce":321910099}]},"public_addr":"172.21.15.169:6801/321910099","cluster_addr":"172.21.15.169:6803/321910099","heartbeat_back_addr":"172.21.15.169:6807/321910099","heartbeat_front_addr":"172.21.15.169:6805/321910099","state":["exists","up"]},{"osd":5,"uuid":"c37153ff-9b2b-48b9-a684-2c82b6f39f15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6825","nonce":243321711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6827","nonce":243321711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6831","nonce":243321711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6829","nonce":243321711}]},"public_addr":"172.21.15.169:6825/243321711","cluster_addr":"172.21.15.169:6827/243321711","heartbeat_back_addr":"172.21.15.169:6831/243321711","heartbeat_front_addr":"172.21.15.169:6829/243321711","state":["exists","up"]},{"osd":6,"uuid":"8db79e1c-f36b-49cb-8d8a-a20c2aec57c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6809","nonce":3441420785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6811","nonce":3441420785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6815","nonce":3441420785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6813","nonce":3441420785}]},"public_addr":"172.21.15.169:6809/3441420785","cluster_addr":"172.21.15.169:6811/3441420785","heartbeat_back_addr":"172.21.15.169:6815/3441420785","heartbeat_front_addr":"172.21.15.169:6813/3441420785","state":["exists","up"]},{"osd":7,"uuid":"928586e9-296c-4b8b-9b27-42f9f155dbdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6817","nonce":1663179020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6819","nonce":1663179020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6823","nonce":1663179020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6821","nonce":1663179020}]},"public_addr":"172.21.15.169:6817/1663179020","cluster_addr":"172.21.15.169:6819/1663179020","heartbeat_back_addr":"172.21.15.169:6823/1663179020","heartbeat_front_addr":"172.21.15.169:6821/1663179020","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.848039+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.826489+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.801120+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.727706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.824343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.791915+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.878073+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.810831+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.33","mappings":[{"from":3,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:00:47.942 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.941+0000 7f60cbf90640 1 -- 172.21.15.157:0/1842351285 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f60a806be60 msgr2=0x7f60a806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:47.942 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.941+0000 7f60cbf90640 1 --2- 172.21.15.157:0/1842351285 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f60a806be60 0x7f60a806e320 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f60c0009ac0 tx=0x7f60c0008040 comp rx=0 tx=0).stop 2024-04-25T00:00:47.942 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.941+0000 7f60cbf90640 1 -- 172.21.15.157:0/1842351285 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60c415cd10 msgr2=0x7f60c4152810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:47.942 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.941+0000 7f60cbf90640 1 --2- 172.21.15.157:0/1842351285 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60c415cd10 0x7f60c4152810 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f60b400b490 tx=0x7f60b400b950 comp rx=0 tx=0).stop 2024-04-25T00:00:47.942 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.941+0000 7f60cbf90640 1 -- 172.21.15.157:0/1842351285 shutdown_connections 2024-04-25T00:00:47.943 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.941+0000 7f60cbf90640 1 --2- 172.21.15.157:0/1842351285 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f60a806be60 0x7f60a806e320 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:47.943 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.941+0000 7f60cbf90640 1 --2- 172.21.15.157:0/1842351285 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60c415cd10 0x7f60c4152810 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:47.943 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.941+0000 7f60cbf90640 1 --2- 172.21.15.157:0/1842351285 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60c40c1f80 0x7f60c41522d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:47.943 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.941+0000 7f60cbf90640 1 --2- 172.21.15.157:0/1842351285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60c406c400 0x7f60c40ac340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:47.943 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.941+0000 7f60cbf90640 1 -- 172.21.15.157:0/1842351285 >> 172.21.15.157:0/1842351285 conn(0x7f60c40a8720 msgr2=0x7f60c40a7830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:47.943 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.942+0000 7f60cbf90640 1 -- 172.21.15.157:0/1842351285 shutdown_connections 2024-04-25T00:00:47.943 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:47.942+0000 7f60cbf90640 1 -- 172.21.15.157:0/1842351285 wait complete. 2024-04-25T00:00:47.966 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:34:47.796655+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:34:52.177208+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:34:52.667022+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T23:34:56.395517+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T00:00:47.966 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:00:48.085 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.074+0000 7f12731af640 1 Processor -- start 2024-04-25T00:00:48.086 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.074+0000 7f12731af640 1 -- start start 2024-04-25T00:00:48.086 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.075+0000 7f12731af640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f126c0c7cd0 0x7f126c0c8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:48.086 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.075+0000 7f12731af640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f126c153620 0x7f126c153ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:48.086 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.075+0000 7f12731af640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f126c150e20 con 0x7f126c150a40 2024-04-25T00:00:48.086 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.075+0000 7f12731af640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f126c0c8670 con 0x7f126c153620 2024-04-25T00:00:48.087 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.075+0000 7f12731af640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f126c1568d0 con 0x7f126c0c7cd0 2024-04-25T00:00:48.087 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.075+0000 7f127174c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f126c153620 0x7f126c153ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:48.087 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.075+0000 7f127174c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f126c153620 0x7f126c153ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51908/0 (socket says 172.21.15.157:51908) 2024-04-25T00:00:48.087 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.075+0000 7f127174c640 1 -- 172.21.15.157:0/3470695132 learned_addr learned my addr 172.21.15.157:0/3470695132 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:00:48.087 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.075+0000 7f126bfff640 1 --2- 172.21.15.157:0/3470695132 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f126c0c7cd0 0x7f126c0c8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:48.087 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.075+0000 7f127174c640 1 -- 172.21.15.157:0/3470695132 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f126c0c7cd0 msgr2=0x7f126c0c8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:48.088 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.075+0000 7f127174c640 1 --2- 172.21.15.157:0/3470695132 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f126c0c7cd0 0x7f126c0c8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:48.088 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.075+0000 7f127174c640 1 -- 172.21.15.157:0/3470695132 >> v1:172.21.15.169:6789/0 conn(0x7f126c150a40 legacy=0x7f126c0a7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:48.088 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.075+0000 7f127174c640 1 -- 172.21.15.157:0/3470695132 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f126c156b70 con 0x7f126c153620 2024-04-25T00:00:48.088 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.076+0000 7f127174c640 1 --2- 172.21.15.157:0/3470695132 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f126c153620 0x7f126c153ab0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1264009e50 tx=0x7f126402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57a1f76c366f6b5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:48.088 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.076+0000 7f126b7fe640 1 -- 172.21.15.157:0/3470695132 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1264036070 con 0x7f126c153620 2024-04-25T00:00:48.088 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.076+0000 7f126b7fe640 1 -- 172.21.15.157:0/3470695132 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1264031600 con 0x7f126c153620 2024-04-25T00:00:48.089 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.076+0000 7f126b7fe640 1 -- 172.21.15.157:0/3470695132 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12640319d0 con 0x7f126c153620 2024-04-25T00:00:48.089 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.076+0000 7f12731af640 1 -- 172.21.15.157:0/3470695132 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f126c153620 msgr2=0x7f126c153ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:48.089 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.076+0000 7f12731af640 1 --2- 172.21.15.157:0/3470695132 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f126c153620 0x7f126c153ab0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1264009e50 tx=0x7f126402a680 comp rx=0 tx=0).stop 2024-04-25T00:00:48.089 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.076+0000 7f12731af640 1 -- 172.21.15.157:0/3470695132 shutdown_connections 2024-04-25T00:00:48.089 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.076+0000 7f12731af640 1 --2- 172.21.15.157:0/3470695132 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f126c153620 0x7f126c153ab0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:48.089 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.076+0000 7f12731af640 1 --2- 172.21.15.157:0/3470695132 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f126c0c7cd0 0x7f126c0c8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:48.089 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.076+0000 7f12731af640 1 -- 172.21.15.157:0/3470695132 >> 172.21.15.157:0/3470695132 conn(0x7f126c0aa7b0 msgr2=0x7f126c0aab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:48.090 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.077+0000 7f12731af640 1 -- 172.21.15.157:0/3470695132 shutdown_connections 2024-04-25T00:00:48.090 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.077+0000 7f12731af640 1 -- 172.21.15.157:0/3470695132 wait complete. 2024-04-25T00:00:48.090 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.078+0000 7f12731af640 1 Processor -- start 2024-04-25T00:00:48.090 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.078+0000 7f12731af640 1 -- start start 2024-04-25T00:00:48.090 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.078+0000 7f12731af640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f126c0c7cd0 0x7f126c0c20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:48.090 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.078+0000 7f12731af640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f126c150a40 0x7f126c151e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:48.090 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.078+0000 7f12731af640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f126c153620 0x7f126c1eb260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:48.091 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.078+0000 7f12731af640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f126c157410 con 0x7f126c153620 2024-04-25T00:00:48.091 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.078+0000 7f12731af640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f126c157130 con 0x7f126c0c7cd0 2024-04-25T00:00:48.091 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.078+0000 7f12731af640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f126c1576f0 con 0x7f126c150a40 2024-04-25T00:00:48.091 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.078+0000 7f127174c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f126c153620 0x7f126c1eb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:48.091 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.078+0000 7f127174c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f126c153620 0x7f126c1eb260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51916/0 (socket says 172.21.15.157:51916) 2024-04-25T00:00:48.091 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.078+0000 7f127174c640 1 -- 172.21.15.157:0/1883598427 learned_addr learned my addr 172.21.15.157:0/1883598427 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:00:48.092 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.078+0000 7f1270f4b640 1 --2- 172.21.15.157:0/1883598427 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f126c0c7cd0 0x7f126c0c20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:48.092 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.078+0000 7f126bfff640 1 --2- 172.21.15.157:0/1883598427 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f126c150a40 0x7f126c151e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:48.092 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.079+0000 7f127174c640 1 -- 172.21.15.157:0/1883598427 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f126c150a40 msgr2=0x7f126c151e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:48.092 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.079+0000 7f127174c640 1 --2- 172.21.15.157:0/1883598427 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f126c150a40 0x7f126c151e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:48.092 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.079+0000 7f127174c640 1 -- 172.21.15.157:0/1883598427 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f126c0c7cd0 msgr2=0x7f126c0c20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:48.092 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.079+0000 7f127174c640 1 --2- 172.21.15.157:0/1883598427 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f126c0c7cd0 0x7f126c0c20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:48.092 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.079+0000 7f127174c640 1 -- 172.21.15.157:0/1883598427 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f126c1526f0 con 0x7f126c153620 2024-04-25T00:00:48.093 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.079+0000 7f127174c640 1 --2- 172.21.15.157:0/1883598427 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f126c153620 0x7f126c1eb260 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1264009e50 tx=0x7f1264004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:48.093 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.079+0000 7f12697fa640 1 -- 172.21.15.157:0/1883598427 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1264041020 con 0x7f126c153620 2024-04-25T00:00:48.093 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.079+0000 7f12697fa640 1 -- 172.21.15.157:0/1883598427 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12640077a0 con 0x7f126c153620 2024-04-25T00:00:48.093 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.079+0000 7f12697fa640 1 -- 172.21.15.157:0/1883598427 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1264036040 con 0x7f126c153620 2024-04-25T00:00:48.093 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.079+0000 7f12731af640 1 -- 172.21.15.157:0/1883598427 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f126c151350 con 0x7f126c153620 2024-04-25T00:00:48.093 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.079+0000 7f12731af640 1 -- 172.21.15.157:0/1883598427 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f126c0c25e0 con 0x7f126c153620 2024-04-25T00:00:48.094 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.080+0000 7f12731af640 1 -- 172.21.15.157:0/1883598427 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f126c0681c0 con 0x7f126c153620 2024-04-25T00:00:48.094 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.081+0000 7f12697fa640 1 -- 172.21.15.157:0/1883598427 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1264044020 con 0x7f126c153620 2024-04-25T00:00:48.094 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.081+0000 7f12697fa640 1 --2- 172.21.15.157:0/1883598427 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f125006be30 0x7f125006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:48.095 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.081+0000 7f12697fa640 1 -- 172.21.15.157:0/1883598427 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f12640abc10 con 0x7f126c153620 2024-04-25T00:00:48.095 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.081+0000 7f1270f4b640 1 --2- 172.21.15.157:0/1883598427 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f125006be30 0x7f125006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:48.095 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.082+0000 7f1270f4b640 1 --2- 172.21.15.157:0/1883598427 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f125006be30 0x7f125006e2f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f126c1566e0 tx=0x7f1260009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:48.095 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.086+0000 7f12697fa640 1 -- 172.21.15.157:0/1883598427 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f126404b030 con 0x7f126c153620 2024-04-25T00:00:48.373 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.371+0000 7f12731af640 1 -- 172.21.15.157:0/1883598427 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f126c0c7a20 con 0x7f126c153620 2024-04-25T00:00:48.374 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.373+0000 7f12697fa640 1 -- 172.21.15.157:0/1883598427 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f1264079c00 con 0x7f126c153620 2024-04-25T00:00:48.376 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2024-04-25T00:00:48.382 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.381+0000 7f12731af640 1 -- 172.21.15.157:0/1883598427 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f125006be30 msgr2=0x7f125006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:48.382 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.381+0000 7f12731af640 1 --2- 172.21.15.157:0/1883598427 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f125006be30 0x7f125006e2f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f126c1566e0 tx=0x7f1260009040 comp rx=0 tx=0).stop 2024-04-25T00:00:48.382 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.381+0000 7f12731af640 1 -- 172.21.15.157:0/1883598427 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f126c153620 msgr2=0x7f126c1eb260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:48.382 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.381+0000 7f12731af640 1 --2- 172.21.15.157:0/1883598427 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f126c153620 0x7f126c1eb260 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1264009e50 tx=0x7f1264004a40 comp rx=0 tx=0).stop 2024-04-25T00:00:48.386 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.381+0000 7f12731af640 1 -- 172.21.15.157:0/1883598427 shutdown_connections 2024-04-25T00:00:48.386 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.381+0000 7f12731af640 1 --2- 172.21.15.157:0/1883598427 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f125006be30 0x7f125006e2f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:48.386 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.381+0000 7f12731af640 1 --2- 172.21.15.157:0/1883598427 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f126c153620 0x7f126c1eb260 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:48.386 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.381+0000 7f12731af640 1 --2- 172.21.15.157:0/1883598427 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f126c150a40 0x7f126c151e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:48.386 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.381+0000 7f12731af640 1 --2- 172.21.15.157:0/1883598427 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f126c0c7cd0 0x7f126c0c20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:48.387 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.381+0000 7f12731af640 1 -- 172.21.15.157:0/1883598427 >> 172.21.15.157:0/1883598427 conn(0x7f126c0aa7b0 msgr2=0x7f126c14d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:48.387 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.382+0000 7f12731af640 1 -- 172.21.15.157:0/1883598427 shutdown_connections 2024-04-25T00:00:48.387 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.382+0000 7f12731af640 1 -- 172.21.15.157:0/1883598427 wait complete. 2024-04-25T00:00:48.405 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:00:48.548 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.544+0000 7f4f538d4640 1 Processor -- start 2024-04-25T00:00:48.548 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.544+0000 7f4f538d4640 1 -- start start 2024-04-25T00:00:48.548 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.545+0000 7f4f538d4640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f4c0b03e0 0x7f4f4c1567e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:48.548 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.545+0000 7f4f538d4640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4f4c0c55d0 0x7f4f4c0c5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:48.548 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.545+0000 7f4f538d4640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f4c0cb330 con 0x7f4f4c1592f0 2024-04-25T00:00:48.548 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.545+0000 7f4f538d4640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f4f4c0b07c0 con 0x7f4f4c0c55d0 2024-04-25T00:00:48.548 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.545+0000 7f4f538d4640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4f4c156310 con 0x7f4f4c0b03e0 2024-04-25T00:00:48.549 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.545+0000 7f4f50e6f640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f4f4c1592f0 0x7f4f4c155c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:55234/0 (socket says 172.21.15.157:55234) 2024-04-25T00:00:48.549 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.546+0000 7f4f50e6f640 1 -- 172.21.15.157:0/3366777465 learned_addr learned my addr 172.21.15.157:0/3366777465 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-25T00:00:48.549 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.546+0000 7f4f51670640 1 --2- 172.21.15.157:0/3366777465 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f4c0b03e0 0x7f4f4c1567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:48.549 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.546+0000 7f4f51e71640 1 --2- 172.21.15.157:0/3366777465 >> v2:172.21.15.169:3300/0 conn(0x7f4f4c0c55d0 0x7f4f4c0c5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:48.549 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.546+0000 7f4f43fff640 1 -- 172.21.15.157:0/3366777465 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 637530978 0 0) 0x7f4f4c0cb330 con 0x7f4f4c1592f0 2024-04-25T00:00:48.549 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.546+0000 7f4f51670640 1 -- 172.21.15.157:0/3366777465 >> v2:172.21.15.169:3300/0 conn(0x7f4f4c0c55d0 msgr2=0x7f4f4c0c5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:48.549 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.546+0000 7f4f51670640 1 --2- 172.21.15.157:0/3366777465 >> v2:172.21.15.169:3300/0 conn(0x7f4f4c0c55d0 0x7f4f4c0c5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:48.549 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.546+0000 7f4f51670640 1 -- 172.21.15.157:0/3366777465 >> v1:172.21.15.157:6789/0 conn(0x7f4f4c1592f0 legacy=0x7f4f4c155c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:48.550 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.546+0000 7f4f51670640 1 -- 172.21.15.157:0/3366777465 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f4c0cb610 con 0x7f4f4c0b03e0 2024-04-25T00:00:48.550 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.547+0000 7f4f51e71640 1 --2- 172.21.15.157:0/3366777465 >> v2:172.21.15.169:3300/0 conn(0x7f4f4c0c55d0 0x7f4f4c0c5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:00:48.550 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.547+0000 7f4f51670640 1 --2- 172.21.15.157:0/3366777465 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f4c0b03e0 0x7f4f4c1567e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4f480097a0 tx=0x7f4f4802a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ff20d0998a44550 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:48.550 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.547+0000 7f4f43fff640 1 -- 172.21.15.157:0/3366777465 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f48037070 con 0x7f4f4c0b03e0 2024-04-25T00:00:48.550 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.548+0000 7f4f43fff640 1 -- 172.21.15.157:0/3366777465 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f48032440 con 0x7f4f4c0b03e0 2024-04-25T00:00:48.551 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.548+0000 7f4f43fff640 1 -- 172.21.15.157:0/3366777465 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4802ae60 con 0x7f4f4c0b03e0 2024-04-25T00:00:48.551 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.549+0000 7f4f538d4640 1 -- 172.21.15.157:0/3366777465 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f4c0b03e0 msgr2=0x7f4f4c1567e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:48.551 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.549+0000 7f4f538d4640 1 --2- 172.21.15.157:0/3366777465 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f4c0b03e0 0x7f4f4c1567e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4f480097a0 tx=0x7f4f4802a110 comp rx=0 tx=0).stop 2024-04-25T00:00:48.551 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.550+0000 7f4f538d4640 1 -- 172.21.15.157:0/3366777465 shutdown_connections 2024-04-25T00:00:48.552 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.550+0000 7f4f538d4640 1 --2- 172.21.15.157:0/3366777465 >> v2:172.21.15.169:3300/0 conn(0x7f4f4c0c55d0 0x7f4f4c0c5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:48.552 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.550+0000 7f4f538d4640 1 --2- 172.21.15.157:0/3366777465 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f4c0b03e0 0x7f4f4c1567e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:48.552 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.550+0000 7f4f538d4640 1 -- 172.21.15.157:0/3366777465 >> 172.21.15.157:0/3366777465 conn(0x7f4f4c067080 msgr2=0x7f4f4c067490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:48.554 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.554+0000 7f4f538d4640 1 -- 172.21.15.157:0/3366777465 shutdown_connections 2024-04-25T00:00:48.555 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.556+0000 7f4f538d4640 1 -- 172.21.15.157:0/3366777465 wait complete. 2024-04-25T00:00:48.557 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.557+0000 7f4f538d4640 1 Processor -- start 2024-04-25T00:00:48.557 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.558+0000 7f4f538d4640 1 -- start start 2024-04-25T00:00:48.561 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.559+0000 7f4f538d4640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f4c0b03e0 0x7f4f4c0ca230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:48.561 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.559+0000 7f4f538d4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f4c0c55d0 0x7f4f4c0af1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:48.561 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.559+0000 7f4f538d4640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f4c1592f0 0x7f4f4c14e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:48.561 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.559+0000 7f4f538d4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4c1556e0 con 0x7f4f4c0c55d0 2024-04-25T00:00:48.561 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.559+0000 7f4f538d4640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4c155400 con 0x7f4f4c0b03e0 2024-04-25T00:00:48.561 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.559+0000 7f4f51670640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f4c0b03e0 0x7f4f4c0ca230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:48.561 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.559+0000 7f4f51e71640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f4c1592f0 0x7f4f4c14e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:48.561 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.559+0000 7f4f51670640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f4c0b03e0 0x7f4f4c0ca230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.157:60160/0 (socket says 172.21.15.157:60160) 2024-04-25T00:00:48.561 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.559+0000 7f4f51670640 1 -- 172.21.15.157:0/237548310 learned_addr learned my addr 172.21.15.157:0/237548310 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:00:48.562 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.559+0000 7f4f50e6f640 1 --2- 172.21.15.157:0/237548310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f4c0c55d0 0x7f4f4c0af1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:48.562 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.559+0000 7f4f538d4640 1 -- 172.21.15.157:0/237548310 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4f4c0cafc0 con 0x7f4f4c1592f0 2024-04-25T00:00:48.562 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.560+0000 7f4f51e71640 1 -- 172.21.15.157:0/237548310 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f4c0b03e0 msgr2=0x7f4f4c0ca230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:48.562 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.560+0000 7f4f51e71640 1 --2- 172.21.15.157:0/237548310 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f4c0b03e0 0x7f4f4c0ca230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:48.562 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.560+0000 7f4f51e71640 1 -- 172.21.15.157:0/237548310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f4c0c55d0 msgr2=0x7f4f4c0af1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:48.562 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.560+0000 7f4f51e71640 1 --2- 172.21.15.157:0/237548310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f4c0c55d0 0x7f4f4c0af1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:48.562 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.560+0000 7f4f51e71640 1 -- 172.21.15.157:0/237548310 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f4c0c1dc0 con 0x7f4f4c1592f0 2024-04-25T00:00:48.562 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.560+0000 7f4f50e6f640 1 --2- 172.21.15.157:0/237548310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f4c0c55d0 0x7f4f4c0af1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:00:48.562 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.560+0000 7f4f51e71640 1 --2- 172.21.15.157:0/237548310 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f4c1592f0 0x7f4f4c14e4e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4f4400efc0 tx=0x7f4f4400c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:48.563 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.561+0000 7f4f41ffb640 1 -- 172.21.15.157:0/237548310 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f44018030 con 0x7f4f4c1592f0 2024-04-25T00:00:48.563 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.561+0000 7f4f538d4640 1 -- 172.21.15.157:0/237548310 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f4c0c20a0 con 0x7f4f4c1592f0 2024-04-25T00:00:48.563 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.561+0000 7f4f41ffb640 1 -- 172.21.15.157:0/237548310 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f4400ccd0 con 0x7f4f4c1592f0 2024-04-25T00:00:48.563 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.561+0000 7f4f538d4640 1 -- 172.21.15.157:0/237548310 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f4c0c2660 con 0x7f4f4c1592f0 2024-04-25T00:00:48.564 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.562+0000 7f4f41ffb640 1 -- 172.21.15.157:0/237548310 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f44003b90 con 0x7f4f4c1592f0 2024-04-25T00:00:48.564 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.563+0000 7f4f538d4640 1 -- 172.21.15.157:0/237548310 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f4c0681c0 con 0x7f4f4c1592f0 2024-04-25T00:00:48.564 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.564+0000 7f4f41ffb640 1 -- 172.21.15.157:0/237548310 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4f4401b070 con 0x7f4f4c1592f0 2024-04-25T00:00:48.564 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.564+0000 7f4f41ffb640 1 --2- 172.21.15.157:0/237548310 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f4f3006be30 0x7f4f3006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:48.564 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.564+0000 7f4f41ffb640 1 -- 172.21.15.157:0/237548310 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f4f4408b5d0 con 0x7f4f4c1592f0 2024-04-25T00:00:48.569 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.569+0000 7f4f51670640 1 --2- 172.21.15.157:0/237548310 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f4f3006be30 0x7f4f3006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:48.571 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.571+0000 7f4f51670640 1 --2- 172.21.15.157:0/237548310 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f4f3006be30 0x7f4f3006e2f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4f48000c00 tx=0x7f4f4802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:48.574 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.574+0000 7f4f41ffb640 1 -- 172.21.15.157:0/237548310 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4f44010040 con 0x7f4f4c1592f0 2024-04-25T00:00:48.930 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.931+0000 7f4f538d4640 1 -- 172.21.15.157:0/237548310 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4f4c06d1d0 con 0x7f4f4c1592f0 2024-04-25T00:00:48.938 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.939+0000 7f4f41ffb640 1 -- 172.21.15.157:0/237548310 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f4f440594a0 con 0x7f4f4c1592f0 2024-04-25T00:00:48.940 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2024-04-25T00:00:48.948 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.946+0000 7f4f538d4640 1 -- 172.21.15.157:0/237548310 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f4f3006be30 msgr2=0x7f4f3006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:48.949 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.946+0000 7f4f538d4640 1 --2- 172.21.15.157:0/237548310 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f4f3006be30 0x7f4f3006e2f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4f48000c00 tx=0x7f4f4802e000 comp rx=0 tx=0).stop 2024-04-25T00:00:48.949 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.946+0000 7f4f538d4640 1 -- 172.21.15.157:0/237548310 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f4c1592f0 msgr2=0x7f4f4c14e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:48.949 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.946+0000 7f4f538d4640 1 --2- 172.21.15.157:0/237548310 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f4c1592f0 0x7f4f4c14e4e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4f4400efc0 tx=0x7f4f4400c490 comp rx=0 tx=0).stop 2024-04-25T00:00:48.949 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.946+0000 7f4f538d4640 1 -- 172.21.15.157:0/237548310 shutdown_connections 2024-04-25T00:00:48.949 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.946+0000 7f4f538d4640 1 --2- 172.21.15.157:0/237548310 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f4f3006be30 0x7f4f3006e2f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:48.949 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.946+0000 7f4f538d4640 1 --2- 172.21.15.157:0/237548310 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f4c1592f0 0x7f4f4c14e4e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:48.949 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.946+0000 7f4f538d4640 1 --2- 172.21.15.157:0/237548310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f4c0c55d0 0x7f4f4c0af1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:48.949 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.946+0000 7f4f538d4640 1 --2- 172.21.15.157:0/237548310 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f4c0b03e0 0x7f4f4c0ca230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:48.949 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.946+0000 7f4f538d4640 1 -- 172.21.15.157:0/237548310 >> 172.21.15.157:0/237548310 conn(0x7f4f4c067080 msgr2=0x7f4f4c067460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:48.950 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.950+0000 7f4f538d4640 1 -- 172.21.15.157:0/237548310 shutdown_connections 2024-04-25T00:00:48.954 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:48.954+0000 7f4f538d4640 1 -- 172.21.15.157:0/237548310 wait complete. 2024-04-25T00:00:48.971 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:00:49.157 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.154+0000 7f1d167d6640 1 Processor -- start 2024-04-25T00:00:49.157 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.154+0000 7f1d167d6640 1 -- start start 2024-04-25T00:00:49.158 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.156+0000 7f1d167d6640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1d0800d7d0 0x7f1d08048740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:49.158 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.156+0000 7f1d167d6640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d0804c210 con 0x7f1d080d81c0 2024-04-25T00:00:49.158 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.156+0000 7f1d167d6640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d0800c730 con 0x7f1d08048360 2024-04-25T00:00:49.158 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.156+0000 7f1d167d6640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f1d0800dc60 con 0x7f1d0800d7d0 2024-04-25T00:00:49.158 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.156+0000 7f1d15fd5640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1d0800d7d0 0x7f1d08048740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:49.158 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.156+0000 7f1d15fd5640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1d0800d7d0 0x7f1d08048740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.157:60170/0 (socket says 172.21.15.157:60170) 2024-04-25T00:00:49.158 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.156+0000 7f1d15fd5640 1 -- 172.21.15.157:0/1729543504 learned_addr learned my addr 172.21.15.157:0/1729543504 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:00:49.158 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.156+0000 7f1d15fd5640 1 -- 172.21.15.157:0/1729543504 >> v1:172.21.15.169:6789/0 conn(0x7f1d08048360 legacy=0x7f1d0804bb00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:00:49.158 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.156+0000 7f1d15fd5640 1 -- 172.21.15.157:0/1729543504 >> v1:172.21.15.157:6789/0 conn(0x7f1d080d81c0 legacy=0x7f1d080e0b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T00:00:49.159 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.156+0000 7f1d15fd5640 1 -- 172.21.15.157:0/1729543504 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d08009b80 con 0x7f1d0800d7d0 2024-04-25T00:00:49.159 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.158+0000 7f1d15fd5640 1 --2- 172.21.15.157:0/1729543504 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d0800d7d0 0x7f1d08048740 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1d00009870 tx=0x7f1d0002a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85b22b4c51aabca6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:49.159 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.159+0000 7f1d07fff640 1 -- 172.21.15.157:0/1729543504 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d00037070 con 0x7f1d0800d7d0 2024-04-25T00:00:49.159 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.159+0000 7f1d07fff640 1 -- 172.21.15.157:0/1729543504 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d00032440 con 0x7f1d0800d7d0 2024-04-25T00:00:49.159 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.159+0000 7f1d07fff640 1 -- 172.21.15.157:0/1729543504 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d00032810 con 0x7f1d0800d7d0 2024-04-25T00:00:49.163 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.160+0000 7f1d167d6640 1 -- 172.21.15.157:0/1729543504 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d0800d7d0 msgr2=0x7f1d08048740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:49.163 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.160+0000 7f1d167d6640 1 --2- 172.21.15.157:0/1729543504 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d0800d7d0 0x7f1d08048740 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1d00009870 tx=0x7f1d0002a270 comp rx=0 tx=0).stop 2024-04-25T00:00:49.167 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.168+0000 7f1d167d6640 1 -- 172.21.15.157:0/1729543504 shutdown_connections 2024-04-25T00:00:49.167 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.168+0000 7f1d167d6640 1 --2- 172.21.15.157:0/1729543504 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d0800d7d0 0x7f1d08048740 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:49.167 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.168+0000 7f1d167d6640 1 -- 172.21.15.157:0/1729543504 >> 172.21.15.157:0/1729543504 conn(0x7f1d0804d9e0 msgr2=0x7f1d0804ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:49.168 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.169+0000 7f1d167d6640 1 -- 172.21.15.157:0/1729543504 shutdown_connections 2024-04-25T00:00:49.172 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.173+0000 7f1d167d6640 1 -- 172.21.15.157:0/1729543504 wait complete. 2024-04-25T00:00:49.174 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.174+0000 7f1d167d6640 1 Processor -- start 2024-04-25T00:00:49.177 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.176+0000 7f1d167d6640 1 -- start start 2024-04-25T00:00:49.177 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.177+0000 7f1d167d6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d0800d7d0 0x7f1d080d4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:49.177 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.177+0000 7f1d167d6640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d08048360 0x7f1d080d10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:49.177 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.177+0000 7f1d167d6640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d080d81c0 0x7f1d080d1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:49.177 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.177+0000 7f1d167d6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1d0800cfd0 con 0x7f1d0800d7d0 2024-04-25T00:00:49.177 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.177+0000 7f1d167d6640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1d0800ccf0 con 0x7f1d080d81c0 2024-04-25T00:00:49.177 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.177+0000 7f1d167d6640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1d0800d2b0 con 0x7f1d08048360 2024-04-25T00:00:49.178 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.178+0000 7f1d157d4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d0800d7d0 0x7f1d080d4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:49.178 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.178+0000 7f1d157d4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d0800d7d0 0x7f1d080d4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51930/0 (socket says 172.21.15.157:51930) 2024-04-25T00:00:49.178 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.178+0000 7f1d157d4640 1 -- 172.21.15.157:0/2697597901 learned_addr learned my addr 172.21.15.157:0/2697597901 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:00:49.178 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.178+0000 7f1d14fd3640 1 --2- 172.21.15.157:0/2697597901 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d08048360 0x7f1d080d10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:49.178 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.178+0000 7f1d15fd5640 1 --2- 172.21.15.157:0/2697597901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d080d81c0 0x7f1d080d1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:49.180 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.179+0000 7f1d157d4640 1 -- 172.21.15.157:0/2697597901 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d08048360 msgr2=0x7f1d080d10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:49.180 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.179+0000 7f1d157d4640 1 --2- 172.21.15.157:0/2697597901 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d08048360 0x7f1d080d10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:49.180 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.179+0000 7f1d157d4640 1 -- 172.21.15.157:0/2697597901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d080d81c0 msgr2=0x7f1d080d1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:49.180 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.179+0000 7f1d157d4640 1 --2- 172.21.15.157:0/2697597901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d080d81c0 0x7f1d080d1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:49.180 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.179+0000 7f1d157d4640 1 -- 172.21.15.157:0/2697597901 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d080d1b70 con 0x7f1d0800d7d0 2024-04-25T00:00:49.182 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.181+0000 7f1d157d4640 1 --2- 172.21.15.157:0/2697597901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d0800d7d0 0x7f1d080d4550 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c00b480 tx=0x7f1d0c00b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:49.182 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.181+0000 7f1d05ffb640 1 -- 172.21.15.157:0/2697597901 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0c014070 con 0x7f1d0800d7d0 2024-04-25T00:00:49.182 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.181+0000 7f1d167d6640 1 -- 172.21.15.157:0/2697597901 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d080c5950 con 0x7f1d0800d7d0 2024-04-25T00:00:49.182 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.181+0000 7f1d167d6640 1 -- 172.21.15.157:0/2697597901 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d080c5f10 con 0x7f1d0800d7d0 2024-04-25T00:00:49.183 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.182+0000 7f1d05ffb640 1 -- 172.21.15.157:0/2697597901 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d0c010870 con 0x7f1d0800d7d0 2024-04-25T00:00:49.183 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.182+0000 7f1d05ffb640 1 -- 172.21.15.157:0/2697597901 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0c014070 con 0x7f1d0800d7d0 2024-04-25T00:00:49.184 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.184+0000 7f1d167d6640 1 -- 172.21.15.157:0/2697597901 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cf0005290 con 0x7f1d0800d7d0 2024-04-25T00:00:49.185 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.185+0000 7f1d05ffb640 1 -- 172.21.15.157:0/2697597901 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1d0c017070 con 0x7f1d0800d7d0 2024-04-25T00:00:49.185 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.185+0000 7f1d05ffb640 1 --2- 172.21.15.157:0/2697597901 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f1cf406c230 0x7f1cf406e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:49.185 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.185+0000 7f1d14fd3640 1 --2- 172.21.15.157:0/2697597901 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f1cf406c230 0x7f1cf406e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:49.185 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.186+0000 7f1d05ffb640 1 -- 172.21.15.157:0/2697597901 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f1d0c010670 con 0x7f1d0800d7d0 2024-04-25T00:00:49.185 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.186+0000 7f1d14fd3640 1 --2- 172.21.15.157:0/2697597901 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f1cf406c230 0x7f1cf406e6f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1d10050ce0 tx=0x7f1d10069040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:49.194 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.195+0000 7f1d05ffb640 1 -- 172.21.15.157:0/2697597901 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1d0c058940 con 0x7f1d0800d7d0 2024-04-25T00:00:49.553 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.554+0000 7f1d167d6640 1 -- 172.21.15.157:0/2697597901 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1cf0005850 con 0x7f1d0800d7d0 2024-04-25T00:00:49.554 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.555+0000 7f1d05ffb640 1 -- 172.21.15.157:0/2697597901 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f1d0c05cc40 con 0x7f1d0800d7d0 2024-04-25T00:00:49.554 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 64 2024-04-25T00:00:49.563 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.559+0000 7f1d167d6640 1 -- 172.21.15.157:0/2697597901 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f1cf406c230 msgr2=0x7f1cf406e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:49.565 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.559+0000 7f1d167d6640 1 --2- 172.21.15.157:0/2697597901 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f1cf406c230 0x7f1cf406e6f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1d10050ce0 tx=0x7f1d10069040 comp rx=0 tx=0).stop 2024-04-25T00:00:49.565 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.559+0000 7f1d167d6640 1 -- 172.21.15.157:0/2697597901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d0800d7d0 msgr2=0x7f1d080d4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:49.565 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.559+0000 7f1d167d6640 1 --2- 172.21.15.157:0/2697597901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d0800d7d0 0x7f1d080d4550 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c00b480 tx=0x7f1d0c00b940 comp rx=0 tx=0).stop 2024-04-25T00:00:49.565 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.560+0000 7f1d167d6640 1 -- 172.21.15.157:0/2697597901 shutdown_connections 2024-04-25T00:00:49.566 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.560+0000 7f1d167d6640 1 --2- 172.21.15.157:0/2697597901 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f1cf406c230 0x7f1cf406e6f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:49.566 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.560+0000 7f1d167d6640 1 --2- 172.21.15.157:0/2697597901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d080d81c0 0x7f1d080d1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:49.566 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.560+0000 7f1d167d6640 1 --2- 172.21.15.157:0/2697597901 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d08048360 0x7f1d080d10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:49.566 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.560+0000 7f1d167d6640 1 --2- 172.21.15.157:0/2697597901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d0800d7d0 0x7f1d080d4550 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:49.566 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.560+0000 7f1d167d6640 1 -- 172.21.15.157:0/2697597901 >> 172.21.15.157:0/2697597901 conn(0x7f1d0804d9e0 msgr2=0x7f1d0804ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:49.566 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.560+0000 7f1d167d6640 1 -- 172.21.15.157:0/2697597901 shutdown_connections 2024-04-25T00:00:49.567 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.560+0000 7f1d167d6640 1 -- 172.21.15.157:0/2697597901 wait complete. 2024-04-25T00:00:49.582 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T00:00:49.763 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.763+0000 7fa11b895640 1 Processor -- start 2024-04-25T00:00:49.764 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.763+0000 7fa11b895640 1 -- start start 2024-04-25T00:00:49.764 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.765+0000 7fa11b895640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa114156f70 0x7fa1140c55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:49.764 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.765+0000 7fa11b895640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa11415cdf0 0x7fa114159d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:49.764 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.765+0000 7fa11b895640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa114159880 con 0x7fa1140c5b10 2024-04-25T00:00:49.764 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.765+0000 7fa11b895640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fa1140b0730 con 0x7fa114156f70 2024-04-25T00:00:49.764 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.765+0000 7fa11b895640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa114157350 con 0x7fa11415cdf0 2024-04-25T00:00:49.767 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.765+0000 7fa119631640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa114156f70 0x7fa1140c55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:49.767 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.765+0000 7fa119631640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa114156f70 0x7fa1140c55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51932/0 (socket says 172.21.15.157:51932) 2024-04-25T00:00:49.769 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.765+0000 7fa119631640 1 -- 172.21.15.157:0/2662764204 learned_addr learned my addr 172.21.15.157:0/2662764204 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:00:49.769 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.765+0000 7fa118e30640 1 --2- 172.21.15.157:0/2662764204 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa11415cdf0 0x7fa114159d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:49.769 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.766+0000 7fa119631640 1 -- 172.21.15.157:0/2662764204 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa11415cdf0 msgr2=0x7fa114159d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:49.769 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.766+0000 7fa119631640 1 --2- 172.21.15.157:0/2662764204 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa11415cdf0 0x7fa114159d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:49.769 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.766+0000 7fa119631640 1 -- 172.21.15.157:0/2662764204 >> v1:172.21.15.157:6789/0 conn(0x7fa1140c5b10 legacy=0x7fa1140c9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:49.769 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.766+0000 7fa119631640 1 -- 172.21.15.157:0/2662764204 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1140c8b50 con 0x7fa114156f70 2024-04-25T00:00:49.769 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.767+0000 7fa119631640 1 --2- 172.21.15.157:0/2662764204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa114156f70 0x7fa1140c55d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa110009870 tx=0x7fa11002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88715fd4f4d20a12 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:49.775 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.774+0000 7fa10bfff640 1 -- 172.21.15.157:0/2662764204 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa110037070 con 0x7fa114156f70 2024-04-25T00:00:49.775 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.774+0000 7fa10bfff640 1 -- 172.21.15.157:0/2662764204 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa110032440 con 0x7fa114156f70 2024-04-25T00:00:49.775 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.774+0000 7fa10bfff640 1 -- 172.21.15.157:0/2662764204 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1100327f0 con 0x7fa114156f70 2024-04-25T00:00:49.775 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.775+0000 7fa11b895640 1 -- 172.21.15.157:0/2662764204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa114156f70 msgr2=0x7fa1140c55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:49.776 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.775+0000 7fa11b895640 1 --2- 172.21.15.157:0/2662764204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa114156f70 0x7fa1140c55d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa110009870 tx=0x7fa11002a1a0 comp rx=0 tx=0).stop 2024-04-25T00:00:49.777 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.777+0000 7fa11b895640 1 -- 172.21.15.157:0/2662764204 shutdown_connections 2024-04-25T00:00:49.777 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.777+0000 7fa11b895640 1 --2- 172.21.15.157:0/2662764204 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa11415cdf0 0x7fa114159d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:49.777 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.777+0000 7fa11b895640 1 --2- 172.21.15.157:0/2662764204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa114156f70 0x7fa1140c55d0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:49.777 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.777+0000 7fa11b895640 1 -- 172.21.15.157:0/2662764204 >> 172.21.15.157:0/2662764204 conn(0x7fa114067080 msgr2=0x7fa114067490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:49.778 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.778+0000 7fa11b895640 1 -- 172.21.15.157:0/2662764204 shutdown_connections 2024-04-25T00:00:49.778 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.778+0000 7fa11b895640 1 -- 172.21.15.157:0/2662764204 wait complete. 2024-04-25T00:00:49.779 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.780+0000 7fa11b895640 1 Processor -- start 2024-04-25T00:00:49.780 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.780+0000 7fa11b895640 1 -- start start 2024-04-25T00:00:49.784 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.781+0000 7fa11b895640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa1140c5b10 0x7fa1141e4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:49.785 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.781+0000 7fa11b895640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa114156f70 0x7fa1140ab000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:49.785 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.781+0000 7fa11b895640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa11415cdf0 0x7fa1140ae900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:49.785 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.781+0000 7fa11b895640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa11415d870 con 0x7fa11415cdf0 2024-04-25T00:00:49.785 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.781+0000 7fa11b895640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa11415d590 con 0x7fa114156f70 2024-04-25T00:00:49.785 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.781+0000 7fa11b895640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa11415db50 con 0x7fa1140c5b10 2024-04-25T00:00:49.785 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.781+0000 7fa119e32640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa11415cdf0 0x7fa1140ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:49.786 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.781+0000 7fa118e30640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa114156f70 0x7fa1140ab000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:49.786 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.781+0000 7fa119e32640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa11415cdf0 0x7fa1140ae900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51944/0 (socket says 172.21.15.157:51944) 2024-04-25T00:00:49.786 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.781+0000 7fa119e32640 1 -- 172.21.15.157:0/4254514387 learned_addr learned my addr 172.21.15.157:0/4254514387 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:00:49.786 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.782+0000 7fa119631640 1 --2- 172.21.15.157:0/4254514387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa1140c5b10 0x7fa1141e4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:49.786 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.782+0000 7fa119e32640 1 -- 172.21.15.157:0/4254514387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa1140c5b10 msgr2=0x7fa1141e4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:49.786 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.782+0000 7fa119e32640 1 --2- 172.21.15.157:0/4254514387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa1140c5b10 0x7fa1141e4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:49.786 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.782+0000 7fa119e32640 1 -- 172.21.15.157:0/4254514387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa114156f70 msgr2=0x7fa1140ab000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:49.787 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.782+0000 7fa119e32640 1 --2- 172.21.15.157:0/4254514387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa114156f70 0x7fa1140ab000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:49.787 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.782+0000 7fa119e32640 1 -- 172.21.15.157:0/4254514387 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1140ad370 con 0x7fa11415cdf0 2024-04-25T00:00:49.787 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.782+0000 7fa119e32640 1 --2- 172.21.15.157:0/4254514387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa11415cdf0 0x7fa1140ae900 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa10c00c780 tx=0x7fa10c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:49.787 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.783+0000 7fa109ffb640 1 -- 172.21.15.157:0/4254514387 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10c017070 con 0x7fa11415cdf0 2024-04-25T00:00:49.788 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.783+0000 7fa11b895640 1 -- 172.21.15.157:0/4254514387 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1141e52c0 con 0x7fa11415cdf0 2024-04-25T00:00:49.788 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.783+0000 7fa11b895640 1 -- 172.21.15.157:0/4254514387 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1140af690 con 0x7fa11415cdf0 2024-04-25T00:00:49.788 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.783+0000 7fa109ffb640 1 -- 172.21.15.157:0/4254514387 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa10c003da0 con 0x7fa11415cdf0 2024-04-25T00:00:49.788 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.783+0000 7fa109ffb640 1 -- 172.21.15.157:0/4254514387 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10c0052b0 con 0x7fa11415cdf0 2024-04-25T00:00:49.789 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.786+0000 7fa11b895640 1 -- 172.21.15.157:0/4254514387 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0f4005290 con 0x7fa11415cdf0 2024-04-25T00:00:49.789 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.787+0000 7fa109ffb640 1 -- 172.21.15.157:0/4254514387 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa10c01a070 con 0x7fa11415cdf0 2024-04-25T00:00:49.789 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.787+0000 7fa109ffb640 1 --2- 172.21.15.157:0/4254514387 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fa0f806bf30 0x7fa0f806e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:49.789 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.787+0000 7fa109ffb640 1 -- 172.21.15.157:0/4254514387 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fa10c08a7c0 con 0x7fa11415cdf0 2024-04-25T00:00:49.790 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.789+0000 7fa119631640 1 --2- 172.21.15.157:0/4254514387 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fa0f806bf30 0x7fa0f806e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:49.794 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.795+0000 7fa109ffb640 1 -- 172.21.15.157:0/4254514387 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa10c058700 con 0x7fa11415cdf0 2024-04-25T00:00:49.795 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:49.795+0000 7fa119631640 1 --2- 172.21.15.157:0/4254514387 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fa0f806bf30 0x7fa0f806e3f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa110034000 tx=0x7fa11002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:50.146 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.147+0000 7fa11b895640 1 -- 172.21.15.157:0/4254514387 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fa0f4005d90 con 0x7fa11415cdf0 2024-04-25T00:00:50.147 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.148+0000 7fa109ffb640 1 -- 172.21.15.157:0/4254514387 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7fa10c05ca00 con 0x7fa11415cdf0 2024-04-25T00:00:50.149 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2024-04-25T00:00:50.152 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.153+0000 7fa11b895640 1 -- 172.21.15.157:0/4254514387 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fa0f806bf30 msgr2=0x7fa0f806e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:50.152 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.153+0000 7fa11b895640 1 --2- 172.21.15.157:0/4254514387 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fa0f806bf30 0x7fa0f806e3f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa110034000 tx=0x7fa11002e000 comp rx=0 tx=0).stop 2024-04-25T00:00:50.152 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.153+0000 7fa11b895640 1 -- 172.21.15.157:0/4254514387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa11415cdf0 msgr2=0x7fa1140ae900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:50.152 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.153+0000 7fa11b895640 1 --2- 172.21.15.157:0/4254514387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa11415cdf0 0x7fa1140ae900 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa10c00c780 tx=0x7fa10c00cc40 comp rx=0 tx=0).stop 2024-04-25T00:00:50.153 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.153+0000 7fa11b895640 1 -- 172.21.15.157:0/4254514387 shutdown_connections 2024-04-25T00:00:50.153 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.153+0000 7fa11b895640 1 --2- 172.21.15.157:0/4254514387 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fa0f806bf30 0x7fa0f806e3f0 secure :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa110034000 tx=0x7fa11002e000 comp rx=0 tx=0).stop 2024-04-25T00:00:50.153 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.153+0000 7fa11b895640 1 --2- 172.21.15.157:0/4254514387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa11415cdf0 0x7fa1140ae900 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:50.153 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.153+0000 7fa11b895640 1 --2- 172.21.15.157:0/4254514387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa114156f70 0x7fa1140ab000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:50.153 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.153+0000 7fa11b895640 1 --2- 172.21.15.157:0/4254514387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa1140c5b10 0x7fa1141e4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:50.153 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.153+0000 7fa11b895640 1 -- 172.21.15.157:0/4254514387 >> 172.21.15.157:0/4254514387 conn(0x7fa114067080 msgr2=0x7fa1140caf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:50.159 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.159+0000 7fa11b895640 1 -- 172.21.15.157:0/4254514387 shutdown_connections 2024-04-25T00:00:50.159 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.159+0000 7fa11b895640 1 -- 172.21.15.157:0/4254514387 wait complete. 2024-04-25T00:00:50.180 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:00:50.368 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.368+0000 7f10e4992640 1 Processor -- start 2024-04-25T00:00:50.369 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.368+0000 7f10e4992640 1 -- start start 2024-04-25T00:00:50.369 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.369+0000 7f10e4992640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10e0156ee0 0x7f10e006ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:50.369 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.369+0000 7f10e4992640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f10e0159ea0 0x7f10e0164030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:50.369 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.369+0000 7f10e4992640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10e00c3a50 con 0x7f10e006f170 2024-04-25T00:00:50.369 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.369+0000 7f10e4992640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f10e015cd00 con 0x7f10e0159ea0 2024-04-25T00:00:50.369 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.369+0000 7f10e4992640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f10e00c3cc0 con 0x7f10e0156ee0 2024-04-25T00:00:50.369 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.370+0000 7f10df7fe640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10e0156ee0 0x7f10e006ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:50.370 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.370+0000 7f10df7fe640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10e0156ee0 0x7f10e006ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.157:45962/0 (socket says 172.21.15.157:45962) 2024-04-25T00:00:50.370 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.370+0000 7f10df7fe640 1 -- 172.21.15.157:0/2144468797 learned_addr learned my addr 172.21.15.157:0/2144468797 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:00:50.371 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.371+0000 7f10dffff640 1 --2- 172.21.15.157:0/2144468797 >> v2:172.21.15.169:3300/0 conn(0x7f10e0159ea0 0x7f10e0164030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:50.372 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.372+0000 7f10df7fe640 1 -- 172.21.15.157:0/2144468797 >> v2:172.21.15.169:3300/0 conn(0x7f10e0159ea0 msgr2=0x7f10e0164030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:50.372 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.372+0000 7f10df7fe640 1 --2- 172.21.15.157:0/2144468797 >> v2:172.21.15.169:3300/0 conn(0x7f10e0159ea0 0x7f10e0164030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:50.372 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.372+0000 7f10df7fe640 1 -- 172.21.15.157:0/2144468797 >> v1:172.21.15.169:6789/0 conn(0x7f10e006f170 legacy=0x7f10e0159790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:50.372 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.372+0000 7f10df7fe640 1 -- 172.21.15.157:0/2144468797 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e015cfc0 con 0x7f10e0156ee0 2024-04-25T00:00:50.376 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.373+0000 7f10df7fe640 1 --2- 172.21.15.157:0/2144468797 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10e0156ee0 0x7f10e006ec30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f10d0009870 tx=0x7f10d002a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a4be78a316d8ca2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:50.376 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.373+0000 7f10de7fc640 1 -- 172.21.15.157:0/2144468797 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d0037070 con 0x7f10e0156ee0 2024-04-25T00:00:50.376 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.373+0000 7f10de7fc640 1 -- 172.21.15.157:0/2144468797 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10d0032440 con 0x7f10e0156ee0 2024-04-25T00:00:50.376 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.374+0000 7f10e4992640 1 -- 172.21.15.157:0/2144468797 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10e0156ee0 msgr2=0x7f10e006ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:50.376 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.374+0000 7f10de7fc640 1 -- 172.21.15.157:0/2144468797 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d0037070 con 0x7f10e0156ee0 2024-04-25T00:00:50.376 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.374+0000 7f10e4992640 1 --2- 172.21.15.157:0/2144468797 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10e0156ee0 0x7f10e006ec30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f10d0009870 tx=0x7f10d002a130 comp rx=0 tx=0).stop 2024-04-25T00:00:50.383 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.378+0000 7f10e4992640 1 -- 172.21.15.157:0/2144468797 shutdown_connections 2024-04-25T00:00:50.383 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.378+0000 7f10e4992640 1 --2- 172.21.15.157:0/2144468797 >> v2:172.21.15.169:3300/0 conn(0x7f10e0159ea0 0x7f10e0164030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:50.383 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.378+0000 7f10e4992640 1 --2- 172.21.15.157:0/2144468797 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10e0156ee0 0x7f10e006ec30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:50.383 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.378+0000 7f10e4992640 1 -- 172.21.15.157:0/2144468797 >> 172.21.15.157:0/2144468797 conn(0x7f10e006d8e0 msgr2=0x7f10e006c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:50.389 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.388+0000 7f10e4992640 1 -- 172.21.15.157:0/2144468797 shutdown_connections 2024-04-25T00:00:50.400 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.400+0000 7f10e4992640 1 -- 172.21.15.157:0/2144468797 wait complete. 2024-04-25T00:00:50.403 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.402+0000 7f10e4992640 1 Processor -- start 2024-04-25T00:00:50.407 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.405+0000 7f10e4992640 1 -- start start 2024-04-25T00:00:50.408 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.406+0000 7f10e4992640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10e006f170 0x7f10e0158dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:50.408 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.406+0000 7f10e4992640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10e0156ee0 0x7f10e01e2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:50.408 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.406+0000 7f10e4992640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10e0159ea0 0x7f10e01e3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:50.408 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.406+0000 7f10e4992640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f10e006bfa0 con 0x7f10e0159ea0 2024-04-25T00:00:50.408 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.406+0000 7f10e4992640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f10e015d580 con 0x7f10e006f170 2024-04-25T00:00:50.408 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.406+0000 7f10e4992640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f10e01572c0 con 0x7f10e0156ee0 2024-04-25T00:00:50.408 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.406+0000 7f10dffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10e0159ea0 0x7f10e01e3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:50.408 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.406+0000 7f10dffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10e0159ea0 0x7f10e01e3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51960/0 (socket says 172.21.15.157:51960) 2024-04-25T00:00:50.408 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.406+0000 7f10dffff640 1 -- 172.21.15.157:0/2110378920 learned_addr learned my addr 172.21.15.157:0/2110378920 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:00:50.408 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.406+0000 7f10dffff640 1 -- 172.21.15.157:0/2110378920 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10e0156ee0 msgr2=0x7f10e01e2d10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T00:00:50.408 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.406+0000 7f10dffff640 1 --2- 172.21.15.157:0/2110378920 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10e0156ee0 0x7f10e01e2d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:50.408 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.406+0000 7f10dffff640 1 -- 172.21.15.157:0/2110378920 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10e006f170 msgr2=0x7f10e0158dd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T00:00:50.408 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.406+0000 7f10dffff640 1 --2- 172.21.15.157:0/2110378920 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10e006f170 0x7f10e0158dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:50.409 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.406+0000 7f10dffff640 1 -- 172.21.15.157:0/2110378920 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e01e3790 con 0x7f10e0159ea0 2024-04-25T00:00:50.409 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.407+0000 7f10dffff640 1 --2- 172.21.15.157:0/2110378920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10e0159ea0 0x7f10e01e3250 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f10d8009dd0 tx=0x7f10d800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:50.409 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.407+0000 7f10c7fff640 1 -- 172.21.15.157:0/2110378920 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d8018020 con 0x7f10e0159ea0 2024-04-25T00:00:50.409 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.407+0000 7f10e4992640 1 -- 172.21.15.157:0/2110378920 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10e01f32d0 con 0x7f10e0159ea0 2024-04-25T00:00:50.409 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.407+0000 7f10e4992640 1 -- 172.21.15.157:0/2110378920 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10e01f2810 con 0x7f10e0159ea0 2024-04-25T00:00:50.409 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.408+0000 7f10c7fff640 1 -- 172.21.15.157:0/2110378920 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10d8014480 con 0x7f10e0159ea0 2024-04-25T00:00:50.409 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.408+0000 7f10c7fff640 1 -- 172.21.15.157:0/2110378920 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d8014870 con 0x7f10e0159ea0 2024-04-25T00:00:50.410 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.410+0000 7f10c7fff640 1 -- 172.21.15.157:0/2110378920 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f10d8005d10 con 0x7f10e0159ea0 2024-04-25T00:00:50.411 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.410+0000 7f10c7fff640 1 --2- 172.21.15.157:0/2110378920 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f10cc06be30 0x7f10cc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:50.411 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.411+0000 7f10c7fff640 1 -- 172.21.15.157:0/2110378920 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f10d808bd80 con 0x7f10e0159ea0 2024-04-25T00:00:50.420 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.416+0000 7f10e4992640 1 -- 172.21.15.157:0/2110378920 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10c0005290 con 0x7f10e0159ea0 2024-04-25T00:00:50.420 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.416+0000 7f10df7fe640 1 --2- 172.21.15.157:0/2110378920 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f10cc06be30 0x7f10cc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:50.425 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.425+0000 7f10c7fff640 1 -- 172.21.15.157:0/2110378920 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f10d8022070 con 0x7f10e0159ea0 2024-04-25T00:00:50.427 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.428+0000 7f10df7fe640 1 --2- 172.21.15.157:0/2110378920 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f10cc06be30 0x7f10cc06e2f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f10d0034000 tx=0x7f10d002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:50.797 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.797+0000 7f10e4992640 1 -- 172.21.15.157:0/2110378920 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f10c0005580 con 0x7f10e0159ea0 2024-04-25T00:00:50.797 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.798+0000 7f10c7fff640 1 -- 172.21.15.157:0/2110378920 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f10e01e3790 con 0x7f10e0159ea0 2024-04-25T00:00:50.799 INFO:teuthology.orchestra.run.smithi157.stderr:listed 0 entries 2024-04-25T00:00:50.802 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.801+0000 7f10e4992640 1 -- 172.21.15.157:0/2110378920 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f10cc06be30 msgr2=0x7f10cc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:50.802 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.801+0000 7f10e4992640 1 --2- 172.21.15.157:0/2110378920 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f10cc06be30 0x7f10cc06e2f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f10d0034000 tx=0x7f10d002e000 comp rx=0 tx=0).stop 2024-04-25T00:00:50.802 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.801+0000 7f10e4992640 1 -- 172.21.15.157:0/2110378920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10e0159ea0 msgr2=0x7f10e01e3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:50.802 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.801+0000 7f10e4992640 1 --2- 172.21.15.157:0/2110378920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10e0159ea0 0x7f10e01e3250 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f10d8009dd0 tx=0x7f10d800c490 comp rx=0 tx=0).stop 2024-04-25T00:00:50.802 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.802+0000 7f10e4992640 1 -- 172.21.15.157:0/2110378920 shutdown_connections 2024-04-25T00:00:50.802 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.802+0000 7f10e4992640 1 --2- 172.21.15.157:0/2110378920 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f10cc06be30 0x7f10cc06e2f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:50.802 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.802+0000 7f10e4992640 1 --2- 172.21.15.157:0/2110378920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10e0159ea0 0x7f10e01e3250 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:50.803 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.802+0000 7f10e4992640 1 --2- 172.21.15.157:0/2110378920 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10e0156ee0 0x7f10e01e2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:50.803 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.802+0000 7f10e4992640 1 --2- 172.21.15.157:0/2110378920 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10e006f170 0x7f10e0158dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:50.803 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.802+0000 7f10e4992640 1 -- 172.21.15.157:0/2110378920 >> 172.21.15.157:0/2110378920 conn(0x7f10e006d8e0 msgr2=0x7f10e0074130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:50.803 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.802+0000 7f10e4992640 1 -- 172.21.15.157:0/2110378920 shutdown_connections 2024-04-25T00:00:50.803 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:50.802+0000 7f10e4992640 1 -- 172.21.15.157:0/2110378920 wait complete. 2024-04-25T00:00:50.817 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-25T00:00:50.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:00:50.817 DEBUG:teuthology.orchestra.run.smithi169:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-25T00:00:50.897 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi169.front.sepia.ceph.com 2024-04-25T00:00:50.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:00:50.898 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T00:00:51.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:00:51.056 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2024-04-25T00:00:51.228 INFO:teuthology.orchestra.run.smithi169.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T00:00:51.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:00:51.229 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:00:51.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:00:51.253 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link delete ceph-brx 2024-04-25T00:00:51.346 DEBUG:teuthology.parallel:result is None 2024-04-25T00:00:51.346 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-25T00:00:51.373 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T00:00:51.373 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:00:51.473 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.473+0000 7f872104d640 1 Processor -- start 2024-04-25T00:00:51.473 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.473+0000 7f872104d640 1 -- start start 2024-04-25T00:00:51.474 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.474+0000 7f872104d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f871c152b40 0x7f871c0c5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:51.475 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.474+0000 7f872104d640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f871c0c5b70 0x7f871c155a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:51.475 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.474+0000 7f872104d640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f871c0c8bb0 con 0x7f871c0be360 2024-04-25T00:00:51.475 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.474+0000 7f872104d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f871c0c8e90 con 0x7f871c152b40 2024-04-25T00:00:51.475 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.474+0000 7f872104d640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f871c159480 con 0x7f871c0c5b70 2024-04-25T00:00:51.475 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.474+0000 7f871ad9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f871c152b40 0x7f871c0c5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:51.475 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.474+0000 7f871ad9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f871c152b40 0x7f871c0c5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51974/0 (socket says 172.21.15.157:51974) 2024-04-25T00:00:51.475 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.474+0000 7f871ad9d640 1 -- 172.21.15.157:0/686719042 learned_addr learned my addr 172.21.15.157:0/686719042 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:00:51.475 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.474+0000 7f871b59e640 1 --2- 172.21.15.157:0/686719042 >> v2:172.21.15.169:3300/0 conn(0x7f871c0c5b70 0x7f871c155a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:51.475 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.474+0000 7f871ad9d640 1 -- 172.21.15.157:0/686719042 >> v2:172.21.15.169:3300/0 conn(0x7f871c0c5b70 msgr2=0x7f871c155a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:51.475 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.474+0000 7f871ad9d640 1 --2- 172.21.15.157:0/686719042 >> v2:172.21.15.169:3300/0 conn(0x7f871c0c5b70 0x7f871c155a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:51.475 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.474+0000 7f871ad9d640 1 -- 172.21.15.157:0/686719042 >> v1:172.21.15.169:6789/0 conn(0x7f871c0be360 legacy=0x7f871c155350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:51.475 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.474+0000 7f871ad9d640 1 -- 172.21.15.157:0/686719042 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f871c159760 con 0x7f871c152b40 2024-04-25T00:00:51.476 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.474+0000 7f871b59e640 1 --2- 172.21.15.157:0/686719042 >> v2:172.21.15.169:3300/0 conn(0x7f871c0c5b70 0x7f871c155a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:00:51.476 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.475+0000 7f871ad9d640 1 --2- 172.21.15.157:0/686719042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f871c152b40 0x7f871c0c5630 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f871c0c90c0 tx=0x7f870c02a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95d8e4d2cdcc833e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:51.476 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.475+0000 7f871a59c640 1 -- 172.21.15.157:0/686719042 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870c037070 con 0x7f871c152b40 2024-04-25T00:00:51.476 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.475+0000 7f871a59c640 1 -- 172.21.15.157:0/686719042 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f870c032440 con 0x7f871c152b40 2024-04-25T00:00:51.476 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.475+0000 7f871a59c640 1 -- 172.21.15.157:0/686719042 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870c032810 con 0x7f871c152b40 2024-04-25T00:00:51.477 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.476+0000 7f872104d640 1 -- 172.21.15.157:0/686719042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f871c152b40 msgr2=0x7f871c0c5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:51.477 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.476+0000 7f872104d640 1 --2- 172.21.15.157:0/686719042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f871c152b40 0x7f871c0c5630 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f871c0c90c0 tx=0x7f870c02a150 comp rx=0 tx=0).stop 2024-04-25T00:00:51.477 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.476+0000 7f872104d640 1 -- 172.21.15.157:0/686719042 shutdown_connections 2024-04-25T00:00:51.477 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.476+0000 7f872104d640 1 --2- 172.21.15.157:0/686719042 >> v2:172.21.15.169:3300/0 conn(0x7f871c0c5b70 0x7f871c155a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:51.477 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.476+0000 7f872104d640 1 --2- 172.21.15.157:0/686719042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f871c152b40 0x7f871c0c5630 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:51.477 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.476+0000 7f872104d640 1 -- 172.21.15.157:0/686719042 >> 172.21.15.157:0/686719042 conn(0x7f871c0672c0 msgr2=0x7f871c0676d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:51.477 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.476+0000 7f872104d640 1 -- 172.21.15.157:0/686719042 shutdown_connections 2024-04-25T00:00:51.477 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.476+0000 7f872104d640 1 -- 172.21.15.157:0/686719042 wait complete. 2024-04-25T00:00:51.477 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.478+0000 7f872104d640 1 Processor -- start 2024-04-25T00:00:51.478 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.478+0000 7f872104d640 1 -- start start 2024-04-25T00:00:51.480 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.479+0000 7f872104d640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f871c0be360 0x7f871c1571d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:51.480 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.479+0000 7f872104d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f871c0c5b70 0x7f871c1584c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:51.480 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.479+0000 7f872104d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f871c152b40 0x7f871c1ebac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:51.480 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.479+0000 7f872104d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f871c0caa00 con 0x7f871c152b40 2024-04-25T00:00:51.480 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.479+0000 7f872104d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f871c0cb210 con 0x7f871c0c5b70 2024-04-25T00:00:51.480 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.479+0000 7f872104d640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f871c0cba40 con 0x7f871c0be360 2024-04-25T00:00:51.480 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.479+0000 7f871b59e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f871c152b40 0x7f871c1ebac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:51.480 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.479+0000 7f871b59e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f871c152b40 0x7f871c1ebac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51976/0 (socket says 172.21.15.157:51976) 2024-04-25T00:00:51.481 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.479+0000 7f871b59e640 1 -- 172.21.15.157:0/1023627956 learned_addr learned my addr 172.21.15.157:0/1023627956 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:00:51.481 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.479+0000 7f8713fff640 1 --2- 172.21.15.157:0/1023627956 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f871c0c5b70 0x7f871c1584c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:51.481 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.479+0000 7f871ad9d640 1 --2- 172.21.15.157:0/1023627956 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f871c0be360 0x7f871c1571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:51.481 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.479+0000 7f871b59e640 1 -- 172.21.15.157:0/1023627956 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f871c0be360 msgr2=0x7f871c1571d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:51.481 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.479+0000 7f871b59e640 1 --2- 172.21.15.157:0/1023627956 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f871c0be360 0x7f871c1571d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:51.481 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.479+0000 7f871b59e640 1 -- 172.21.15.157:0/1023627956 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f871c0c5b70 msgr2=0x7f871c1584c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:51.481 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.479+0000 7f871b59e640 1 --2- 172.21.15.157:0/1023627956 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f871c0c5b70 0x7f871c1584c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:51.481 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.479+0000 7f871b59e640 1 -- 172.21.15.157:0/1023627956 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f871c157c70 con 0x7f871c152b40 2024-04-25T00:00:51.481 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.479+0000 7f871b59e640 1 --2- 172.21.15.157:0/1023627956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f871c152b40 0x7f871c1ebac0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8714007e70 tx=0x7f871400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:51.481 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.479+0000 7f87137fe640 1 -- 172.21.15.157:0/1023627956 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871401a070 con 0x7f871c152b40 2024-04-25T00:00:51.481 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.479+0000 7f87137fe640 1 -- 172.21.15.157:0/1023627956 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8714015600 con 0x7f871c152b40 2024-04-25T00:00:51.481 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.479+0000 7f872104d640 1 -- 172.21.15.157:0/1023627956 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f871c157f50 con 0x7f871c152b40 2024-04-25T00:00:51.481 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.479+0000 7f87137fe640 1 -- 172.21.15.157:0/1023627956 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87140159f0 con 0x7f871c152b40 2024-04-25T00:00:51.481 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.480+0000 7f872104d640 1 -- 172.21.15.157:0/1023627956 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f871c157710 con 0x7f871c152b40 2024-04-25T00:00:51.482 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.480+0000 7f872104d640 1 -- 172.21.15.157:0/1023627956 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f871c0c5950 con 0x7f871c152b40 2024-04-25T00:00:51.482 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.481+0000 7f87137fe640 1 -- 172.21.15.157:0/1023627956 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f871401d070 con 0x7f871c152b40 2024-04-25T00:00:51.482 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.481+0000 7f87137fe640 1 --2- 172.21.15.157:0/1023627956 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f86fc06be30 0x7f86fc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:51.483 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.482+0000 7f87137fe640 1 -- 172.21.15.157:0/1023627956 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f871408b460 con 0x7f871c152b40 2024-04-25T00:00:51.483 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.482+0000 7f871ad9d640 1 --2- 172.21.15.157:0/1023627956 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f86fc06be30 0x7f86fc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:51.483 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.482+0000 7f871ad9d640 1 --2- 172.21.15.157:0/1023627956 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f86fc06be30 0x7f86fc06e2f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f870c034000 tx=0x7f870c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:51.485 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.486+0000 7f87137fe640 1 -- 172.21.15.157:0/1023627956 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8714010070 con 0x7f871c152b40 2024-04-25T00:00:51.664 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.664+0000 7f872104d640 1 -- 172.21.15.157:0/1023627956 --> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f871c158280 con 0x7f86fc06be30 2024-04-25T00:00:51.680 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.680+0000 7f87137fe640 1 -- 172.21.15.157:0/1023627956 <== mgr.4105 v2:172.21.15.169:6832/662378289 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+336828 (secure 0 0 0) 0x7f871c158280 con 0x7f86fc06be30 2024-04-25T00:00:51.684 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-25T00:00:51.686 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2024-04-25T00:00:51.687 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.683+0000 7f872104d640 1 -- 172.21.15.157:0/1023627956 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f86fc06be30 msgr2=0x7f86fc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:51.687 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.683+0000 7f872104d640 1 --2- 172.21.15.157:0/1023627956 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f86fc06be30 0x7f86fc06e2f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f870c034000 tx=0x7f870c02e000 comp rx=0 tx=0).stop 2024-04-25T00:00:51.688 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.683+0000 7f872104d640 1 -- 172.21.15.157:0/1023627956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f871c152b40 msgr2=0x7f871c1ebac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:51.688 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.683+0000 7f872104d640 1 --2- 172.21.15.157:0/1023627956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f871c152b40 0x7f871c1ebac0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8714007e70 tx=0x7f871400d4a0 comp rx=0 tx=0).stop 2024-04-25T00:00:51.688 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.683+0000 7f872104d640 1 -- 172.21.15.157:0/1023627956 shutdown_connections 2024-04-25T00:00:51.688 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.683+0000 7f872104d640 1 --2- 172.21.15.157:0/1023627956 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f86fc06be30 0x7f86fc06e2f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:51.688 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.683+0000 7f872104d640 1 --2- 172.21.15.157:0/1023627956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f871c152b40 0x7f871c1ebac0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:51.688 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.683+0000 7f872104d640 1 --2- 172.21.15.157:0/1023627956 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f871c0c5b70 0x7f871c1584c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:51.688 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.683+0000 7f872104d640 1 --2- 172.21.15.157:0/1023627956 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f871c0be360 0x7f871c1571d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:51.688 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.683+0000 7f872104d640 1 -- 172.21.15.157:0/1023627956 >> 172.21.15.157:0/1023627956 conn(0x7f871c0672c0 msgr2=0x7f871c14da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:51.688 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.683+0000 7f872104d640 1 -- 172.21.15.157:0/1023627956 shutdown_connections 2024-04-25T00:00:51.688 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.683+0000 7f872104d640 1 -- 172.21.15.157:0/1023627956 wait complete. 2024-04-25T00:00:51.694 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":802,"stamp":"2024-04-25T00:00:50.172506+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":345915495,"num_objects":916,"num_object_clones":0,"num_object_copies":3178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":916,"num_whiteouts":0,"num_read":15578,"num_read_kb":3146093,"num_write":422156,"num_write_kb":32315938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":63,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":588160,"ondisk_log_size":588160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":7848880,"kb_used_data":304860,"kb_used_omap":12,"kb_used_meta":7543923,"kb_avail":747125840,"statfs":{"total":773094113280,"available":765056860160,"internally_reserved":0,"allocated":312176640,"data_stored":685320380,"data_compressed":65225103,"data_compressed_allocated":96555008,"data_compressed_original":471117824,"omap_allocated":12752,"internal_metadata":7724977712},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,7,15,22,34,54,56,253,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":55,"apply_latency_ms":55,"commit_latency_ns":55000000,"apply_latency_ns":55000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-118121358,"num_objects":-623,"num_object_clones":0,"num_object_copies":-1468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-623,"num_whiteouts":0,"num_read":128,"num_read_kb":25182,"num_write":3659,"num_write_kb":61598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-64,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008122"},"pg_stats":[{"pgid":"2.1f","version":"27'2678","reported_seq":2716,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.212495+0000","last_change":"2024-04-24T23:34:53.362409+0000","last_active":"2024-04-25T00:00:47.212495+0000","last_peered":"2024-04-25T00:00:47.212495+0000","last_clean":"2024-04-25T00:00:47.212495+0000","last_became_active":"2024-04-24T23:34:53.362104+0000","last_became_peered":"2024-04-24T23:34:53.362104+0000","last_unstale":"2024-04-25T00:00:47.212495+0000","last_undegraded":"2024-04-25T00:00:47.212495+0000","last_fullsized":"2024-04-25T00:00:47.212495+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":2678,"log_dups_size":0,"ondisk_log_size":2678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:53.159219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":2687,"num_write_kb":18218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"27'2525","reported_seq":2561,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:44.806511+0000","last_change":"2024-04-24T23:34:54.373560+0000","last_active":"2024-04-25T00:00:44.806511+0000","last_peered":"2024-04-25T00:00:44.806511+0000","last_clean":"2024-04-25T00:00:44.806511+0000","last_became_active":"2024-04-24T23:34:54.373289+0000","last_became_peered":"2024-04-24T23:34:54.373289+0000","last_unstale":"2024-04-25T00:00:44.806511+0000","last_undegraded":"2024-04-25T00:00:44.806511+0000","last_fullsized":"2024-04-25T00:00:44.806511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2525,"log_dups_size":0,"ondisk_log_size":2525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:31.447899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"27'9404","reported_seq":9750,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:45.918901+0000","last_change":"2024-04-24T23:34:58.730165+0000","last_active":"2024-04-25T00:00:45.918901+0000","last_peered":"2024-04-25T00:00:45.918901+0000","last_clean":"2024-04-25T00:00:45.918901+0000","last_became_active":"2024-04-24T23:34:58.729816+0000","last_became_peered":"2024-04-24T23:34:58.729816+0000","last_unstale":"2024-04-25T00:00:45.918901+0000","last_undegraded":"2024-04-25T00:00:45.918901+0000","last_fullsized":"2024-04-25T00:00:45.918901+0000","mapping_epoch":23,"log_start":"27'3100","ondisk_log_start":"27'3100","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":6304,"log_dups_size":0,"ondisk_log_size":6304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:02:02.335135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5938304,"num_objects":24,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":319,"num_read_kb":81026,"num_write":6703,"num_write_kb":1030241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"27'6587","reported_seq":6625,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:45.133750+0000","last_change":"2024-04-24T23:34:53.361650+0000","last_active":"2024-04-25T00:00:45.133750+0000","last_peered":"2024-04-25T00:00:45.133750+0000","last_clean":"2024-04-25T00:00:45.133750+0000","last_became_active":"2024-04-24T23:34:53.361366+0000","last_became_peered":"2024-04-24T23:34:53.361366+0000","last_unstale":"2024-04-25T00:00:45.133750+0000","last_undegraded":"2024-04-25T00:00:45.133750+0000","last_fullsized":"2024-04-25T00:00:45.133750+0000","mapping_epoch":18,"log_start":"27'300","ondisk_log_start":"27'300","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":6287,"log_dups_size":0,"ondisk_log_size":6287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:56:15.784223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2022402,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6588,"num_write_kb":51489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"27'2521","reported_seq":2557,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:49.128653+0000","last_change":"2024-04-24T23:34:54.383602+0000","last_active":"2024-04-25T00:00:49.128653+0000","last_peered":"2024-04-25T00:00:49.128653+0000","last_clean":"2024-04-25T00:00:49.128653+0000","last_became_active":"2024-04-24T23:34:54.383160+0000","last_became_peered":"2024-04-24T23:34:54.383160+0000","last_unstale":"2024-04-25T00:00:49.128653+0000","last_undegraded":"2024-04-25T00:00:49.128653+0000","last_fullsized":"2024-04-25T00:00:49.128653+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2521,"log_dups_size":0,"ondisk_log_size":2521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:15.477061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"27'9141","reported_seq":9338,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:46.124918+0000","last_change":"2024-04-24T23:34:58.404416+0000","last_active":"2024-04-25T00:00:46.124918+0000","last_peered":"2024-04-25T00:00:46.124918+0000","last_clean":"2024-04-25T00:00:46.124918+0000","last_became_active":"2024-04-24T23:34:58.403984+0000","last_became_peered":"2024-04-24T23:34:58.403984+0000","last_unstale":"2024-04-25T00:00:46.124918+0000","last_undegraded":"2024-04-25T00:00:46.124918+0000","last_fullsized":"2024-04-25T00:00:46.124918+0000","mapping_epoch":23,"log_start":"27'1960","ondisk_log_start":"27'1960","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7181,"log_dups_size":0,"ondisk_log_size":7181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:46.237082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3253957,"num_objects":19,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":169,"num_read_kb":42926,"num_write":6398,"num_write_kb":1082204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"27'10053","reported_seq":10208,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:49.138658+0000","last_change":"2024-04-24T23:34:53.366730+0000","last_active":"2024-04-25T00:00:49.138658+0000","last_peered":"2024-04-25T00:00:49.138658+0000","last_clean":"2024-04-25T00:00:49.138658+0000","last_became_active":"2024-04-24T23:34:53.366430+0000","last_became_peered":"2024-04-24T23:34:53.366430+0000","last_unstale":"2024-04-25T00:00:49.138658+0000","last_undegraded":"2024-04-25T00:00:49.138658+0000","last_fullsized":"2024-04-25T00:00:49.138658+0000","mapping_epoch":18,"log_start":"27'3200","ondisk_log_start":"27'3200","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":6853,"log_dups_size":0,"ondisk_log_size":6853,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:41.422591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5022938,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":120,"num_read_kb":893,"num_write":10045,"num_write_kb":77054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"27'2616","reported_seq":2650,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.280419+0000","last_change":"2024-04-24T23:34:54.373592+0000","last_active":"2024-04-25T00:00:47.280419+0000","last_peered":"2024-04-25T00:00:47.280419+0000","last_clean":"2024-04-25T00:00:47.280419+0000","last_became_active":"2024-04-24T23:34:54.372844+0000","last_became_peered":"2024-04-24T23:34:54.372844+0000","last_unstale":"2024-04-25T00:00:47.280419+0000","last_undegraded":"2024-04-25T00:00:47.280419+0000","last_fullsized":"2024-04-25T00:00:47.280419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2616,"log_dups_size":0,"ondisk_log_size":2616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:49.372040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"27'9348","reported_seq":9962,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:46.556129+0000","last_change":"2024-04-24T23:34:58.399224+0000","last_active":"2024-04-25T00:00:46.556129+0000","last_peered":"2024-04-25T00:00:46.556129+0000","last_clean":"2024-04-25T00:00:46.556129+0000","last_became_active":"2024-04-24T23:34:58.397817+0000","last_became_peered":"2024-04-24T23:34:58.397817+0000","last_unstale":"2024-04-25T00:00:46.556129+0000","last_undegraded":"2024-04-25T00:00:46.556129+0000","last_fullsized":"2024-04-25T00:00:46.556129+0000","mapping_epoch":23,"log_start":"27'2112","ondisk_log_start":"27'2112","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7236,"log_dups_size":0,"ondisk_log_size":7236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:53.085366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3289544,"num_objects":25,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":588,"num_read_kb":149352,"num_write":6655,"num_write_kb":891145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"27'5158","reported_seq":5193,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.195334+0000","last_change":"2024-04-24T23:34:53.362994+0000","last_active":"2024-04-25T00:00:47.195334+0000","last_peered":"2024-04-25T00:00:47.195334+0000","last_clean":"2024-04-25T00:00:47.195334+0000","last_became_active":"2024-04-24T23:34:53.362337+0000","last_became_peered":"2024-04-24T23:34:53.362337+0000","last_unstale":"2024-04-25T00:00:47.195334+0000","last_undegraded":"2024-04-25T00:00:47.195334+0000","last_fullsized":"2024-04-25T00:00:47.195334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":5158,"log_dups_size":0,"ondisk_log_size":5158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:12.258211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":5165,"num_write_kb":44324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"27'2532","reported_seq":2566,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.851337+0000","last_change":"2024-04-24T23:34:54.379218+0000","last_active":"2024-04-25T00:00:47.851337+0000","last_peered":"2024-04-25T00:00:47.851337+0000","last_clean":"2024-04-25T00:00:47.851337+0000","last_became_active":"2024-04-24T23:34:54.376019+0000","last_became_peered":"2024-04-24T23:34:54.376019+0000","last_unstale":"2024-04-25T00:00:47.851337+0000","last_undegraded":"2024-04-25T00:00:47.851337+0000","last_fullsized":"2024-04-25T00:00:47.851337+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2532,"log_dups_size":0,"ondisk_log_size":2532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:41:48.207542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"27'8705","reported_seq":9204,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:46.391347+0000","last_change":"2024-04-24T23:34:58.397152+0000","last_active":"2024-04-25T00:00:46.391347+0000","last_peered":"2024-04-25T00:00:46.391347+0000","last_clean":"2024-04-25T00:00:46.391347+0000","last_became_active":"2024-04-24T23:34:58.396658+0000","last_became_peered":"2024-04-24T23:34:58.396658+0000","last_unstale":"2024-04-25T00:00:46.391347+0000","last_undegraded":"2024-04-25T00:00:46.391347+0000","last_fullsized":"2024-04-25T00:00:46.391347+0000","mapping_epoch":23,"log_start":"27'1509","ondisk_log_start":"27'1509","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7196,"log_dups_size":0,"ondisk_log_size":7196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:41.360776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4269424,"num_objects":28,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":472,"num_read_kb":119888,"num_write":5922,"num_write_kb":848200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"27'2537","reported_seq":2572,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:44.457268+0000","last_change":"2024-04-24T23:34:54.377859+0000","last_active":"2024-04-25T00:00:44.457268+0000","last_peered":"2024-04-25T00:00:44.457268+0000","last_clean":"2024-04-25T00:00:44.457268+0000","last_became_active":"2024-04-24T23:34:54.377576+0000","last_became_peered":"2024-04-24T23:34:54.377576+0000","last_unstale":"2024-04-25T00:00:44.457268+0000","last_undegraded":"2024-04-25T00:00:44.457268+0000","last_fullsized":"2024-04-25T00:00:44.457268+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2537,"log_dups_size":0,"ondisk_log_size":2537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:47.446459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"27'6834","reported_seq":6870,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.226938+0000","last_change":"2024-04-24T23:34:53.360903+0000","last_active":"2024-04-25T00:00:47.226938+0000","last_peered":"2024-04-25T00:00:47.226938+0000","last_clean":"2024-04-25T00:00:47.226938+0000","last_became_active":"2024-04-24T23:34:53.360635+0000","last_became_peered":"2024-04-24T23:34:53.360635+0000","last_unstale":"2024-04-25T00:00:47.226938+0000","last_undegraded":"2024-04-25T00:00:47.226938+0000","last_fullsized":"2024-04-25T00:00:47.226938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":6834,"log_dups_size":0,"ondisk_log_size":6834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:42.205997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6840,"num_write_kb":53782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"27'10158","reported_seq":10440,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:46.376285+0000","last_change":"2024-04-24T23:34:58.407146+0000","last_active":"2024-04-25T00:00:46.376285+0000","last_peered":"2024-04-25T00:00:46.376285+0000","last_clean":"2024-04-25T00:00:46.376285+0000","last_became_active":"2024-04-24T23:34:58.406844+0000","last_became_peered":"2024-04-24T23:34:58.406844+0000","last_unstale":"2024-04-25T00:00:46.376285+0000","last_undegraded":"2024-04-25T00:00:46.376285+0000","last_fullsized":"2024-04-25T00:00:46.376285+0000","mapping_epoch":23,"log_start":"27'2941","ondisk_log_start":"27'2941","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7217,"log_dups_size":0,"ondisk_log_size":7217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:41:52.244767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4910152,"num_objects":31,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":253,"num_read_kb":64262,"num_write":7410,"num_write_kb":1029421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"27'2561","reported_seq":2596,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:44.968398+0000","last_change":"2024-04-24T23:34:54.366724+0000","last_active":"2024-04-25T00:00:44.968398+0000","last_peered":"2024-04-25T00:00:44.968398+0000","last_clean":"2024-04-25T00:00:44.968398+0000","last_became_active":"2024-04-24T23:34:54.366426+0000","last_became_peered":"2024-04-24T23:34:54.366426+0000","last_unstale":"2024-04-25T00:00:44.968398+0000","last_undegraded":"2024-04-25T00:00:44.968398+0000","last_fullsized":"2024-04-25T00:00:44.968398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2561,"log_dups_size":0,"ondisk_log_size":2561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:47.035356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"27'7909","reported_seq":7946,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:05.230406+0000","last_change":"2024-04-24T23:34:53.359140+0000","last_active":"2024-04-25T00:00:05.230406+0000","last_peered":"2024-04-25T00:00:05.230406+0000","last_clean":"2024-04-25T00:00:05.230406+0000","last_became_active":"2024-04-24T23:34:53.358841+0000","last_became_peered":"2024-04-24T23:34:53.358841+0000","last_unstale":"2024-04-25T00:00:05.230406+0000","last_undegraded":"2024-04-25T00:00:05.230406+0000","last_fullsized":"2024-04-25T00:00:05.230406+0000","mapping_epoch":18,"log_start":"27'500","ondisk_log_start":"27'500","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":7409,"log_dups_size":0,"ondisk_log_size":7409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:41.422591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7908,"num_write_kb":58475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"27'9659","reported_seq":10064,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:44.950505+0000","last_change":"2024-04-24T23:34:58.400128+0000","last_active":"2024-04-25T00:00:44.950505+0000","last_peered":"2024-04-25T00:00:44.950505+0000","last_clean":"2024-04-25T00:00:44.950505+0000","last_became_active":"2024-04-24T23:34:58.399375+0000","last_became_peered":"2024-04-24T23:34:58.399375+0000","last_unstale":"2024-04-25T00:00:44.950505+0000","last_undegraded":"2024-04-25T00:00:44.950505+0000","last_fullsized":"2024-04-25T00:00:44.950505+0000","mapping_epoch":23,"log_start":"27'2456","ondisk_log_start":"27'2456","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7203,"log_dups_size":0,"ondisk_log_size":7203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:53.409265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6924224,"num_objects":35,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":403,"num_read_kb":102362,"num_write":6939,"num_write_kb":969056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"27'2625","reported_seq":2660,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:44.403956+0000","last_change":"2024-04-24T23:34:54.365012+0000","last_active":"2024-04-25T00:00:44.403956+0000","last_peered":"2024-04-25T00:00:44.403956+0000","last_clean":"2024-04-25T00:00:44.403956+0000","last_became_active":"2024-04-24T23:34:54.364744+0000","last_became_peered":"2024-04-24T23:34:54.364744+0000","last_unstale":"2024-04-25T00:00:44.403956+0000","last_undegraded":"2024-04-25T00:00:44.403956+0000","last_fullsized":"2024-04-25T00:00:44.403956+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2625,"log_dups_size":0,"ondisk_log_size":2625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:35.774122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"27'4020","reported_seq":4057,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-24T23:58:44.283649+0000","last_change":"2024-04-24T23:34:53.360843+0000","last_active":"2024-04-24T23:58:44.283649+0000","last_peered":"2024-04-24T23:58:44.283649+0000","last_clean":"2024-04-24T23:58:44.283649+0000","last_became_active":"2024-04-24T23:34:53.360570+0000","last_became_peered":"2024-04-24T23:34:53.360570+0000","last_unstale":"2024-04-24T23:58:44.283649+0000","last_undegraded":"2024-04-24T23:58:44.283649+0000","last_fullsized":"2024-04-24T23:58:44.283649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":4020,"log_dups_size":0,"ondisk_log_size":4020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:21.089467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4020,"num_write_kb":27320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"27'9575","reported_seq":9994,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:45.249126+0000","last_change":"2024-04-24T23:34:58.731835+0000","last_active":"2024-04-25T00:00:45.249126+0000","last_peered":"2024-04-25T00:00:45.249126+0000","last_clean":"2024-04-25T00:00:45.249126+0000","last_became_active":"2024-04-24T23:34:58.731485+0000","last_became_peered":"2024-04-24T23:34:58.731485+0000","last_unstale":"2024-04-25T00:00:45.249126+0000","last_undegraded":"2024-04-25T00:00:45.249126+0000","last_fullsized":"2024-04-25T00:00:45.249126+0000","mapping_epoch":23,"log_start":"27'2216","ondisk_log_start":"27'2216","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7359,"log_dups_size":0,"ondisk_log_size":7359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:56.609480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5803439,"num_objects":17,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":392,"num_read_kb":99568,"num_write":6958,"num_write_kb":902680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"27'2530","reported_seq":2565,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:44.799276+0000","last_change":"2024-04-24T23:34:54.362447+0000","last_active":"2024-04-25T00:00:44.799276+0000","last_peered":"2024-04-25T00:00:44.799276+0000","last_clean":"2024-04-25T00:00:44.799276+0000","last_became_active":"2024-04-24T23:34:54.362141+0000","last_became_peered":"2024-04-24T23:34:54.362141+0000","last_unstale":"2024-04-25T00:00:44.799276+0000","last_undegraded":"2024-04-25T00:00:44.799276+0000","last_fullsized":"2024-04-25T00:00:44.799276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2530,"log_dups_size":0,"ondisk_log_size":2530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:51.227333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"27'2570","reported_seq":2606,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.182556+0000","last_change":"2024-04-24T23:34:53.366452+0000","last_active":"2024-04-25T00:00:47.182556+0000","last_peered":"2024-04-25T00:00:47.182556+0000","last_clean":"2024-04-25T00:00:47.182556+0000","last_became_active":"2024-04-24T23:34:53.366110+0000","last_became_peered":"2024-04-24T23:34:53.366110+0000","last_unstale":"2024-04-25T00:00:47.182556+0000","last_undegraded":"2024-04-25T00:00:47.182556+0000","last_fullsized":"2024-04-25T00:00:47.182556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":2570,"log_dups_size":0,"ondisk_log_size":2570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:14.716741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2576,"num_write_kb":22186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"27'9154","reported_seq":9327,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:46.615785+0000","last_change":"2024-04-24T23:34:58.402849+0000","last_active":"2024-04-25T00:00:46.615785+0000","last_peered":"2024-04-25T00:00:46.615785+0000","last_clean":"2024-04-25T00:00:46.615785+0000","last_became_active":"2024-04-24T23:34:58.402365+0000","last_became_peered":"2024-04-24T23:34:58.402365+0000","last_unstale":"2024-04-25T00:00:46.615785+0000","last_undegraded":"2024-04-25T00:00:46.615785+0000","last_fullsized":"2024-04-25T00:00:46.615785+0000","mapping_epoch":23,"log_start":"27'1949","ondisk_log_start":"27'1949","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7205,"log_dups_size":0,"ondisk_log_size":7205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:07:32.822427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6020711,"num_objects":43,"num_object_clones":0,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":151,"num_read_kb":38354,"num_write":6506,"num_write_kb":977639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"27'2514","reported_seq":2549,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:49.836629+0000","last_change":"2024-04-24T23:34:54.370300+0000","last_active":"2024-04-25T00:00:49.836629+0000","last_peered":"2024-04-25T00:00:49.836629+0000","last_clean":"2024-04-25T00:00:49.836629+0000","last_became_active":"2024-04-24T23:34:54.369835+0000","last_became_peered":"2024-04-24T23:34:54.369835+0000","last_unstale":"2024-04-25T00:00:49.836629+0000","last_undegraded":"2024-04-25T00:00:49.836629+0000","last_fullsized":"2024-04-25T00:00:49.836629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2514,"log_dups_size":0,"ondisk_log_size":2514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:42.199856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"27'4478","reported_seq":4516,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:01.006427+0000","last_change":"2024-04-24T23:34:53.366110+0000","last_active":"2024-04-25T00:00:01.006427+0000","last_peered":"2024-04-25T00:00:01.006427+0000","last_clean":"2024-04-25T00:00:01.006427+0000","last_became_active":"2024-04-24T23:34:53.365783+0000","last_became_peered":"2024-04-24T23:34:53.365783+0000","last_unstale":"2024-04-25T00:00:01.006427+0000","last_undegraded":"2024-04-25T00:00:01.006427+0000","last_fullsized":"2024-04-25T00:00:01.006427+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":4478,"log_dups_size":0,"ondisk_log_size":4478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:41.422591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":4488,"num_write_kb":39872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"27'9186","reported_seq":9406,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:46.594939+0000","last_change":"2024-04-24T23:34:58.728191+0000","last_active":"2024-04-25T00:00:46.594939+0000","last_peered":"2024-04-25T00:00:46.594939+0000","last_clean":"2024-04-25T00:00:46.594939+0000","last_became_active":"2024-04-24T23:34:58.727653+0000","last_became_peered":"2024-04-24T23:34:58.727653+0000","last_unstale":"2024-04-25T00:00:46.594939+0000","last_undegraded":"2024-04-25T00:00:46.594939+0000","last_fullsized":"2024-04-25T00:00:46.594939+0000","mapping_epoch":23,"log_start":"27'2006","ondisk_log_start":"27'2006","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7180,"log_dups_size":0,"ondisk_log_size":7180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:30.299011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5124378,"num_objects":31,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":190,"num_read_kb":48260,"num_write":6538,"num_write_kb":840404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"27'2471","reported_seq":2506,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:44.965107+0000","last_change":"2024-04-24T23:34:54.368278+0000","last_active":"2024-04-25T00:00:44.965107+0000","last_peered":"2024-04-25T00:00:44.965107+0000","last_clean":"2024-04-25T00:00:44.965107+0000","last_became_active":"2024-04-24T23:34:54.367991+0000","last_became_peered":"2024-04-24T23:34:54.367991+0000","last_unstale":"2024-04-25T00:00:44.965107+0000","last_undegraded":"2024-04-25T00:00:44.965107+0000","last_fullsized":"2024-04-25T00:00:44.965107+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2471,"log_dups_size":0,"ondisk_log_size":2471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:49:31.018719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"27'7464","reported_seq":8126,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:43.649617+0000","last_change":"2024-04-24T23:34:53.364189+0000","last_active":"2024-04-25T00:00:43.649617+0000","last_peered":"2024-04-25T00:00:43.649617+0000","last_clean":"2024-04-25T00:00:43.649617+0000","last_became_active":"2024-04-24T23:34:53.363842+0000","last_became_peered":"2024-04-24T23:34:53.363842+0000","last_unstale":"2024-04-25T00:00:43.649617+0000","last_undegraded":"2024-04-25T00:00:43.649617+0000","last_fullsized":"2024-04-25T00:00:43.649617+0000","mapping_epoch":18,"log_start":"27'100","ondisk_log_start":"27'100","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":7364,"log_dups_size":0,"ondisk_log_size":7364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:12.258211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":626,"num_read_kb":4509,"num_write":10302,"num_write_kb":61529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"27'8867","reported_seq":9175,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:45.755441+0000","last_change":"2024-04-24T23:34:58.729450+0000","last_active":"2024-04-25T00:00:45.755441+0000","last_peered":"2024-04-25T00:00:45.755441+0000","last_clean":"2024-04-25T00:00:45.755441+0000","last_became_active":"2024-04-24T23:34:58.729082+0000","last_became_peered":"2024-04-24T23:34:58.729082+0000","last_unstale":"2024-04-25T00:00:45.755441+0000","last_undegraded":"2024-04-25T00:00:45.755441+0000","last_fullsized":"2024-04-25T00:00:45.755441+0000","mapping_epoch":23,"log_start":"27'2609","ondisk_log_start":"27'2609","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":6258,"log_dups_size":0,"ondisk_log_size":6258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:45.372053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3433546,"num_objects":14,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":278,"num_read_kb":70612,"num_write":6185,"num_write_kb":897704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"27'2575","reported_seq":2611,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:44.888358+0000","last_change":"2024-04-24T23:34:54.357246+0000","last_active":"2024-04-25T00:00:44.888358+0000","last_peered":"2024-04-25T00:00:44.888358+0000","last_clean":"2024-04-25T00:00:44.888358+0000","last_became_active":"2024-04-24T23:34:54.356560+0000","last_became_peered":"2024-04-24T23:34:54.356560+0000","last_unstale":"2024-04-25T00:00:44.888358+0000","last_undegraded":"2024-04-25T00:00:44.888358+0000","last_fullsized":"2024-04-25T00:00:44.888358+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2575,"log_dups_size":0,"ondisk_log_size":2575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:42.199856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"27'6261","reported_seq":6298,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.213097+0000","last_change":"2024-04-24T23:34:53.366296+0000","last_active":"2024-04-25T00:00:47.213097+0000","last_peered":"2024-04-25T00:00:47.213097+0000","last_clean":"2024-04-25T00:00:47.213097+0000","last_became_active":"2024-04-24T23:34:53.365636+0000","last_became_peered":"2024-04-24T23:34:53.365636+0000","last_unstale":"2024-04-25T00:00:47.213097+0000","last_undegraded":"2024-04-25T00:00:47.213097+0000","last_fullsized":"2024-04-25T00:00:47.213097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":6261,"log_dups_size":0,"ondisk_log_size":6261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:50.222015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":6274,"num_write_kb":49208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"27'9152","reported_seq":9686,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:48.941905+0000","last_change":"2024-04-24T23:34:58.728271+0000","last_active":"2024-04-25T00:00:48.941905+0000","last_peered":"2024-04-25T00:00:48.941905+0000","last_clean":"2024-04-25T00:00:48.941905+0000","last_became_active":"2024-04-24T23:34:58.727885+0000","last_became_peered":"2024-04-24T23:34:58.727885+0000","last_unstale":"2024-04-25T00:00:48.941905+0000","last_undegraded":"2024-04-25T00:00:48.941905+0000","last_fullsized":"2024-04-25T00:00:48.941905+0000","mapping_epoch":23,"log_start":"27'1968","ondisk_log_start":"27'1968","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7184,"log_dups_size":0,"ondisk_log_size":7184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:49.461271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":504,"num_read_kb":128016,"num_write":6457,"num_write_kb":842362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"27'2479","reported_seq":2515,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.782154+0000","last_change":"2024-04-24T23:34:54.372245+0000","last_active":"2024-04-25T00:00:47.782154+0000","last_peered":"2024-04-25T00:00:47.782154+0000","last_clean":"2024-04-25T00:00:47.782154+0000","last_became_active":"2024-04-24T23:34:54.371947+0000","last_became_peered":"2024-04-24T23:34:54.371947+0000","last_unstale":"2024-04-25T00:00:47.782154+0000","last_undegraded":"2024-04-25T00:00:47.782154+0000","last_fullsized":"2024-04-25T00:00:47.782154+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2479,"log_dups_size":0,"ondisk_log_size":2479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:47:52.460508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"27'7217","reported_seq":7255,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:22.697628+0000","last_change":"2024-04-24T23:34:53.360352+0000","last_active":"2024-04-25T00:00:22.697628+0000","last_peered":"2024-04-25T00:00:22.697628+0000","last_clean":"2024-04-25T00:00:22.697628+0000","last_became_active":"2024-04-24T23:34:53.360082+0000","last_became_peered":"2024-04-24T23:34:53.360082+0000","last_unstale":"2024-04-25T00:00:22.697628+0000","last_undegraded":"2024-04-25T00:00:22.697628+0000","last_fullsized":"2024-04-25T00:00:22.697628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":7217,"log_dups_size":0,"ondisk_log_size":7217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:50.222015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":7215,"num_write_kb":70161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"27'9096","reported_seq":9280,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:45.286121+0000","last_change":"2024-04-24T23:34:58.404176+0000","last_active":"2024-04-25T00:00:45.286121+0000","last_peered":"2024-04-25T00:00:45.286121+0000","last_clean":"2024-04-25T00:00:45.286121+0000","last_became_active":"2024-04-24T23:34:58.403735+0000","last_became_peered":"2024-04-24T23:34:58.403735+0000","last_unstale":"2024-04-25T00:00:45.286121+0000","last_undegraded":"2024-04-25T00:00:45.286121+0000","last_fullsized":"2024-04-25T00:00:45.286121+0000","mapping_epoch":23,"log_start":"27'1709","ondisk_log_start":"27'1709","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7387,"log_dups_size":0,"ondisk_log_size":7387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:49.461271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3287360,"num_objects":22,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":155,"num_read_kb":39370,"num_write":6428,"num_write_kb":922251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"27'2510","reported_seq":2546,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:44.883767+0000","last_change":"2024-04-24T23:34:54.367638+0000","last_active":"2024-04-25T00:00:44.883767+0000","last_peered":"2024-04-25T00:00:44.883767+0000","last_clean":"2024-04-25T00:00:44.883767+0000","last_became_active":"2024-04-24T23:34:54.367355+0000","last_became_peered":"2024-04-24T23:34:54.367355+0000","last_unstale":"2024-04-25T00:00:44.883767+0000","last_undegraded":"2024-04-25T00:00:44.883767+0000","last_fullsized":"2024-04-25T00:00:44.883767+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2510,"log_dups_size":0,"ondisk_log_size":2510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:15.722059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"27'8389","reported_seq":8427,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.171711+0000","last_change":"2024-04-24T23:34:53.356111+0000","last_active":"2024-04-25T00:00:47.171711+0000","last_peered":"2024-04-25T00:00:47.171711+0000","last_clean":"2024-04-25T00:00:47.171711+0000","last_became_active":"2024-04-24T23:34:53.355710+0000","last_became_peered":"2024-04-24T23:34:53.355710+0000","last_unstale":"2024-04-25T00:00:47.171711+0000","last_undegraded":"2024-04-25T00:00:47.171711+0000","last_fullsized":"2024-04-25T00:00:47.171711+0000","mapping_epoch":18,"log_start":"27'600","ondisk_log_start":"27'600","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":7789,"log_dups_size":0,"ondisk_log_size":7789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:21.089467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":8401,"num_write_kb":63182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"27'9604","reported_seq":9968,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:45.056118+0000","last_change":"2024-04-24T23:34:58.403016+0000","last_active":"2024-04-25T00:00:45.056118+0000","last_peered":"2024-04-25T00:00:45.056118+0000","last_clean":"2024-04-25T00:00:45.056118+0000","last_became_active":"2024-04-24T23:34:58.402598+0000","last_became_peered":"2024-04-24T23:34:58.402598+0000","last_unstale":"2024-04-25T00:00:45.056118+0000","last_undegraded":"2024-04-25T00:00:45.056118+0000","last_fullsized":"2024-04-25T00:00:45.056118+0000","mapping_epoch":23,"log_start":"27'2193","ondisk_log_start":"27'2193","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7411,"log_dups_size":0,"ondisk_log_size":7411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:47:56.497734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2988236,"num_objects":18,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":334,"num_read_kb":84836,"num_write":6810,"num_write_kb":885456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"27'2530","reported_seq":2566,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:44.936154+0000","last_change":"2024-04-24T23:34:54.356601+0000","last_active":"2024-04-25T00:00:44.936154+0000","last_peered":"2024-04-25T00:00:44.936154+0000","last_clean":"2024-04-25T00:00:44.936154+0000","last_became_active":"2024-04-24T23:34:54.356326+0000","last_became_peered":"2024-04-24T23:34:54.356326+0000","last_unstale":"2024-04-25T00:00:44.936154+0000","last_undegraded":"2024-04-25T00:00:44.936154+0000","last_fullsized":"2024-04-25T00:00:44.936154+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2530,"log_dups_size":0,"ondisk_log_size":2530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:47.446459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"27'7177","reported_seq":8144,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.164180+0000","last_change":"2024-04-24T23:34:53.359238+0000","last_active":"2024-04-25T00:00:47.164180+0000","last_peered":"2024-04-25T00:00:47.164180+0000","last_clean":"2024-04-25T00:00:47.164180+0000","last_became_active":"2024-04-24T23:34:53.358937+0000","last_became_peered":"2024-04-24T23:34:53.358937+0000","last_unstale":"2024-04-25T00:00:47.164180+0000","last_undegraded":"2024-04-25T00:00:47.164180+0000","last_fullsized":"2024-04-25T00:00:47.164180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":7177,"log_dups_size":0,"ondisk_log_size":7177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:50.222015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":929,"num_read_kb":4700,"num_write":7175,"num_write_kb":61982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"27'8943","reported_seq":9533,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:45.360737+0000","last_change":"2024-04-24T23:34:58.403258+0000","last_active":"2024-04-25T00:00:45.360737+0000","last_peered":"2024-04-25T00:00:45.360737+0000","last_clean":"2024-04-25T00:00:45.360737+0000","last_became_active":"2024-04-24T23:34:58.402569+0000","last_became_peered":"2024-04-24T23:34:58.402569+0000","last_unstale":"2024-04-25T00:00:45.360737+0000","last_undegraded":"2024-04-25T00:00:45.360737+0000","last_fullsized":"2024-04-25T00:00:45.360737+0000","mapping_epoch":23,"log_start":"27'1538","ondisk_log_start":"27'1538","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7405,"log_dups_size":0,"ondisk_log_size":7405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:32:08.782070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6134362,"num_objects":21,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":564,"num_read_kb":143256,"num_write":6220,"num_write_kb":909081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"27'2480","reported_seq":2516,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:49.530584+0000","last_change":"2024-04-24T23:34:54.377543+0000","last_active":"2024-04-25T00:00:49.530584+0000","last_peered":"2024-04-25T00:00:49.530584+0000","last_clean":"2024-04-25T00:00:49.530584+0000","last_became_active":"2024-04-24T23:34:54.377228+0000","last_became_peered":"2024-04-24T23:34:54.377228+0000","last_unstale":"2024-04-25T00:00:49.530584+0000","last_undegraded":"2024-04-25T00:00:49.530584+0000","last_fullsized":"2024-04-25T00:00:49.530584+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2480,"log_dups_size":0,"ondisk_log_size":2480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:42.255038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"27'7206","reported_seq":7243,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.222770+0000","last_change":"2024-04-24T23:34:53.362675+0000","last_active":"2024-04-25T00:00:47.222770+0000","last_peered":"2024-04-25T00:00:47.222770+0000","last_clean":"2024-04-25T00:00:47.222770+0000","last_became_active":"2024-04-24T23:34:53.362425+0000","last_became_peered":"2024-04-24T23:34:53.362425+0000","last_unstale":"2024-04-25T00:00:47.222770+0000","last_undegraded":"2024-04-25T00:00:47.222770+0000","last_fullsized":"2024-04-25T00:00:47.222770+0000","mapping_epoch":18,"log_start":"27'300","ondisk_log_start":"27'300","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":6906,"log_dups_size":0,"ondisk_log_size":6906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:21.089467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":7208,"num_write_kb":58030,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"27'9329","reported_seq":9851,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:49.782889+0000","last_change":"2024-04-24T23:34:58.728573+0000","last_active":"2024-04-25T00:00:49.782889+0000","last_peered":"2024-04-25T00:00:49.782889+0000","last_clean":"2024-04-25T00:00:49.782889+0000","last_became_active":"2024-04-24T23:34:58.727284+0000","last_became_peered":"2024-04-24T23:34:58.727284+0000","last_unstale":"2024-04-25T00:00:49.782889+0000","last_undegraded":"2024-04-25T00:00:49.782889+0000","last_fullsized":"2024-04-25T00:00:49.782889+0000","mapping_epoch":23,"log_start":"27'2114","ondisk_log_start":"27'2114","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7215,"log_dups_size":0,"ondisk_log_size":7215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:32:08.782070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":493,"num_read_kb":125222,"num_write":6691,"num_write_kb":1110316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"27'2568","reported_seq":2604,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:44.518051+0000","last_change":"2024-04-24T23:34:54.382820+0000","last_active":"2024-04-25T00:00:44.518051+0000","last_peered":"2024-04-25T00:00:44.518051+0000","last_clean":"2024-04-25T00:00:44.518051+0000","last_became_active":"2024-04-24T23:34:54.382564+0000","last_became_peered":"2024-04-24T23:34:54.382564+0000","last_unstale":"2024-04-25T00:00:44.518051+0000","last_undegraded":"2024-04-25T00:00:44.518051+0000","last_fullsized":"2024-04-25T00:00:44.518051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2568,"log_dups_size":0,"ondisk_log_size":2568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:45.622611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"27'5189","reported_seq":5227,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.187361+0000","last_change":"2024-04-24T23:34:53.357743+0000","last_active":"2024-04-25T00:00:47.187361+0000","last_peered":"2024-04-25T00:00:47.187361+0000","last_clean":"2024-04-25T00:00:47.187361+0000","last_became_active":"2024-04-24T23:34:53.357456+0000","last_became_peered":"2024-04-24T23:34:53.357456+0000","last_unstale":"2024-04-25T00:00:47.187361+0000","last_undegraded":"2024-04-25T00:00:47.187361+0000","last_fullsized":"2024-04-25T00:00:47.187361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":5189,"log_dups_size":0,"ondisk_log_size":5189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:57:14.381644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5191,"num_write_kb":40339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":193,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-24T23:56:58.615940+0000","last_change":"2024-04-24T23:34:49.778966+0000","last_active":"2024-04-24T23:56:58.615940+0000","last_peered":"2024-04-24T23:56:58.615940+0000","last_clean":"2024-04-24T23:56:58.615940+0000","last_became_active":"2024-04-24T23:34:49.778083+0000","last_became_peered":"2024-04-24T23:34:49.778083+0000","last_unstale":"2024-04-24T23:56:58.615940+0000","last_undegraded":"2024-04-24T23:56:58.615940+0000","last_fullsized":"2024-04-24T23:56:58.615940+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:48.587078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:48.587078+0000","last_clean_scrub_stamp":"2024-04-24T23:34:48.587078+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:46:59.471089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"27'8813","reported_seq":9330,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:44.690274+0000","last_change":"2024-04-24T23:34:58.729779+0000","last_active":"2024-04-25T00:00:44.690274+0000","last_peered":"2024-04-25T00:00:44.690274+0000","last_clean":"2024-04-25T00:00:44.690274+0000","last_became_active":"2024-04-24T23:34:58.729370+0000","last_became_peered":"2024-04-24T23:34:58.729370+0000","last_unstale":"2024-04-25T00:00:44.690274+0000","last_undegraded":"2024-04-25T00:00:44.690274+0000","last_fullsized":"2024-04-25T00:00:44.690274+0000","mapping_epoch":23,"log_start":"27'902","ondisk_log_start":"27'902","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7911,"log_dups_size":0,"ondisk_log_size":7911,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:30.299011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2352473,"num_objects":16,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":487,"num_read_kb":123698,"num_write":6192,"num_write_kb":909586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"27'2518","reported_seq":2552,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:48.969863+0000","last_change":"2024-04-24T23:34:54.384106+0000","last_active":"2024-04-25T00:00:48.969863+0000","last_peered":"2024-04-25T00:00:48.969863+0000","last_clean":"2024-04-25T00:00:48.969863+0000","last_became_active":"2024-04-24T23:34:54.383589+0000","last_became_peered":"2024-04-24T23:34:54.383589+0000","last_unstale":"2024-04-25T00:00:48.969863+0000","last_undegraded":"2024-04-25T00:00:48.969863+0000","last_fullsized":"2024-04-25T00:00:48.969863+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2518,"log_dups_size":0,"ondisk_log_size":2518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:41:48.207542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"27'8231","reported_seq":8268,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.191600+0000","last_change":"2024-04-24T23:34:53.354077+0000","last_active":"2024-04-25T00:00:47.191600+0000","last_peered":"2024-04-25T00:00:47.191600+0000","last_clean":"2024-04-25T00:00:47.191600+0000","last_became_active":"2024-04-24T23:34:53.353804+0000","last_became_peered":"2024-04-24T23:34:53.353804+0000","last_unstale":"2024-04-25T00:00:47.191600+0000","last_undegraded":"2024-04-25T00:00:47.191600+0000","last_fullsized":"2024-04-25T00:00:47.191600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":8231,"log_dups_size":0,"ondisk_log_size":8231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:41.422591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8225,"num_write_kb":58738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"27'9206","reported_seq":9393,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:46.047173+0000","last_change":"2024-04-24T23:34:58.728722+0000","last_active":"2024-04-25T00:00:46.047173+0000","last_peered":"2024-04-25T00:00:46.047173+0000","last_clean":"2024-04-25T00:00:46.047173+0000","last_became_active":"2024-04-24T23:34:58.727649+0000","last_became_peered":"2024-04-24T23:34:58.727649+0000","last_unstale":"2024-04-25T00:00:46.047173+0000","last_undegraded":"2024-04-25T00:00:46.047173+0000","last_fullsized":"2024-04-25T00:00:46.047173+0000","mapping_epoch":23,"log_start":"27'1207","ondisk_log_start":"27'1207","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7999,"log_dups_size":0,"ondisk_log_size":7999,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:41:52.244767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5413297,"num_objects":32,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":158,"num_read_kb":40132,"num_write":6534,"num_write_kb":960960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"27'2542","reported_seq":2577,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.151045+0000","last_change":"2024-04-24T23:34:54.373371+0000","last_active":"2024-04-25T00:00:47.151045+0000","last_peered":"2024-04-25T00:00:47.151045+0000","last_clean":"2024-04-25T00:00:47.151045+0000","last_became_active":"2024-04-24T23:34:54.372553+0000","last_became_peered":"2024-04-24T23:34:54.372553+0000","last_unstale":"2024-04-25T00:00:47.151045+0000","last_undegraded":"2024-04-25T00:00:47.151045+0000","last_fullsized":"2024-04-25T00:00:47.151045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2542,"log_dups_size":0,"ondisk_log_size":2542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:26.261786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"27'8389","reported_seq":8426,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:40.547701+0000","last_change":"2024-04-24T23:34:53.353505+0000","last_active":"2024-04-25T00:00:40.547701+0000","last_peered":"2024-04-25T00:00:40.547701+0000","last_clean":"2024-04-25T00:00:40.547701+0000","last_became_active":"2024-04-24T23:34:53.353186+0000","last_became_peered":"2024-04-24T23:34:53.353186+0000","last_unstale":"2024-04-25T00:00:40.547701+0000","last_undegraded":"2024-04-25T00:00:40.547701+0000","last_fullsized":"2024-04-25T00:00:40.547701+0000","mapping_epoch":18,"log_start":"27'1000","ondisk_log_start":"27'1000","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":7389,"log_dups_size":0,"ondisk_log_size":7389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:42.205997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388678,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8388,"num_write_kb":58880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"27'9582","reported_seq":9937,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:48.891190+0000","last_change":"2024-04-24T23:34:58.400988+0000","last_active":"2024-04-25T00:00:48.891190+0000","last_peered":"2024-04-25T00:00:48.891190+0000","last_clean":"2024-04-25T00:00:48.891190+0000","last_became_active":"2024-04-24T23:34:58.400549+0000","last_became_peered":"2024-04-24T23:34:58.400549+0000","last_unstale":"2024-04-25T00:00:48.891190+0000","last_undegraded":"2024-04-25T00:00:48.891190+0000","last_fullsized":"2024-04-25T00:00:48.891190+0000","mapping_epoch":23,"log_start":"27'2356","ondisk_log_start":"27'2356","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7226,"log_dups_size":0,"ondisk_log_size":7226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:47:56.497734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":326,"num_read_kb":82804,"num_write":6847,"num_write_kb":886980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"27'2418","reported_seq":2452,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:48.956305+0000","last_change":"2024-04-24T23:34:54.375234+0000","last_active":"2024-04-25T00:00:48.956305+0000","last_peered":"2024-04-25T00:00:48.956305+0000","last_clean":"2024-04-25T00:00:48.956305+0000","last_became_active":"2024-04-24T23:34:54.374963+0000","last_became_peered":"2024-04-24T23:34:54.374963+0000","last_unstale":"2024-04-25T00:00:48.956305+0000","last_undegraded":"2024-04-25T00:00:48.956305+0000","last_fullsized":"2024-04-25T00:00:48.956305+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2418,"log_dups_size":0,"ondisk_log_size":2418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:47:52.460508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"27'8937","reported_seq":8974,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.146208+0000","last_change":"2024-04-24T23:34:53.360531+0000","last_active":"2024-04-25T00:00:47.146208+0000","last_peered":"2024-04-25T00:00:47.146208+0000","last_clean":"2024-04-25T00:00:47.146208+0000","last_became_active":"2024-04-24T23:34:53.360247+0000","last_became_peered":"2024-04-24T23:34:53.360247+0000","last_unstale":"2024-04-25T00:00:47.146208+0000","last_undegraded":"2024-04-25T00:00:47.146208+0000","last_fullsized":"2024-04-25T00:00:47.146208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":8937,"log_dups_size":0,"ondisk_log_size":8937,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:57:14.381644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8935,"num_write_kb":79464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"27'9249","reported_seq":10432,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:48.466460+0000","last_change":"2024-04-24T23:34:58.729798+0000","last_active":"2024-04-25T00:00:48.466460+0000","last_peered":"2024-04-25T00:00:48.466460+0000","last_clean":"2024-04-25T00:00:48.466460+0000","last_became_active":"2024-04-24T23:34:58.729345+0000","last_became_peered":"2024-04-24T23:34:58.729345+0000","last_unstale":"2024-04-25T00:00:48.466460+0000","last_undegraded":"2024-04-25T00:00:48.466460+0000","last_fullsized":"2024-04-25T00:00:48.466460+0000","mapping_epoch":23,"log_start":"27'1527","ondisk_log_start":"27'1527","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7722,"log_dups_size":0,"ondisk_log_size":7722,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:06.670518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":427171,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1154,"num_read_kb":293116,"num_write":6507,"num_write_kb":1022776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"27'2518","reported_seq":2552,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:49.157303+0000","last_change":"2024-04-24T23:34:54.377593+0000","last_active":"2024-04-25T00:00:49.157303+0000","last_peered":"2024-04-25T00:00:49.157303+0000","last_clean":"2024-04-25T00:00:49.157303+0000","last_became_active":"2024-04-24T23:34:54.377270+0000","last_became_peered":"2024-04-24T23:34:54.377270+0000","last_unstale":"2024-04-25T00:00:49.157303+0000","last_undegraded":"2024-04-25T00:00:49.157303+0000","last_fullsized":"2024-04-25T00:00:49.157303+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2518,"log_dups_size":0,"ondisk_log_size":2518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:26.261786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"27'4549","reported_seq":4586,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:08.704934+0000","last_change":"2024-04-24T23:34:53.352981+0000","last_active":"2024-04-25T00:00:08.704934+0000","last_peered":"2024-04-25T00:00:08.704934+0000","last_clean":"2024-04-25T00:00:08.704934+0000","last_became_active":"2024-04-24T23:34:53.352669+0000","last_became_peered":"2024-04-24T23:34:53.352669+0000","last_unstale":"2024-04-25T00:00:08.704934+0000","last_undegraded":"2024-04-25T00:00:08.704934+0000","last_fullsized":"2024-04-25T00:00:08.704934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":4549,"log_dups_size":0,"ondisk_log_size":4549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:09:49.550050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4539,"num_write_kb":43853,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"27'9026","reported_seq":9792,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:46.587640+0000","last_change":"2024-04-24T23:35:43.529974+0000","last_active":"2024-04-25T00:00:46.587640+0000","last_peered":"2024-04-25T00:00:46.587640+0000","last_clean":"2024-04-25T00:00:46.587640+0000","last_became_active":"2024-04-24T23:35:43.529375+0000","last_became_peered":"2024-04-24T23:35:43.529375+0000","last_unstale":"2024-04-25T00:00:46.587640+0000","last_undegraded":"2024-04-25T00:00:46.587640+0000","last_fullsized":"2024-04-25T00:00:46.587640+0000","mapping_epoch":26,"log_start":"27'1812","ondisk_log_start":"27'1812","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7214,"log_dups_size":0,"ondisk_log_size":7214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:48.286783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4258061,"num_objects":31,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":744,"num_read_kb":188976,"num_write":6336,"num_write_kb":872530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3,6],"acting":[4,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"27'2587","reported_seq":2621,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:48.619362+0000","last_change":"2024-04-24T23:34:54.376800+0000","last_active":"2024-04-25T00:00:48.619362+0000","last_peered":"2024-04-25T00:00:48.619362+0000","last_clean":"2024-04-25T00:00:48.619362+0000","last_became_active":"2024-04-24T23:34:54.376512+0000","last_became_peered":"2024-04-24T23:34:54.376512+0000","last_unstale":"2024-04-25T00:00:48.619362+0000","last_undegraded":"2024-04-25T00:00:48.619362+0000","last_fullsized":"2024-04-25T00:00:48.619362+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2587,"log_dups_size":0,"ondisk_log_size":2587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:49.048141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"27'3996","reported_seq":4033,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.147213+0000","last_change":"2024-04-24T23:34:53.355307+0000","last_active":"2024-04-25T00:00:47.147213+0000","last_peered":"2024-04-25T00:00:47.147213+0000","last_clean":"2024-04-25T00:00:47.147213+0000","last_became_active":"2024-04-24T23:34:53.354988+0000","last_became_peered":"2024-04-24T23:34:53.354988+0000","last_unstale":"2024-04-25T00:00:47.147213+0000","last_undegraded":"2024-04-25T00:00:47.147213+0000","last_fullsized":"2024-04-25T00:00:47.147213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":3996,"log_dups_size":0,"ondisk_log_size":3996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:39.457116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4001,"num_write_kb":39473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"27'9856","reported_seq":10228,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.120845+0000","last_change":"2024-04-24T23:34:58.399188+0000","last_active":"2024-04-25T00:00:50.120845+0000","last_peered":"2024-04-25T00:00:50.120845+0000","last_clean":"2024-04-25T00:00:50.120845+0000","last_became_active":"2024-04-24T23:34:58.398457+0000","last_became_peered":"2024-04-24T23:34:58.398457+0000","last_unstale":"2024-04-25T00:00:50.120845+0000","last_undegraded":"2024-04-25T00:00:50.120845+0000","last_fullsized":"2024-04-25T00:00:50.120845+0000","mapping_epoch":23,"log_start":"27'2619","ondisk_log_start":"27'2619","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7237,"log_dups_size":0,"ondisk_log_size":7237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:50:21.658540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":344,"num_read_kb":87376,"num_write":7128,"num_write_kb":1170154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"27'2449","reported_seq":2484,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:44.996971+0000","last_change":"2024-04-24T23:34:54.372150+0000","last_active":"2024-04-25T00:00:44.996971+0000","last_peered":"2024-04-25T00:00:44.996971+0000","last_clean":"2024-04-25T00:00:44.996971+0000","last_became_active":"2024-04-24T23:34:54.371703+0000","last_became_peered":"2024-04-24T23:34:54.371703+0000","last_unstale":"2024-04-25T00:00:44.996971+0000","last_undegraded":"2024-04-25T00:00:44.996971+0000","last_fullsized":"2024-04-25T00:00:44.996971+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2449,"log_dups_size":0,"ondisk_log_size":2449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:15.477061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"27'4711","reported_seq":4748,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:25.348241+0000","last_change":"2024-04-24T23:34:53.363787+0000","last_active":"2024-04-25T00:00:25.348241+0000","last_peered":"2024-04-25T00:00:25.348241+0000","last_clean":"2024-04-25T00:00:25.348241+0000","last_became_active":"2024-04-24T23:34:53.363515+0000","last_became_peered":"2024-04-24T23:34:53.363515+0000","last_unstale":"2024-04-25T00:00:25.348241+0000","last_undegraded":"2024-04-25T00:00:25.348241+0000","last_fullsized":"2024-04-25T00:00:25.348241+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":4711,"log_dups_size":0,"ondisk_log_size":4711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:50.222015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4704,"num_write_kb":43976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"27'9316","reported_seq":9703,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:46.062715+0000","last_change":"2024-04-24T23:34:58.728420+0000","last_active":"2024-04-25T00:00:46.062715+0000","last_peered":"2024-04-25T00:00:46.062715+0000","last_clean":"2024-04-25T00:00:46.062715+0000","last_became_active":"2024-04-24T23:34:58.727425+0000","last_became_peered":"2024-04-24T23:34:58.727425+0000","last_unstale":"2024-04-25T00:00:46.062715+0000","last_undegraded":"2024-04-25T00:00:46.062715+0000","last_fullsized":"2024-04-25T00:00:46.062715+0000","mapping_epoch":23,"log_start":"27'3035","ondisk_log_start":"27'3035","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":6281,"log_dups_size":0,"ondisk_log_size":6281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:19:14.487689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4376076,"num_objects":25,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":367,"num_read_kb":93218,"num_write":6599,"num_write_kb":922243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"27'2557","reported_seq":2591,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.325385+0000","last_change":"2024-04-24T23:34:54.379296+0000","last_active":"2024-04-25T00:00:47.325385+0000","last_peered":"2024-04-25T00:00:47.325385+0000","last_clean":"2024-04-25T00:00:47.325385+0000","last_became_active":"2024-04-24T23:34:54.379016+0000","last_became_peered":"2024-04-24T23:34:54.379016+0000","last_unstale":"2024-04-25T00:00:47.325385+0000","last_undegraded":"2024-04-25T00:00:47.325385+0000","last_fullsized":"2024-04-25T00:00:47.325385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2557,"log_dups_size":0,"ondisk_log_size":2557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:37.323551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"27'2767","reported_seq":2804,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.170814+0000","last_change":"2024-04-24T23:34:53.360210+0000","last_active":"2024-04-25T00:00:47.170814+0000","last_peered":"2024-04-25T00:00:47.170814+0000","last_clean":"2024-04-25T00:00:47.170814+0000","last_became_active":"2024-04-24T23:34:53.359581+0000","last_became_peered":"2024-04-24T23:34:53.359581+0000","last_unstale":"2024-04-25T00:00:47.170814+0000","last_undegraded":"2024-04-25T00:00:47.170814+0000","last_fullsized":"2024-04-25T00:00:47.170814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":2767,"log_dups_size":0,"ondisk_log_size":2767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:14.716741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2773,"num_write_kb":30370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"27'9043","reported_seq":9471,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:45.335278+0000","last_change":"2024-04-24T23:34:58.728491+0000","last_active":"2024-04-25T00:00:45.335278+0000","last_peered":"2024-04-25T00:00:45.335278+0000","last_clean":"2024-04-25T00:00:45.335278+0000","last_became_active":"2024-04-24T23:34:58.727686+0000","last_became_peered":"2024-04-24T23:34:58.727686+0000","last_unstale":"2024-04-25T00:00:45.335278+0000","last_undegraded":"2024-04-25T00:00:45.335278+0000","last_fullsized":"2024-04-25T00:00:45.335278+0000","mapping_epoch":23,"log_start":"27'2730","ondisk_log_start":"27'2730","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":6313,"log_dups_size":0,"ondisk_log_size":6313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:46.282712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4413096,"num_objects":43,"num_object_clones":0,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":441,"num_read_kb":112014,"num_write":6490,"num_write_kb":907150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"27'2526","reported_seq":2561,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.023712+0000","last_change":"2024-04-24T23:34:54.372009+0000","last_active":"2024-04-25T00:00:50.023712+0000","last_peered":"2024-04-25T00:00:50.023712+0000","last_clean":"2024-04-25T00:00:50.023712+0000","last_became_active":"2024-04-24T23:34:54.371720+0000","last_became_peered":"2024-04-24T23:34:54.371720+0000","last_unstale":"2024-04-25T00:00:50.023712+0000","last_undegraded":"2024-04-25T00:00:50.023712+0000","last_fullsized":"2024-04-25T00:00:50.023712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2526,"log_dups_size":0,"ondisk_log_size":2526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:04:07.487367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"27'6531","reported_seq":6567,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:35.064277+0000","last_change":"2024-04-24T23:34:53.353738+0000","last_active":"2024-04-25T00:00:35.064277+0000","last_peered":"2024-04-25T00:00:35.064277+0000","last_clean":"2024-04-25T00:00:35.064277+0000","last_became_active":"2024-04-24T23:34:53.353452+0000","last_became_peered":"2024-04-24T23:34:53.353452+0000","last_unstale":"2024-04-25T00:00:35.064277+0000","last_undegraded":"2024-04-25T00:00:35.064277+0000","last_fullsized":"2024-04-25T00:00:35.064277+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":6531,"log_dups_size":0,"ondisk_log_size":6531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:33:43.693095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971554,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6522,"num_write_kb":57515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"27'9599","reported_seq":10048,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:48.575361+0000","last_change":"2024-04-24T23:34:58.728503+0000","last_active":"2024-04-25T00:00:48.575361+0000","last_peered":"2024-04-25T00:00:48.575361+0000","last_clean":"2024-04-25T00:00:48.575361+0000","last_became_active":"2024-04-24T23:34:58.727921+0000","last_became_peered":"2024-04-24T23:34:58.727921+0000","last_unstale":"2024-04-25T00:00:48.575361+0000","last_undegraded":"2024-04-25T00:00:48.575361+0000","last_fullsized":"2024-04-25T00:00:48.575361+0000","mapping_epoch":23,"log_start":"27'1833","ondisk_log_start":"27'1833","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7766,"log_dups_size":0,"ondisk_log_size":7766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:19:40.928966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1518653,"num_objects":16,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":421,"num_read_kb":106934,"num_write":6985,"num_write_kb":1058941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"27'2469","reported_seq":2502,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:48.895019+0000","last_change":"2024-04-24T23:34:54.368532+0000","last_active":"2024-04-25T00:00:48.895019+0000","last_peered":"2024-04-25T00:00:48.895019+0000","last_clean":"2024-04-25T00:00:48.895019+0000","last_became_active":"2024-04-24T23:34:54.367903+0000","last_became_peered":"2024-04-24T23:34:54.367903+0000","last_unstale":"2024-04-25T00:00:48.895019+0000","last_undegraded":"2024-04-25T00:00:48.895019+0000","last_fullsized":"2024-04-25T00:00:48.895019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2469,"log_dups_size":0,"ondisk_log_size":2469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:49.372040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'9563","reported_seq":10200,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:34.614071+0000","last_change":"2024-04-24T23:34:53.362758+0000","last_active":"2024-04-25T00:00:34.614071+0000","last_peered":"2024-04-25T00:00:34.614071+0000","last_clean":"2024-04-25T00:00:34.614071+0000","last_became_active":"2024-04-24T23:34:53.362230+0000","last_became_peered":"2024-04-24T23:34:53.362230+0000","last_unstale":"2024-04-25T00:00:34.614071+0000","last_undegraded":"2024-04-25T00:00:34.614071+0000","last_fullsized":"2024-04-25T00:00:34.614071+0000","mapping_epoch":18,"log_start":"27'1600","ondisk_log_start":"27'1600","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":7963,"log_dups_size":0,"ondisk_log_size":7963,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:14.716741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389280,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":602,"num_read_kb":4494,"num_write":9563,"num_write_kb":76044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"27'8959","reported_seq":9364,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:45.247476+0000","last_change":"2024-04-24T23:34:58.400881+0000","last_active":"2024-04-25T00:00:45.247476+0000","last_peered":"2024-04-25T00:00:45.247476+0000","last_clean":"2024-04-25T00:00:45.247476+0000","last_became_active":"2024-04-24T23:34:58.400429+0000","last_became_peered":"2024-04-24T23:34:58.400429+0000","last_unstale":"2024-04-25T00:00:45.247476+0000","last_undegraded":"2024-04-25T00:00:45.247476+0000","last_fullsized":"2024-04-25T00:00:45.247476+0000","mapping_epoch":23,"log_start":"27'1625","ondisk_log_start":"27'1625","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7334,"log_dups_size":0,"ondisk_log_size":7334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:50:21.658540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1794492,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":377,"num_read_kb":95758,"num_write":6289,"num_write_kb":953618,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"27'2530","reported_seq":2564,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:44.880833+0000","last_change":"2024-04-24T23:34:54.373603+0000","last_active":"2024-04-25T00:00:44.880833+0000","last_peered":"2024-04-25T00:00:44.880833+0000","last_clean":"2024-04-25T00:00:44.880833+0000","last_became_active":"2024-04-24T23:34:54.373347+0000","last_became_peered":"2024-04-24T23:34:54.373347+0000","last_unstale":"2024-04-25T00:00:44.880833+0000","last_undegraded":"2024-04-25T00:00:44.880833+0000","last_fullsized":"2024-04-25T00:00:44.880833+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2530,"log_dups_size":0,"ondisk_log_size":2530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:13.263529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"27'5921","reported_seq":5957,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:42.665448+0000","last_change":"2024-04-24T23:34:53.354673+0000","last_active":"2024-04-25T00:00:42.665448+0000","last_peered":"2024-04-25T00:00:42.665448+0000","last_clean":"2024-04-25T00:00:42.665448+0000","last_became_active":"2024-04-24T23:34:53.354394+0000","last_became_peered":"2024-04-24T23:34:53.354394+0000","last_unstale":"2024-04-25T00:00:42.665448+0000","last_undegraded":"2024-04-25T00:00:42.665448+0000","last_fullsized":"2024-04-25T00:00:42.665448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":5921,"log_dups_size":0,"ondisk_log_size":5921,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:14.716741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5922,"num_write_kb":41234,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"27'9078","reported_seq":9483,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:46.574937+0000","last_change":"2024-04-24T23:34:58.397739+0000","last_active":"2024-04-25T00:00:46.574937+0000","last_peered":"2024-04-25T00:00:46.574937+0000","last_clean":"2024-04-25T00:00:46.574937+0000","last_became_active":"2024-04-24T23:34:58.397353+0000","last_became_peered":"2024-04-24T23:34:58.397353+0000","last_unstale":"2024-04-25T00:00:46.574937+0000","last_undegraded":"2024-04-25T00:00:46.574937+0000","last_fullsized":"2024-04-25T00:00:46.574937+0000","mapping_epoch":23,"log_start":"27'1901","ondisk_log_start":"27'1901","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7177,"log_dups_size":0,"ondisk_log_size":7177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:55.672101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4144570,"num_objects":28,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":378,"num_read_kb":96012,"num_write":6395,"num_write_kb":974866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"27'2564","reported_seq":2598,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:49.547142+0000","last_change":"2024-04-24T23:34:54.383348+0000","last_active":"2024-04-25T00:00:49.547142+0000","last_peered":"2024-04-25T00:00:49.547142+0000","last_clean":"2024-04-25T00:00:49.547142+0000","last_became_active":"2024-04-24T23:34:54.383082+0000","last_became_peered":"2024-04-24T23:34:54.383082+0000","last_unstale":"2024-04-25T00:00:49.547142+0000","last_undegraded":"2024-04-25T00:00:49.547142+0000","last_fullsized":"2024-04-25T00:00:49.547142+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2564,"log_dups_size":0,"ondisk_log_size":2564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:56:17.763159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"27'7101","reported_seq":7137,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.223449+0000","last_change":"2024-04-24T23:34:53.364540+0000","last_active":"2024-04-25T00:00:47.223449+0000","last_peered":"2024-04-25T00:00:47.223449+0000","last_clean":"2024-04-25T00:00:47.223449+0000","last_became_active":"2024-04-24T23:34:53.364259+0000","last_became_peered":"2024-04-24T23:34:53.364259+0000","last_unstale":"2024-04-25T00:00:47.223449+0000","last_undegraded":"2024-04-25T00:00:47.223449+0000","last_fullsized":"2024-04-25T00:00:47.223449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":7101,"log_dups_size":0,"ondisk_log_size":7101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:41.422591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":7102,"num_write_kb":53865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"27'8979","reported_seq":9068,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:45.779269+0000","last_change":"2024-04-24T23:34:58.405865+0000","last_active":"2024-04-25T00:00:45.779269+0000","last_peered":"2024-04-25T00:00:45.779269+0000","last_clean":"2024-04-25T00:00:45.779269+0000","last_became_active":"2024-04-24T23:34:58.405032+0000","last_became_peered":"2024-04-24T23:34:58.405032+0000","last_unstale":"2024-04-25T00:00:45.779269+0000","last_undegraded":"2024-04-25T00:00:45.779269+0000","last_fullsized":"2024-04-25T00:00:45.779269+0000","mapping_epoch":23,"log_start":"27'2712","ondisk_log_start":"27'2712","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":6267,"log_dups_size":0,"ondisk_log_size":6267,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:54.012389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4968479,"num_objects":25,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":69,"num_read_kb":17526,"num_write":6345,"num_write_kb":965302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"27'2534","reported_seq":2568,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:48.509523+0000","last_change":"2024-04-24T23:34:54.383631+0000","last_active":"2024-04-25T00:00:48.509523+0000","last_peered":"2024-04-25T00:00:48.509523+0000","last_clean":"2024-04-25T00:00:48.509523+0000","last_became_active":"2024-04-24T23:34:54.383354+0000","last_became_peered":"2024-04-24T23:34:54.383354+0000","last_unstale":"2024-04-25T00:00:48.509523+0000","last_undegraded":"2024-04-25T00:00:48.509523+0000","last_fullsized":"2024-04-25T00:00:48.509523+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2534,"log_dups_size":0,"ondisk_log_size":2534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:50:17.621315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"27'6415","reported_seq":7297,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-24T23:59:16.514501+0000","last_change":"2024-04-24T23:34:53.362409+0000","last_active":"2024-04-24T23:59:16.514501+0000","last_peered":"2024-04-24T23:59:16.514501+0000","last_clean":"2024-04-24T23:59:16.514501+0000","last_became_active":"2024-04-24T23:34:53.362103+0000","last_became_peered":"2024-04-24T23:34:53.362103+0000","last_unstale":"2024-04-24T23:59:16.514501+0000","last_undegraded":"2024-04-24T23:59:16.514501+0000","last_fullsized":"2024-04-24T23:59:16.514501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":6415,"log_dups_size":0,"ondisk_log_size":6415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:57:14.381644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":847,"num_read_kb":4657,"num_write":6414,"num_write_kb":49337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"27'9358","reported_seq":9642,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:45.950273+0000","last_change":"2024-04-24T23:34:58.730088+0000","last_active":"2024-04-25T00:00:45.950273+0000","last_peered":"2024-04-25T00:00:45.950273+0000","last_clean":"2024-04-25T00:00:45.950273+0000","last_became_active":"2024-04-24T23:34:58.729217+0000","last_became_peered":"2024-04-24T23:34:58.729217+0000","last_unstale":"2024-04-25T00:00:45.950273+0000","last_undegraded":"2024-04-25T00:00:45.950273+0000","last_fullsized":"2024-04-25T00:00:45.950273+0000","mapping_epoch":23,"log_start":"27'3013","ondisk_log_start":"27'3013","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":6345,"log_dups_size":0,"ondisk_log_size":6345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:12:14.243770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1226372,"num_objects":22,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":259,"num_read_kb":65786,"num_write":6570,"num_write_kb":1055419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"27'9147","reported_seq":9276,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:49.915079+0000","last_change":"2024-04-24T23:34:58.728892+0000","last_active":"2024-04-25T00:00:49.915079+0000","last_peered":"2024-04-25T00:00:49.915079+0000","last_clean":"2024-04-25T00:00:49.915079+0000","last_became_active":"2024-04-24T23:34:58.727914+0000","last_became_peered":"2024-04-24T23:34:58.727914+0000","last_unstale":"2024-04-25T00:00:49.915079+0000","last_undegraded":"2024-04-25T00:00:49.915079+0000","last_fullsized":"2024-04-25T00:00:49.915079+0000","mapping_epoch":23,"log_start":"27'1931","ondisk_log_start":"27'1931","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7216,"log_dups_size":0,"ondisk_log_size":7216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:56.609480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":99,"num_read_kb":25146,"num_write":6551,"num_write_kb":920441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"27'2477","reported_seq":2511,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:44.578170+0000","last_change":"2024-04-24T23:34:54.363973+0000","last_active":"2024-04-25T00:00:44.578170+0000","last_peered":"2024-04-25T00:00:44.578170+0000","last_clean":"2024-04-25T00:00:44.578170+0000","last_became_active":"2024-04-24T23:34:54.363724+0000","last_became_peered":"2024-04-24T23:34:54.363724+0000","last_unstale":"2024-04-25T00:00:44.578170+0000","last_undegraded":"2024-04-25T00:00:44.578170+0000","last_fullsized":"2024-04-25T00:00:44.578170+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2477,"log_dups_size":0,"ondisk_log_size":2477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:03:24.745895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"27'9687","reported_seq":9723,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.373826+0000","last_change":"2024-04-24T23:34:53.360041+0000","last_active":"2024-04-25T00:00:47.373826+0000","last_peered":"2024-04-25T00:00:47.373826+0000","last_clean":"2024-04-25T00:00:47.373826+0000","last_became_active":"2024-04-24T23:34:53.359707+0000","last_became_peered":"2024-04-24T23:34:53.359707+0000","last_unstale":"2024-04-25T00:00:47.373826+0000","last_undegraded":"2024-04-25T00:00:47.373826+0000","last_fullsized":"2024-04-25T00:00:47.373826+0000","mapping_epoch":18,"log_start":"27'1900","ondisk_log_start":"27'1900","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":7787,"log_dups_size":0,"ondisk_log_size":7787,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:50.222015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9677,"num_write_kb":67972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"27'9562","reported_seq":10651,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:46.082792+0000","last_change":"2024-04-24T23:35:43.834517+0000","last_active":"2024-04-25T00:00:46.082792+0000","last_peered":"2024-04-25T00:00:46.082792+0000","last_clean":"2024-04-25T00:00:46.082792+0000","last_became_active":"2024-04-24T23:35:43.834066+0000","last_became_peered":"2024-04-24T23:35:43.834066+0000","last_unstale":"2024-04-25T00:00:46.082792+0000","last_undegraded":"2024-04-25T00:00:46.082792+0000","last_fullsized":"2024-04-25T00:00:46.082792+0000","mapping_epoch":26,"log_start":"27'1608","ondisk_log_start":"27'1608","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7954,"log_dups_size":0,"ondisk_log_size":7954,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:48.286783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5985156,"num_objects":38,"num_object_clones":0,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":1054,"num_read_kb":267716,"num_write":6861,"num_write_kb":899615,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2,0],"acting":[6,5,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"27'2568","reported_seq":2602,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:44.370129+0000","last_change":"2024-04-24T23:34:54.369729+0000","last_active":"2024-04-25T00:00:44.370129+0000","last_peered":"2024-04-25T00:00:44.370129+0000","last_clean":"2024-04-25T00:00:44.370129+0000","last_became_active":"2024-04-24T23:34:54.369155+0000","last_became_peered":"2024-04-24T23:34:54.369155+0000","last_unstale":"2024-04-25T00:00:44.370129+0000","last_undegraded":"2024-04-25T00:00:44.370129+0000","last_fullsized":"2024-04-25T00:00:44.370129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2568,"log_dups_size":0,"ondisk_log_size":2568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:46:28.181996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"27'4840","reported_seq":4876,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-24T23:59:32.942342+0000","last_change":"2024-04-24T23:34:53.365328+0000","last_active":"2024-04-24T23:59:32.942342+0000","last_peered":"2024-04-24T23:59:32.942342+0000","last_clean":"2024-04-24T23:59:32.942342+0000","last_became_active":"2024-04-24T23:34:53.365029+0000","last_became_peered":"2024-04-24T23:34:53.365029+0000","last_unstale":"2024-04-24T23:59:32.942342+0000","last_undegraded":"2024-04-24T23:59:32.942342+0000","last_fullsized":"2024-04-24T23:59:32.942342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":4840,"log_dups_size":0,"ondisk_log_size":4840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:48:30.961870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4845,"num_write_kb":44132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"27'9225","reported_seq":9261,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-24T23:59:35.593651+0000","last_change":"2024-04-24T23:34:53.365123+0000","last_active":"2024-04-24T23:59:35.593651+0000","last_peered":"2024-04-24T23:59:35.593651+0000","last_clean":"2024-04-24T23:59:35.593651+0000","last_became_active":"2024-04-24T23:34:53.364821+0000","last_became_peered":"2024-04-24T23:34:53.364821+0000","last_unstale":"2024-04-24T23:59:35.593651+0000","last_undegraded":"2024-04-24T23:59:35.593651+0000","last_fullsized":"2024-04-24T23:59:35.593651+0000","mapping_epoch":18,"log_start":"27'1300","ondisk_log_start":"27'1300","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":7925,"log_dups_size":0,"ondisk_log_size":7925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:39.457116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9217,"num_write_kb":71599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"27'2524","reported_seq":2558,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.139570+0000","last_change":"2024-04-24T23:34:54.376037+0000","last_active":"2024-04-25T00:00:50.139570+0000","last_peered":"2024-04-25T00:00:50.139570+0000","last_clean":"2024-04-25T00:00:50.139570+0000","last_became_active":"2024-04-24T23:34:54.375689+0000","last_became_peered":"2024-04-24T23:34:54.375689+0000","last_unstale":"2024-04-25T00:00:50.139570+0000","last_undegraded":"2024-04-25T00:00:50.139570+0000","last_fullsized":"2024-04-25T00:00:50.139570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2524,"log_dups_size":0,"ondisk_log_size":2524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:47.446459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"27'9696","reported_seq":9920,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:48.257507+0000","last_change":"2024-04-24T23:35:43.834538+0000","last_active":"2024-04-25T00:00:48.257507+0000","last_peered":"2024-04-25T00:00:48.257507+0000","last_clean":"2024-04-25T00:00:48.257507+0000","last_became_active":"2024-04-24T23:35:43.834061+0000","last_became_peered":"2024-04-24T23:35:43.834061+0000","last_unstale":"2024-04-25T00:00:48.257507+0000","last_undegraded":"2024-04-25T00:00:48.257507+0000","last_fullsized":"2024-04-25T00:00:48.257507+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":9696,"log_dups_size":0,"ondisk_log_size":9696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:41:52.244767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3304017,"num_objects":33,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":189,"num_read_kb":48006,"num_write":6965,"num_write_kb":973944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3,5],"acting":[2,0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"27'7146","reported_seq":7182,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-24T23:58:12.312669+0000","last_change":"2024-04-24T23:34:53.363066+0000","last_active":"2024-04-24T23:58:12.312669+0000","last_peered":"2024-04-24T23:58:12.312669+0000","last_clean":"2024-04-24T23:58:12.312669+0000","last_became_active":"2024-04-24T23:34:53.362548+0000","last_became_peered":"2024-04-24T23:34:53.362548+0000","last_unstale":"2024-04-24T23:58:12.312669+0000","last_undegraded":"2024-04-24T23:58:12.312669+0000","last_fullsized":"2024-04-24T23:58:12.312669+0000","mapping_epoch":18,"log_start":"27'800","ondisk_log_start":"27'800","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":6346,"log_dups_size":0,"ondisk_log_size":6346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:21:13.338964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7133,"num_write_kb":66117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"27'2549","reported_seq":2583,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:44.471789+0000","last_change":"2024-04-24T23:34:54.365704+0000","last_active":"2024-04-25T00:00:44.471789+0000","last_peered":"2024-04-25T00:00:44.471789+0000","last_clean":"2024-04-25T00:00:44.471789+0000","last_became_active":"2024-04-24T23:34:54.364814+0000","last_became_peered":"2024-04-24T23:34:54.364814+0000","last_unstale":"2024-04-25T00:00:44.471789+0000","last_undegraded":"2024-04-25T00:00:44.471789+0000","last_fullsized":"2024-04-25T00:00:44.471789+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2549,"log_dups_size":0,"ondisk_log_size":2549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:22.094785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"27'9291","reported_seq":9497,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:45.850805+0000","last_change":"2024-04-24T23:34:58.407186+0000","last_active":"2024-04-25T00:00:45.850805+0000","last_peered":"2024-04-25T00:00:45.850805+0000","last_clean":"2024-04-25T00:00:45.850805+0000","last_became_active":"2024-04-24T23:34:58.406811+0000","last_became_peered":"2024-04-24T23:34:58.406811+0000","last_unstale":"2024-04-25T00:00:45.850805+0000","last_undegraded":"2024-04-25T00:00:45.850805+0000","last_fullsized":"2024-04-25T00:00:45.850805+0000","mapping_epoch":23,"log_start":"27'3012","ondisk_log_start":"27'3012","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":6279,"log_dups_size":0,"ondisk_log_size":6279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:55.861555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6226351,"num_objects":17,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":178,"num_read_kb":45212,"num_write":6534,"num_write_kb":1041117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"27'2439","reported_seq":2475,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:44.473291+0000","last_change":"2024-04-24T23:34:54.375142+0000","last_active":"2024-04-25T00:00:44.473291+0000","last_peered":"2024-04-25T00:00:44.473291+0000","last_clean":"2024-04-25T00:00:44.473291+0000","last_became_active":"2024-04-24T23:34:54.374844+0000","last_became_peered":"2024-04-24T23:34:54.374844+0000","last_unstale":"2024-04-25T00:00:44.473291+0000","last_undegraded":"2024-04-25T00:00:44.473291+0000","last_fullsized":"2024-04-25T00:00:44.473291+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2439,"log_dups_size":0,"ondisk_log_size":2439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:29:07.593498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"27'2583","reported_seq":2619,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:48.181599+0000","last_change":"2024-04-24T23:34:54.383574+0000","last_active":"2024-04-25T00:00:48.181599+0000","last_peered":"2024-04-25T00:00:48.181599+0000","last_clean":"2024-04-25T00:00:48.181599+0000","last_became_active":"2024-04-24T23:34:54.383253+0000","last_became_peered":"2024-04-24T23:34:54.383253+0000","last_unstale":"2024-04-25T00:00:48.181599+0000","last_undegraded":"2024-04-25T00:00:48.181599+0000","last_fullsized":"2024-04-25T00:00:48.181599+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2583,"log_dups_size":0,"ondisk_log_size":2583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:43.211314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"27'2465","reported_seq":2487,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.944438+0000","last_change":"2024-04-24T23:35:43.526600+0000","last_active":"2024-04-25T00:00:47.944438+0000","last_peered":"2024-04-25T00:00:47.944438+0000","last_clean":"2024-04-25T00:00:47.944438+0000","last_became_active":"2024-04-24T23:35:43.525929+0000","last_became_peered":"2024-04-24T23:35:43.525929+0000","last_unstale":"2024-04-25T00:00:47.944438+0000","last_undegraded":"2024-04-25T00:00:47.944438+0000","last_fullsized":"2024-04-25T00:00:47.944438+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2465,"log_dups_size":0,"ondisk_log_size":2465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:49.372040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"27'2575","reported_seq":2611,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:48.937515+0000","last_change":"2024-04-24T23:34:54.366271+0000","last_active":"2024-04-25T00:00:48.937515+0000","last_peered":"2024-04-25T00:00:48.937515+0000","last_clean":"2024-04-25T00:00:48.937515+0000","last_became_active":"2024-04-24T23:34:54.366005+0000","last_became_peered":"2024-04-24T23:34:54.366005+0000","last_unstale":"2024-04-25T00:00:48.937515+0000","last_undegraded":"2024-04-25T00:00:48.937515+0000","last_fullsized":"2024-04-25T00:00:48.937515+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2575,"log_dups_size":0,"ondisk_log_size":2575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:03:24.745895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"27'2559","reported_seq":2594,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:44.819878+0000","last_change":"2024-04-24T23:34:54.360529+0000","last_active":"2024-04-25T00:00:44.819878+0000","last_peered":"2024-04-25T00:00:44.819878+0000","last_clean":"2024-04-25T00:00:44.819878+0000","last_became_active":"2024-04-24T23:34:54.360266+0000","last_became_peered":"2024-04-24T23:34:54.360266+0000","last_unstale":"2024-04-25T00:00:44.819878+0000","last_undegraded":"2024-04-25T00:00:44.819878+0000","last_fullsized":"2024-04-25T00:00:44.819878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2559,"log_dups_size":0,"ondisk_log_size":2559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:49:47.246193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"27'2432","reported_seq":2467,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:44.680703+0000","last_change":"2024-04-24T23:34:54.365744+0000","last_active":"2024-04-25T00:00:44.680703+0000","last_peered":"2024-04-25T00:00:44.680703+0000","last_clean":"2024-04-25T00:00:44.680703+0000","last_became_active":"2024-04-24T23:34:54.365493+0000","last_became_peered":"2024-04-24T23:34:54.365493+0000","last_unstale":"2024-04-25T00:00:44.680703+0000","last_undegraded":"2024-04-25T00:00:44.680703+0000","last_fullsized":"2024-04-25T00:00:44.680703+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2432,"log_dups_size":0,"ondisk_log_size":2432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:15.477061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"27'2562","reported_seq":2597,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:44.988811+0000","last_change":"2024-04-24T23:34:54.379642+0000","last_active":"2024-04-25T00:00:44.988811+0000","last_peered":"2024-04-25T00:00:44.988811+0000","last_clean":"2024-04-25T00:00:44.988811+0000","last_became_active":"2024-04-24T23:34:54.378568+0000","last_became_peered":"2024-04-24T23:34:54.378568+0000","last_unstale":"2024-04-25T00:00:44.988811+0000","last_undegraded":"2024-04-25T00:00:44.988811+0000","last_fullsized":"2024-04-25T00:00:44.988811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2562,"log_dups_size":0,"ondisk_log_size":2562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:42.199856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"27'2530","reported_seq":2565,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:45.015897+0000","last_change":"2024-04-24T23:34:54.363700+0000","last_active":"2024-04-25T00:00:45.015897+0000","last_peered":"2024-04-25T00:00:45.015897+0000","last_clean":"2024-04-25T00:00:45.015897+0000","last_became_active":"2024-04-24T23:34:54.363439+0000","last_became_peered":"2024-04-24T23:34:54.363439+0000","last_unstale":"2024-04-25T00:00:45.015897+0000","last_undegraded":"2024-04-25T00:00:45.015897+0000","last_fullsized":"2024-04-25T00:00:45.015897+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2530,"log_dups_size":0,"ondisk_log_size":2530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:28:56.048178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"27'2531","reported_seq":2566,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:48.430106+0000","last_change":"2024-04-24T23:34:54.375460+0000","last_active":"2024-04-25T00:00:48.430106+0000","last_peered":"2024-04-25T00:00:48.430106+0000","last_clean":"2024-04-25T00:00:48.430106+0000","last_became_active":"2024-04-24T23:34:54.375123+0000","last_became_peered":"2024-04-24T23:34:54.375123+0000","last_unstale":"2024-04-25T00:00:48.430106+0000","last_undegraded":"2024-04-25T00:00:48.430106+0000","last_fullsized":"2024-04-25T00:00:48.430106+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2531,"log_dups_size":0,"ondisk_log_size":2531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:18.207267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"27'2496","reported_seq":2531,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:44.958661+0000","last_change":"2024-04-24T23:34:54.369509+0000","last_active":"2024-04-25T00:00:44.958661+0000","last_peered":"2024-04-25T00:00:44.958661+0000","last_clean":"2024-04-25T00:00:44.958661+0000","last_became_active":"2024-04-24T23:34:54.369219+0000","last_became_peered":"2024-04-24T23:34:54.369219+0000","last_unstale":"2024-04-25T00:00:44.958661+0000","last_undegraded":"2024-04-25T00:00:44.958661+0000","last_fullsized":"2024-04-25T00:00:44.958661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2496,"log_dups_size":0,"ondisk_log_size":2496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:09:37.649503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"27'2551","reported_seq":2572,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:48.387420+0000","last_change":"2024-04-24T23:35:43.933911+0000","last_active":"2024-04-25T00:00:48.387420+0000","last_peered":"2024-04-25T00:00:48.387420+0000","last_clean":"2024-04-25T00:00:48.387420+0000","last_became_active":"2024-04-24T23:35:43.933686+0000","last_became_peered":"2024-04-24T23:35:43.933686+0000","last_unstale":"2024-04-25T00:00:48.387420+0000","last_undegraded":"2024-04-25T00:00:48.387420+0000","last_fullsized":"2024-04-25T00:00:48.387420+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2551,"log_dups_size":0,"ondisk_log_size":2551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:26.261786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2b","version":"27'2585","reported_seq":2620,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:48.808345+0000","last_change":"2024-04-24T23:34:54.371052+0000","last_active":"2024-04-25T00:00:48.808345+0000","last_peered":"2024-04-25T00:00:48.808345+0000","last_clean":"2024-04-25T00:00:48.808345+0000","last_became_active":"2024-04-24T23:34:54.370793+0000","last_became_peered":"2024-04-24T23:34:54.370793+0000","last_unstale":"2024-04-25T00:00:48.808345+0000","last_undegraded":"2024-04-25T00:00:48.808345+0000","last_fullsized":"2024-04-25T00:00:48.808345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2585,"log_dups_size":0,"ondisk_log_size":2585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:29:07.593498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"27'2564","reported_seq":2599,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:45.080910+0000","last_change":"2024-04-24T23:34:54.377190+0000","last_active":"2024-04-25T00:00:45.080910+0000","last_peered":"2024-04-25T00:00:45.080910+0000","last_clean":"2024-04-25T00:00:45.080910+0000","last_became_active":"2024-04-24T23:34:54.376940+0000","last_became_peered":"2024-04-24T23:34:54.376940+0000","last_unstale":"2024-04-25T00:00:45.080910+0000","last_undegraded":"2024-04-25T00:00:45.080910+0000","last_fullsized":"2024-04-25T00:00:45.080910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2564,"log_dups_size":0,"ondisk_log_size":2564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:03:24.745895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"27'2570","reported_seq":2604,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:49.071176+0000","last_change":"2024-04-24T23:34:54.368770+0000","last_active":"2024-04-25T00:00:49.071176+0000","last_peered":"2024-04-25T00:00:49.071176+0000","last_clean":"2024-04-25T00:00:49.071176+0000","last_became_active":"2024-04-24T23:34:54.368228+0000","last_became_peered":"2024-04-24T23:34:54.368228+0000","last_unstale":"2024-04-25T00:00:49.071176+0000","last_undegraded":"2024-04-25T00:00:49.071176+0000","last_fullsized":"2024-04-25T00:00:49.071176+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2570,"log_dups_size":0,"ondisk_log_size":2570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:51.634876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"27'2510","reported_seq":2545,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.391247+0000","last_change":"2024-04-24T23:34:54.373038+0000","last_active":"2024-04-25T00:00:47.391247+0000","last_peered":"2024-04-25T00:00:47.391247+0000","last_clean":"2024-04-25T00:00:47.391247+0000","last_became_active":"2024-04-24T23:34:54.372766+0000","last_became_peered":"2024-04-24T23:34:54.372766+0000","last_unstale":"2024-04-25T00:00:47.391247+0000","last_undegraded":"2024-04-25T00:00:47.391247+0000","last_fullsized":"2024-04-25T00:00:47.391247+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2510,"log_dups_size":0,"ondisk_log_size":2510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:13.263529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"27'2577","reported_seq":2612,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.872643+0000","last_change":"2024-04-24T23:34:54.383733+0000","last_active":"2024-04-25T00:00:47.872643+0000","last_peered":"2024-04-25T00:00:47.872643+0000","last_clean":"2024-04-25T00:00:47.872643+0000","last_became_active":"2024-04-24T23:34:54.383463+0000","last_became_peered":"2024-04-24T23:34:54.383463+0000","last_unstale":"2024-04-25T00:00:47.872643+0000","last_undegraded":"2024-04-25T00:00:47.872643+0000","last_fullsized":"2024-04-25T00:00:47.872643+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2577,"log_dups_size":0,"ondisk_log_size":2577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:51.634876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"27'2508","reported_seq":2543,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:48.402598+0000","last_change":"2024-04-24T23:34:54.380959+0000","last_active":"2024-04-25T00:00:48.402598+0000","last_peered":"2024-04-25T00:00:48.402598+0000","last_clean":"2024-04-25T00:00:48.402598+0000","last_became_active":"2024-04-24T23:34:54.380702+0000","last_became_peered":"2024-04-24T23:34:54.380702+0000","last_unstale":"2024-04-25T00:00:48.402598+0000","last_undegraded":"2024-04-25T00:00:48.402598+0000","last_fullsized":"2024-04-25T00:00:48.402598+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2508,"log_dups_size":0,"ondisk_log_size":2508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:45.424046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"27'2625","reported_seq":2660,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:44.836426+0000","last_change":"2024-04-24T23:34:54.358461+0000","last_active":"2024-04-25T00:00:44.836426+0000","last_peered":"2024-04-25T00:00:44.836426+0000","last_clean":"2024-04-25T00:00:44.836426+0000","last_became_active":"2024-04-24T23:34:54.358203+0000","last_became_peered":"2024-04-24T23:34:54.358203+0000","last_unstale":"2024-04-25T00:00:44.836426+0000","last_undegraded":"2024-04-25T00:00:44.836426+0000","last_fullsized":"2024-04-25T00:00:44.836426+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2625,"log_dups_size":0,"ondisk_log_size":2625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:18.207267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"27'2489","reported_seq":2524,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:49.786683+0000","last_change":"2024-04-24T23:34:54.367860+0000","last_active":"2024-04-25T00:00:49.786683+0000","last_peered":"2024-04-25T00:00:49.786683+0000","last_clean":"2024-04-25T00:00:49.786683+0000","last_became_active":"2024-04-24T23:34:54.367595+0000","last_became_peered":"2024-04-24T23:34:54.367595+0000","last_unstale":"2024-04-25T00:00:49.786683+0000","last_undegraded":"2024-04-25T00:00:49.786683+0000","last_fullsized":"2024-04-25T00:00:49.786683+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2489,"log_dups_size":0,"ondisk_log_size":2489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:40:40.928386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"27'2462","reported_seq":2482,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:45.113749+0000","last_change":"2024-04-24T23:35:43.529991+0000","last_active":"2024-04-25T00:00:45.113749+0000","last_peered":"2024-04-25T00:00:45.113749+0000","last_clean":"2024-04-25T00:00:45.113749+0000","last_became_active":"2024-04-24T23:35:43.529631+0000","last_became_peered":"2024-04-24T23:35:43.529631+0000","last_unstale":"2024-04-25T00:00:45.113749+0000","last_undegraded":"2024-04-25T00:00:45.113749+0000","last_fullsized":"2024-04-25T00:00:45.113749+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2462,"log_dups_size":0,"ondisk_log_size":2462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:40:39.166560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.34","version":"27'2422","reported_seq":2456,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:44.924916+0000","last_change":"2024-04-24T23:34:54.377573+0000","last_active":"2024-04-25T00:00:44.924916+0000","last_peered":"2024-04-25T00:00:44.924916+0000","last_clean":"2024-04-25T00:00:44.924916+0000","last_became_active":"2024-04-24T23:34:54.377277+0000","last_became_peered":"2024-04-24T23:34:54.377277+0000","last_unstale":"2024-04-25T00:00:44.924916+0000","last_undegraded":"2024-04-25T00:00:44.924916+0000","last_fullsized":"2024-04-25T00:00:44.924916+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2422,"log_dups_size":0,"ondisk_log_size":2422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:43.211314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"27'2466","reported_seq":2500,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:44.929859+0000","last_change":"2024-04-24T23:34:54.388134+0000","last_active":"2024-04-25T00:00:44.929859+0000","last_peered":"2024-04-25T00:00:44.929859+0000","last_clean":"2024-04-25T00:00:44.929859+0000","last_became_active":"2024-04-24T23:34:54.387781+0000","last_became_peered":"2024-04-24T23:34:54.387781+0000","last_unstale":"2024-04-25T00:00:44.929859+0000","last_undegraded":"2024-04-25T00:00:44.929859+0000","last_fullsized":"2024-04-25T00:00:44.929859+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2466,"log_dups_size":0,"ondisk_log_size":2466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:56:17.763159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"27'2585","reported_seq":2619,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:44.928263+0000","last_change":"2024-04-24T23:34:54.388182+0000","last_active":"2024-04-25T00:00:44.928263+0000","last_peered":"2024-04-25T00:00:44.928263+0000","last_clean":"2024-04-25T00:00:44.928263+0000","last_became_active":"2024-04-24T23:34:54.387913+0000","last_became_peered":"2024-04-24T23:34:54.387913+0000","last_unstale":"2024-04-25T00:00:44.928263+0000","last_undegraded":"2024-04-25T00:00:44.928263+0000","last_fullsized":"2024-04-25T00:00:44.928263+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2585,"log_dups_size":0,"ondisk_log_size":2585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:18.207267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"27'2556","reported_seq":2589,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.919316+0000","last_change":"2024-04-24T23:34:54.383303+0000","last_active":"2024-04-25T00:00:47.919316+0000","last_peered":"2024-04-25T00:00:47.919316+0000","last_clean":"2024-04-25T00:00:47.919316+0000","last_became_active":"2024-04-24T23:34:54.383024+0000","last_became_peered":"2024-04-24T23:34:54.383024+0000","last_unstale":"2024-04-25T00:00:47.919316+0000","last_undegraded":"2024-04-25T00:00:47.919316+0000","last_fullsized":"2024-04-25T00:00:47.919316+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2556,"log_dups_size":0,"ondisk_log_size":2556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:45.424046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"27'2492","reported_seq":2526,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:44.928077+0000","last_change":"2024-04-24T23:34:54.382803+0000","last_active":"2024-04-25T00:00:44.928077+0000","last_peered":"2024-04-25T00:00:44.928077+0000","last_clean":"2024-04-25T00:00:44.928077+0000","last_became_active":"2024-04-24T23:34:54.382482+0000","last_became_peered":"2024-04-24T23:34:54.382482+0000","last_unstale":"2024-04-25T00:00:44.928077+0000","last_undegraded":"2024-04-25T00:00:44.928077+0000","last_fullsized":"2024-04-25T00:00:44.928077+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2492,"log_dups_size":0,"ondisk_log_size":2492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:27:33.721584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"27'2525","reported_seq":2558,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:49.069349+0000","last_change":"2024-04-24T23:34:54.369167+0000","last_active":"2024-04-25T00:00:49.069349+0000","last_peered":"2024-04-25T00:00:49.069349+0000","last_clean":"2024-04-25T00:00:49.069349+0000","last_became_active":"2024-04-24T23:34:54.368663+0000","last_became_peered":"2024-04-24T23:34:54.368663+0000","last_unstale":"2024-04-25T00:00:49.069349+0000","last_undegraded":"2024-04-25T00:00:49.069349+0000","last_fullsized":"2024-04-25T00:00:49.069349+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2525,"log_dups_size":0,"ondisk_log_size":2525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:18.207267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"27'2452","reported_seq":2485,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:48.608029+0000","last_change":"2024-04-24T23:34:54.380345+0000","last_active":"2024-04-25T00:00:48.608029+0000","last_peered":"2024-04-25T00:00:48.608029+0000","last_clean":"2024-04-25T00:00:48.608029+0000","last_became_active":"2024-04-24T23:34:54.380052+0000","last_became_peered":"2024-04-24T23:34:54.380052+0000","last_unstale":"2024-04-25T00:00:48.608029+0000","last_undegraded":"2024-04-25T00:00:48.608029+0000","last_fullsized":"2024-04-25T00:00:48.608029+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2452,"log_dups_size":0,"ondisk_log_size":2452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:42.199856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"27'2497","reported_seq":2530,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:49.181531+0000","last_change":"2024-04-24T23:34:54.365114+0000","last_active":"2024-04-25T00:00:49.181531+0000","last_peered":"2024-04-25T00:00:49.181531+0000","last_clean":"2024-04-25T00:00:49.181531+0000","last_became_active":"2024-04-24T23:34:54.364605+0000","last_became_peered":"2024-04-24T23:34:54.364605+0000","last_unstale":"2024-04-25T00:00:49.181531+0000","last_undegraded":"2024-04-25T00:00:49.181531+0000","last_fullsized":"2024-04-25T00:00:49.181531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2497,"log_dups_size":0,"ondisk_log_size":2497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:07:28.785202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"27'2544","reported_seq":2566,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.971798+0000","last_change":"2024-04-24T23:35:43.525630+0000","last_active":"2024-04-25T00:00:47.971798+0000","last_peered":"2024-04-25T00:00:47.971798+0000","last_clean":"2024-04-25T00:00:47.971798+0000","last_became_active":"2024-04-24T23:35:43.525191+0000","last_became_peered":"2024-04-24T23:35:43.525191+0000","last_unstale":"2024-04-25T00:00:47.971798+0000","last_undegraded":"2024-04-25T00:00:47.971798+0000","last_fullsized":"2024-04-25T00:00:47.971798+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2544,"log_dups_size":0,"ondisk_log_size":2544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:07:28.785202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"27'2608","reported_seq":2644,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.023985+0000","last_change":"2024-04-24T23:34:54.366079+0000","last_active":"2024-04-25T00:00:50.023985+0000","last_peered":"2024-04-25T00:00:50.023985+0000","last_clean":"2024-04-25T00:00:50.023985+0000","last_became_active":"2024-04-24T23:34:54.365814+0000","last_became_peered":"2024-04-24T23:34:54.365814+0000","last_unstale":"2024-04-25T00:00:50.023985+0000","last_undegraded":"2024-04-25T00:00:50.023985+0000","last_fullsized":"2024-04-25T00:00:50.023985+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2608,"log_dups_size":0,"ondisk_log_size":2608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:18.207267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"27'2449","reported_seq":2485,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:44.815128+0000","last_change":"2024-04-24T23:34:54.368976+0000","last_active":"2024-04-25T00:00:44.815128+0000","last_peered":"2024-04-25T00:00:44.815128+0000","last_clean":"2024-04-25T00:00:44.815128+0000","last_became_active":"2024-04-24T23:34:54.368663+0000","last_became_peered":"2024-04-24T23:34:54.368663+0000","last_unstale":"2024-04-25T00:00:44.815128+0000","last_undegraded":"2024-04-25T00:00:44.815128+0000","last_fullsized":"2024-04-25T00:00:44.815128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2449,"log_dups_size":0,"ondisk_log_size":2449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:56:17.763159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"27'2453","reported_seq":2488,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:49.117795+0000","last_change":"2024-04-24T23:34:54.373810+0000","last_active":"2024-04-25T00:00:49.117795+0000","last_peered":"2024-04-25T00:00:49.117795+0000","last_clean":"2024-04-25T00:00:49.117795+0000","last_became_active":"2024-04-24T23:34:54.373531+0000","last_became_peered":"2024-04-24T23:34:54.373531+0000","last_unstale":"2024-04-25T00:00:49.117795+0000","last_undegraded":"2024-04-25T00:00:49.117795+0000","last_fullsized":"2024-04-25T00:00:49.117795+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2453,"log_dups_size":0,"ondisk_log_size":2453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:50:17.621315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":111785901,"num_objects":673,"num_object_clones":0,"num_object_copies":2692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":673,"num_whiteouts":0,"num_read":12310,"num_read_kb":3126740,"num_write":210948,"num_write_kb":30633802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24567808,"data_stored":221478912,"data_compressed":10295227,"data_compressed_allocated":16637952,"data_compressed_original":213549056,"omap_allocated":0,"internal_metadata":0},"log_size":230878,"ondisk_log_size":230878,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":114,"num_object_clones":0,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1006,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161566,"ondisk_log_size":161566,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":233539226,"num_objects":127,"num_object_clones":0,"num_object_copies":254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":127,"num_whiteouts":0,"num_read":3152,"num_read_kb":19253,"num_write":210043,"num_write_kb":1679720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":63,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":287272960,"data_stored":469797651,"data_compressed":55273429,"data_compressed_allocated":80859136,"data_compressed_original":263442432,"omap_allocated":0,"internal_metadata":0},"log_size":195599,"ondisk_log_size":195599,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65248,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542458,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":963152,"kb_used_data":29704,"kb_used_omap":1,"kb_used_meta":933438,"kb_avail":93408688,"statfs":{"total":96636764160,"available":95650496512,"internally_reserved":0,"allocated":30416896,"data_stored":56977813,"data_compressed":5989303,"data_compressed_allocated":9019392,"data_compressed_original":35758080,"omap_allocated":1590,"internal_metadata":955840970},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,9,8,11,10,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542456,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1121104,"kb_used_data":43332,"kb_used_omap":1,"kb_used_meta":1077758,"kb_avail":93250736,"statfs":{"total":96636764160,"available":95488753664,"internally_reserved":0,"allocated":44371968,"data_stored":102701999,"data_compressed":9657573,"data_compressed_allocated":14041088,"data_compressed_original":72548352,"omap_allocated":1594,"internal_metadata":1103624646},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542457,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":978456,"kb_used_data":27024,"kb_used_omap":1,"kb_used_meta":951422,"kb_avail":93393384,"statfs":{"total":96636764160,"available":95634825216,"internally_reserved":0,"allocated":27672576,"data_stored":58276474,"data_compressed":5800819,"data_compressed_allocated":8785920,"data_compressed_original":39563264,"omap_allocated":1595,"internal_metadata":974256581},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542456,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":919584,"kb_used_data":44376,"kb_used_omap":1,"kb_used_meta":875198,"kb_avail":93452256,"statfs":{"total":96636764160,"available":95695110144,"internally_reserved":0,"allocated":45441024,"data_stored":102992542,"data_compressed":9448676,"data_compressed_allocated":13889536,"data_compressed_original":71622656,"omap_allocated":1595,"internal_metadata":896203205},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,12,12,26,43,46,253],"upper_bound":1024},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542458,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":663352,"kb_used_data":36592,"kb_used_omap":1,"kb_used_meta":626750,"kb_avail":93708488,"statfs":{"total":96636764160,"available":95957491712,"internally_reserved":0,"allocated":37470208,"data_stored":74025504,"data_compressed":7080739,"data_compressed_allocated":10862592,"data_compressed_original":47587328,"omap_allocated":1595,"internal_metadata":641792453},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542457,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1233860,"kb_used_data":43564,"kb_used_omap":1,"kb_used_meta":1190270,"kb_avail":93137980,"statfs":{"total":96636764160,"available":95373291520,"internally_reserved":0,"allocated":44609536,"data_stored":108677824,"data_compressed":9380730,"data_compressed_allocated":13918208,"data_compressed_original":78163968,"omap_allocated":1595,"internal_metadata":1218836933},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542458,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":823368,"kb_used_data":34304,"kb_used_omap":1,"kb_used_meta":789054,"kb_avail":93548472,"statfs":{"total":96636764160,"available":95793635328,"internally_reserved":0,"allocated":35127296,"data_stored":63717002,"data_compressed":7508106,"data_compressed_allocated":10883072,"data_compressed_original":39657472,"omap_allocated":1595,"internal_metadata":807991749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4],"upper_bound":16},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575162,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1146004,"kb_used_data":45964,"kb_used_omap":1,"kb_used_meta":1100030,"kb_avail":93225836,"statfs":{"total":96636764160,"available":95463256064,"internally_reserved":0,"allocated":47067136,"data_stored":117951222,"data_compressed":10359157,"data_compressed_allocated":15155200,"data_compressed_original":86216704,"omap_allocated":1593,"internal_metadata":1126431175},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32624,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32624,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":34275328,"data_stored":59417162,"data_compressed":7311941,"data_compressed_allocated":10522624,"data_compressed_original":35676160,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40267776,"data_stored":62914720,"data_compressed":7244862,"data_compressed_allocated":10850304,"data_compressed_original":33505280,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":34680832,"data_stored":54525952,"data_compressed":6190216,"data_compressed_allocated":9224192,"data_compressed_original":29069312,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":41914368,"data_stored":69131938,"data_compressed":8117877,"data_compressed_allocated":11812864,"data_compressed_original":39038976,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":41668608,"data_stored":67108990,"data_compressed":7785480,"data_compressed_allocated":11411456,"data_compressed_original":36864000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":25022464,"data_stored":41943130,"data_compressed":5058379,"data_compressed_allocated":7299072,"data_compressed_original":24223744,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40677376,"data_stored":67789199,"data_compressed":8032433,"data_compressed_allocated":11632640,"data_compressed_original":38752256,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28766208,"data_stored":46966560,"data_compressed":5532241,"data_compressed_allocated":8105984,"data_compressed_original":26312704,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":385024,"data_stored":2519040,"data_compressed":113842,"data_compressed_allocated":229376,"data_compressed_original":2363392,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4562944,"data_stored":46469120,"data_compressed":2170165,"data_compressed_allocated":3207168,"data_compressed_original":45113344,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2228224,"data_stored":16973824,"data_compressed":779748,"data_compressed_allocated":1454080,"data_compressed_original":16199680,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5431296,"data_stored":53047296,"data_compressed":2469993,"data_compressed_allocated":3739648,"data_compressed_original":51355648,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4603904,"data_stored":42082304,"data_compressed":1962538,"data_compressed_allocated":3190784,"data_compressed_original":40669184,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1859584,"data_stored":12443648,"data_compressed":567456,"data_compressed_allocated":1142784,"data_compressed_original":11726848,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4456448,"data_stored":40402944,"data_compressed":1886828,"data_compressed_allocated":2969600,"data_compressed_original":38916096,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1040384,"data_stored":7540736,"data_compressed":344657,"data_compressed_allocated":704512,"data_compressed_original":7204864,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:00:51.700 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:00:51.834 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.834+0000 7fe0efa64640 1 Processor -- start 2024-04-25T00:00:51.834 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.834+0000 7fe0efa64640 1 -- start start 2024-04-25T00:00:51.836 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.835+0000 7fe0efa64640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe0e80be360 0x7fe0e8155350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:51.837 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.835+0000 7fe0efa64640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0e80c8bb0 con 0x7fe0e8155890 2024-04-25T00:00:51.837 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.835+0000 7fe0efa64640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0e8159480 con 0x7fe0e8152b40 2024-04-25T00:00:51.837 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.835+0000 7fe0efa64640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fe0e80c8e90 con 0x7fe0e80be360 2024-04-25T00:00:51.837 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.835+0000 7fe0ee001640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fe0e8155890 0x7fe0e815fcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:55248/0 (socket says 172.21.15.157:55248) 2024-04-25T00:00:51.837 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.835+0000 7fe0ee001640 1 -- 172.21.15.157:0/4219847292 learned_addr learned my addr 172.21.15.157:0/4219847292 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-25T00:00:51.837 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.835+0000 7fe0ecfff640 1 --2- 172.21.15.157:0/4219847292 >> v2:172.21.15.169:3300/0 conn(0x7fe0e80be360 0x7fe0e8155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:51.837 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.835+0000 7fe0dffff640 1 -- 172.21.15.157:0/4219847292 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 825724965 0 0) 0x7fe0e80c8bb0 con 0x7fe0e8155890 2024-04-25T00:00:51.837 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.835+0000 7fe0dffff640 1 -- 172.21.15.157:0/4219847292 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe0d4003520 con 0x7fe0e8155890 2024-04-25T00:00:51.837 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.835+0000 7fe0dffff640 1 -- 172.21.15.157:0/4219847292 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2250897146 0 0) 0x7fe0e8159480 con 0x7fe0e8152b40 2024-04-25T00:00:51.837 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.835+0000 7fe0dffff640 1 -- 172.21.15.157:0/4219847292 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe0e80c8bb0 con 0x7fe0e8152b40 2024-04-25T00:00:51.837 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.835+0000 7fe0ecfff640 1 -- 172.21.15.157:0/4219847292 >> v1:172.21.15.169:6789/0 conn(0x7fe0e8152b40 legacy=0x7fe0e80c5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:51.837 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.835+0000 7fe0ecfff640 1 -- 172.21.15.157:0/4219847292 >> v1:172.21.15.157:6789/0 conn(0x7fe0e8155890 legacy=0x7fe0e815fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:51.838 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.835+0000 7fe0ecfff640 1 -- 172.21.15.157:0/4219847292 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0e8159760 con 0x7fe0e80be360 2024-04-25T00:00:51.838 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.836+0000 7fe0ecfff640 1 --2- 172.21.15.157:0/4219847292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0e80be360 0x7fe0e8155350 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe0d80097a0 tx=0x7fe0d802a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=429ee7ab127cc35d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:51.838 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.836+0000 7fe0dffff640 1 -- 172.21.15.157:0/4219847292 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d8037070 con 0x7fe0e80be360 2024-04-25T00:00:51.838 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.836+0000 7fe0dffff640 1 -- 172.21.15.157:0/4219847292 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0d8032440 con 0x7fe0e80be360 2024-04-25T00:00:51.838 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.836+0000 7fe0dffff640 1 -- 172.21.15.157:0/4219847292 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d802ae60 con 0x7fe0e80be360 2024-04-25T00:00:51.838 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.837+0000 7fe0efa64640 1 -- 172.21.15.157:0/4219847292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0e80be360 msgr2=0x7fe0e8155350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:51.838 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.837+0000 7fe0efa64640 1 --2- 172.21.15.157:0/4219847292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0e80be360 0x7fe0e8155350 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe0d80097a0 tx=0x7fe0d802a110 comp rx=0 tx=0).stop 2024-04-25T00:00:51.838 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.837+0000 7fe0efa64640 1 -- 172.21.15.157:0/4219847292 shutdown_connections 2024-04-25T00:00:51.838 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.837+0000 7fe0efa64640 1 --2- 172.21.15.157:0/4219847292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0e80be360 0x7fe0e8155350 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:51.838 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.837+0000 7fe0efa64640 1 -- 172.21.15.157:0/4219847292 >> 172.21.15.157:0/4219847292 conn(0x7fe0e80672c0 msgr2=0x7fe0e80676d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:51.838 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.837+0000 7fe0efa64640 1 -- 172.21.15.157:0/4219847292 shutdown_connections 2024-04-25T00:00:51.838 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.837+0000 7fe0efa64640 1 -- 172.21.15.157:0/4219847292 wait complete. 2024-04-25T00:00:51.841 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.838+0000 7fe0efa64640 1 Processor -- start 2024-04-25T00:00:51.841 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.838+0000 7fe0efa64640 1 -- start start 2024-04-25T00:00:51.841 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.839+0000 7fe0efa64640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0e80be360 0x7fe0e8157a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:51.842 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.839+0000 7fe0efa64640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0e8152b40 0x7fe0e814b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:51.842 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.839+0000 7fe0efa64640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0e8155890 0x7fe0e8149db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:51.842 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.839+0000 7fe0efa64640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e80caa00 con 0x7fe0e8152b40 2024-04-25T00:00:51.842 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.839+0000 7fe0efa64640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e80cb210 con 0x7fe0e8155890 2024-04-25T00:00:51.842 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.839+0000 7fe0efa64640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe0e8155d00 con 0x7fe0e80be360 2024-04-25T00:00:51.842 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.839+0000 7fe0ecfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0e8152b40 0x7fe0e814b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:51.842 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.839+0000 7fe0ecfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0e8152b40 0x7fe0e814b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51990/0 (socket says 172.21.15.157:51990) 2024-04-25T00:00:51.842 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.839+0000 7fe0ecfff640 1 -- 172.21.15.157:0/2998448113 learned_addr learned my addr 172.21.15.157:0/2998448113 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:00:51.842 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.839+0000 7fe0ed800640 1 --2- 172.21.15.157:0/2998448113 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0e80be360 0x7fe0e8157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:51.842 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.839+0000 7fe0ee001640 1 --2- 172.21.15.157:0/2998448113 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0e8155890 0x7fe0e8149db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:51.842 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.839+0000 7fe0ecfff640 1 -- 172.21.15.157:0/2998448113 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0e80be360 msgr2=0x7fe0e8157a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:51.842 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.839+0000 7fe0ecfff640 1 --2- 172.21.15.157:0/2998448113 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0e80be360 0x7fe0e8157a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:51.843 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.839+0000 7fe0ecfff640 1 -- 172.21.15.157:0/2998448113 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0e8155890 msgr2=0x7fe0e8149db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:51.843 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.839+0000 7fe0ecfff640 1 --2- 172.21.15.157:0/2998448113 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0e8155890 0x7fe0e8149db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:51.843 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.839+0000 7fe0ecfff640 1 -- 172.21.15.157:0/2998448113 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0e80ca3e0 con 0x7fe0e8152b40 2024-04-25T00:00:51.843 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.839+0000 7fe0ecfff640 1 --2- 172.21.15.157:0/2998448113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0e8152b40 0x7fe0e814b320 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe0d80098c0 tx=0x7fe0d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:51.843 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.839+0000 7fe0ddffb640 1 -- 172.21.15.157:0/2998448113 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d8041030 con 0x7fe0e8152b40 2024-04-25T00:00:51.843 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.839+0000 7fe0ddffb640 1 -- 172.21.15.157:0/2998448113 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0d803d4e0 con 0x7fe0e8152b40 2024-04-25T00:00:51.843 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.840+0000 7fe0efa64640 1 -- 172.21.15.157:0/2998448113 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0e8157280 con 0x7fe0e8152b40 2024-04-25T00:00:51.843 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.840+0000 7fe0ddffb640 1 -- 172.21.15.157:0/2998448113 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d8037040 con 0x7fe0e8152b40 2024-04-25T00:00:51.843 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.840+0000 7fe0efa64640 1 -- 172.21.15.157:0/2998448113 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0e81495a0 con 0x7fe0e8152b40 2024-04-25T00:00:51.843 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.840+0000 7fe0efa64640 1 -- 172.21.15.157:0/2998448113 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0e80681c0 con 0x7fe0e8152b40 2024-04-25T00:00:51.843 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.841+0000 7fe0ddffb640 1 -- 172.21.15.157:0/2998448113 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe0d8044070 con 0x7fe0e8152b40 2024-04-25T00:00:51.843 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.841+0000 7fe0ddffb640 1 --2- 172.21.15.157:0/2998448113 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fe0cc06be30 0x7fe0cc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:51.843 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.841+0000 7fe0ddffb640 1 -- 172.21.15.157:0/2998448113 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fe0d80ab7b0 con 0x7fe0e8152b40 2024-04-25T00:00:51.844 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.842+0000 7fe0ed800640 1 --2- 172.21.15.157:0/2998448113 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fe0cc06be30 0x7fe0cc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:51.844 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.842+0000 7fe0ed800640 1 --2- 172.21.15.157:0/2998448113 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fe0cc06be30 0x7fe0cc06e2f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4007fd0 tx=0x7fe0e4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:51.845 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:51.846+0000 7fe0ddffb640 1 -- 172.21.15.157:0/2998448113 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe0d80799a0 con 0x7fe0e8152b40 2024-04-25T00:00:52.022 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.022+0000 7fe0efa64640 1 -- 172.21.15.157:0/2998448113 --> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe0e80bdcf0 con 0x7fe0cc06be30 2024-04-25T00:00:52.036 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.037+0000 7fe0ddffb640 1 -- 172.21.15.157:0/2998448113 <== mgr.4105 v2:172.21.15.169:6832/662378289 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+336828 (secure 0 0 0) 0x7fe0e80bdcf0 con 0x7fe0cc06be30 2024-04-25T00:00:52.040 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-25T00:00:52.042 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2024-04-25T00:00:52.044 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.040+0000 7fe0efa64640 1 -- 172.21.15.157:0/2998448113 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fe0cc06be30 msgr2=0x7fe0cc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:52.044 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.040+0000 7fe0efa64640 1 --2- 172.21.15.157:0/2998448113 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fe0cc06be30 0x7fe0cc06e2f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4007fd0 tx=0x7fe0e4009040 comp rx=0 tx=0).stop 2024-04-25T00:00:52.044 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.040+0000 7fe0efa64640 1 -- 172.21.15.157:0/2998448113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0e8152b40 msgr2=0x7fe0e814b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:52.044 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.040+0000 7fe0efa64640 1 --2- 172.21.15.157:0/2998448113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0e8152b40 0x7fe0e814b320 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe0d80098c0 tx=0x7fe0d80046c0 comp rx=0 tx=0).stop 2024-04-25T00:00:52.044 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.040+0000 7fe0efa64640 1 -- 172.21.15.157:0/2998448113 shutdown_connections 2024-04-25T00:00:52.044 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.040+0000 7fe0efa64640 1 --2- 172.21.15.157:0/2998448113 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fe0cc06be30 0x7fe0cc06e2f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:52.044 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.040+0000 7fe0efa64640 1 --2- 172.21.15.157:0/2998448113 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0e8155890 0x7fe0e8149db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:52.044 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.040+0000 7fe0efa64640 1 --2- 172.21.15.157:0/2998448113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0e8152b40 0x7fe0e814b320 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:52.044 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.040+0000 7fe0efa64640 1 --2- 172.21.15.157:0/2998448113 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0e80be360 0x7fe0e8157a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:52.045 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.040+0000 7fe0efa64640 1 -- 172.21.15.157:0/2998448113 >> 172.21.15.157:0/2998448113 conn(0x7fe0e80672c0 msgr2=0x7fe0e81de9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:52.045 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.040+0000 7fe0efa64640 1 -- 172.21.15.157:0/2998448113 shutdown_connections 2024-04-25T00:00:52.045 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.040+0000 7fe0efa64640 1 -- 172.21.15.157:0/2998448113 wait complete. 2024-04-25T00:00:52.051 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":802,"stamp":"2024-04-25T00:00:50.172506+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":345915495,"num_objects":916,"num_object_clones":0,"num_object_copies":3178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":916,"num_whiteouts":0,"num_read":15578,"num_read_kb":3146093,"num_write":422156,"num_write_kb":32315938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":63,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":588160,"ondisk_log_size":588160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":7848880,"kb_used_data":304860,"kb_used_omap":12,"kb_used_meta":7543923,"kb_avail":747125840,"statfs":{"total":773094113280,"available":765056860160,"internally_reserved":0,"allocated":312176640,"data_stored":685320380,"data_compressed":65225103,"data_compressed_allocated":96555008,"data_compressed_original":471117824,"omap_allocated":12752,"internal_metadata":7724977712},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,7,15,22,34,54,56,253,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":55,"apply_latency_ms":55,"commit_latency_ns":55000000,"apply_latency_ns":55000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-118121358,"num_objects":-623,"num_object_clones":0,"num_object_copies":-1468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-623,"num_whiteouts":0,"num_read":128,"num_read_kb":25182,"num_write":3659,"num_write_kb":61598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-64,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008122"},"pg_stats":[{"pgid":"2.1f","version":"27'2678","reported_seq":2716,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.212495+0000","last_change":"2024-04-24T23:34:53.362409+0000","last_active":"2024-04-25T00:00:47.212495+0000","last_peered":"2024-04-25T00:00:47.212495+0000","last_clean":"2024-04-25T00:00:47.212495+0000","last_became_active":"2024-04-24T23:34:53.362104+0000","last_became_peered":"2024-04-24T23:34:53.362104+0000","last_unstale":"2024-04-25T00:00:47.212495+0000","last_undegraded":"2024-04-25T00:00:47.212495+0000","last_fullsized":"2024-04-25T00:00:47.212495+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":2678,"log_dups_size":0,"ondisk_log_size":2678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:53.159219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":2687,"num_write_kb":18218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"27'2525","reported_seq":2561,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:44.806511+0000","last_change":"2024-04-24T23:34:54.373560+0000","last_active":"2024-04-25T00:00:44.806511+0000","last_peered":"2024-04-25T00:00:44.806511+0000","last_clean":"2024-04-25T00:00:44.806511+0000","last_became_active":"2024-04-24T23:34:54.373289+0000","last_became_peered":"2024-04-24T23:34:54.373289+0000","last_unstale":"2024-04-25T00:00:44.806511+0000","last_undegraded":"2024-04-25T00:00:44.806511+0000","last_fullsized":"2024-04-25T00:00:44.806511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2525,"log_dups_size":0,"ondisk_log_size":2525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:31.447899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"27'9404","reported_seq":9750,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:45.918901+0000","last_change":"2024-04-24T23:34:58.730165+0000","last_active":"2024-04-25T00:00:45.918901+0000","last_peered":"2024-04-25T00:00:45.918901+0000","last_clean":"2024-04-25T00:00:45.918901+0000","last_became_active":"2024-04-24T23:34:58.729816+0000","last_became_peered":"2024-04-24T23:34:58.729816+0000","last_unstale":"2024-04-25T00:00:45.918901+0000","last_undegraded":"2024-04-25T00:00:45.918901+0000","last_fullsized":"2024-04-25T00:00:45.918901+0000","mapping_epoch":23,"log_start":"27'3100","ondisk_log_start":"27'3100","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":6304,"log_dups_size":0,"ondisk_log_size":6304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:02:02.335135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5938304,"num_objects":24,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":319,"num_read_kb":81026,"num_write":6703,"num_write_kb":1030241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"27'6587","reported_seq":6625,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:45.133750+0000","last_change":"2024-04-24T23:34:53.361650+0000","last_active":"2024-04-25T00:00:45.133750+0000","last_peered":"2024-04-25T00:00:45.133750+0000","last_clean":"2024-04-25T00:00:45.133750+0000","last_became_active":"2024-04-24T23:34:53.361366+0000","last_became_peered":"2024-04-24T23:34:53.361366+0000","last_unstale":"2024-04-25T00:00:45.133750+0000","last_undegraded":"2024-04-25T00:00:45.133750+0000","last_fullsized":"2024-04-25T00:00:45.133750+0000","mapping_epoch":18,"log_start":"27'300","ondisk_log_start":"27'300","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":6287,"log_dups_size":0,"ondisk_log_size":6287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:56:15.784223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2022402,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6588,"num_write_kb":51489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"27'2521","reported_seq":2557,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:49.128653+0000","last_change":"2024-04-24T23:34:54.383602+0000","last_active":"2024-04-25T00:00:49.128653+0000","last_peered":"2024-04-25T00:00:49.128653+0000","last_clean":"2024-04-25T00:00:49.128653+0000","last_became_active":"2024-04-24T23:34:54.383160+0000","last_became_peered":"2024-04-24T23:34:54.383160+0000","last_unstale":"2024-04-25T00:00:49.128653+0000","last_undegraded":"2024-04-25T00:00:49.128653+0000","last_fullsized":"2024-04-25T00:00:49.128653+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2521,"log_dups_size":0,"ondisk_log_size":2521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:15.477061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"27'9141","reported_seq":9338,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:46.124918+0000","last_change":"2024-04-24T23:34:58.404416+0000","last_active":"2024-04-25T00:00:46.124918+0000","last_peered":"2024-04-25T00:00:46.124918+0000","last_clean":"2024-04-25T00:00:46.124918+0000","last_became_active":"2024-04-24T23:34:58.403984+0000","last_became_peered":"2024-04-24T23:34:58.403984+0000","last_unstale":"2024-04-25T00:00:46.124918+0000","last_undegraded":"2024-04-25T00:00:46.124918+0000","last_fullsized":"2024-04-25T00:00:46.124918+0000","mapping_epoch":23,"log_start":"27'1960","ondisk_log_start":"27'1960","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7181,"log_dups_size":0,"ondisk_log_size":7181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:46.237082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3253957,"num_objects":19,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":169,"num_read_kb":42926,"num_write":6398,"num_write_kb":1082204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"27'10053","reported_seq":10208,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:49.138658+0000","last_change":"2024-04-24T23:34:53.366730+0000","last_active":"2024-04-25T00:00:49.138658+0000","last_peered":"2024-04-25T00:00:49.138658+0000","last_clean":"2024-04-25T00:00:49.138658+0000","last_became_active":"2024-04-24T23:34:53.366430+0000","last_became_peered":"2024-04-24T23:34:53.366430+0000","last_unstale":"2024-04-25T00:00:49.138658+0000","last_undegraded":"2024-04-25T00:00:49.138658+0000","last_fullsized":"2024-04-25T00:00:49.138658+0000","mapping_epoch":18,"log_start":"27'3200","ondisk_log_start":"27'3200","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":6853,"log_dups_size":0,"ondisk_log_size":6853,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:41.422591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5022938,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":120,"num_read_kb":893,"num_write":10045,"num_write_kb":77054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"27'2616","reported_seq":2650,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.280419+0000","last_change":"2024-04-24T23:34:54.373592+0000","last_active":"2024-04-25T00:00:47.280419+0000","last_peered":"2024-04-25T00:00:47.280419+0000","last_clean":"2024-04-25T00:00:47.280419+0000","last_became_active":"2024-04-24T23:34:54.372844+0000","last_became_peered":"2024-04-24T23:34:54.372844+0000","last_unstale":"2024-04-25T00:00:47.280419+0000","last_undegraded":"2024-04-25T00:00:47.280419+0000","last_fullsized":"2024-04-25T00:00:47.280419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2616,"log_dups_size":0,"ondisk_log_size":2616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:49.372040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"27'9348","reported_seq":9962,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:46.556129+0000","last_change":"2024-04-24T23:34:58.399224+0000","last_active":"2024-04-25T00:00:46.556129+0000","last_peered":"2024-04-25T00:00:46.556129+0000","last_clean":"2024-04-25T00:00:46.556129+0000","last_became_active":"2024-04-24T23:34:58.397817+0000","last_became_peered":"2024-04-24T23:34:58.397817+0000","last_unstale":"2024-04-25T00:00:46.556129+0000","last_undegraded":"2024-04-25T00:00:46.556129+0000","last_fullsized":"2024-04-25T00:00:46.556129+0000","mapping_epoch":23,"log_start":"27'2112","ondisk_log_start":"27'2112","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7236,"log_dups_size":0,"ondisk_log_size":7236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:53.085366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3289544,"num_objects":25,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":588,"num_read_kb":149352,"num_write":6655,"num_write_kb":891145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"27'5158","reported_seq":5193,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.195334+0000","last_change":"2024-04-24T23:34:53.362994+0000","last_active":"2024-04-25T00:00:47.195334+0000","last_peered":"2024-04-25T00:00:47.195334+0000","last_clean":"2024-04-25T00:00:47.195334+0000","last_became_active":"2024-04-24T23:34:53.362337+0000","last_became_peered":"2024-04-24T23:34:53.362337+0000","last_unstale":"2024-04-25T00:00:47.195334+0000","last_undegraded":"2024-04-25T00:00:47.195334+0000","last_fullsized":"2024-04-25T00:00:47.195334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":5158,"log_dups_size":0,"ondisk_log_size":5158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:12.258211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":5165,"num_write_kb":44324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"27'2532","reported_seq":2566,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.851337+0000","last_change":"2024-04-24T23:34:54.379218+0000","last_active":"2024-04-25T00:00:47.851337+0000","last_peered":"2024-04-25T00:00:47.851337+0000","last_clean":"2024-04-25T00:00:47.851337+0000","last_became_active":"2024-04-24T23:34:54.376019+0000","last_became_peered":"2024-04-24T23:34:54.376019+0000","last_unstale":"2024-04-25T00:00:47.851337+0000","last_undegraded":"2024-04-25T00:00:47.851337+0000","last_fullsized":"2024-04-25T00:00:47.851337+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2532,"log_dups_size":0,"ondisk_log_size":2532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:41:48.207542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"27'8705","reported_seq":9204,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:46.391347+0000","last_change":"2024-04-24T23:34:58.397152+0000","last_active":"2024-04-25T00:00:46.391347+0000","last_peered":"2024-04-25T00:00:46.391347+0000","last_clean":"2024-04-25T00:00:46.391347+0000","last_became_active":"2024-04-24T23:34:58.396658+0000","last_became_peered":"2024-04-24T23:34:58.396658+0000","last_unstale":"2024-04-25T00:00:46.391347+0000","last_undegraded":"2024-04-25T00:00:46.391347+0000","last_fullsized":"2024-04-25T00:00:46.391347+0000","mapping_epoch":23,"log_start":"27'1509","ondisk_log_start":"27'1509","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7196,"log_dups_size":0,"ondisk_log_size":7196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:41.360776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4269424,"num_objects":28,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":472,"num_read_kb":119888,"num_write":5922,"num_write_kb":848200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"27'2537","reported_seq":2572,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:44.457268+0000","last_change":"2024-04-24T23:34:54.377859+0000","last_active":"2024-04-25T00:00:44.457268+0000","last_peered":"2024-04-25T00:00:44.457268+0000","last_clean":"2024-04-25T00:00:44.457268+0000","last_became_active":"2024-04-24T23:34:54.377576+0000","last_became_peered":"2024-04-24T23:34:54.377576+0000","last_unstale":"2024-04-25T00:00:44.457268+0000","last_undegraded":"2024-04-25T00:00:44.457268+0000","last_fullsized":"2024-04-25T00:00:44.457268+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2537,"log_dups_size":0,"ondisk_log_size":2537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:47.446459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"27'6834","reported_seq":6870,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.226938+0000","last_change":"2024-04-24T23:34:53.360903+0000","last_active":"2024-04-25T00:00:47.226938+0000","last_peered":"2024-04-25T00:00:47.226938+0000","last_clean":"2024-04-25T00:00:47.226938+0000","last_became_active":"2024-04-24T23:34:53.360635+0000","last_became_peered":"2024-04-24T23:34:53.360635+0000","last_unstale":"2024-04-25T00:00:47.226938+0000","last_undegraded":"2024-04-25T00:00:47.226938+0000","last_fullsized":"2024-04-25T00:00:47.226938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":6834,"log_dups_size":0,"ondisk_log_size":6834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:42.205997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6840,"num_write_kb":53782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"27'10158","reported_seq":10440,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:46.376285+0000","last_change":"2024-04-24T23:34:58.407146+0000","last_active":"2024-04-25T00:00:46.376285+0000","last_peered":"2024-04-25T00:00:46.376285+0000","last_clean":"2024-04-25T00:00:46.376285+0000","last_became_active":"2024-04-24T23:34:58.406844+0000","last_became_peered":"2024-04-24T23:34:58.406844+0000","last_unstale":"2024-04-25T00:00:46.376285+0000","last_undegraded":"2024-04-25T00:00:46.376285+0000","last_fullsized":"2024-04-25T00:00:46.376285+0000","mapping_epoch":23,"log_start":"27'2941","ondisk_log_start":"27'2941","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7217,"log_dups_size":0,"ondisk_log_size":7217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:41:52.244767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4910152,"num_objects":31,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":253,"num_read_kb":64262,"num_write":7410,"num_write_kb":1029421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"27'2561","reported_seq":2596,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:44.968398+0000","last_change":"2024-04-24T23:34:54.366724+0000","last_active":"2024-04-25T00:00:44.968398+0000","last_peered":"2024-04-25T00:00:44.968398+0000","last_clean":"2024-04-25T00:00:44.968398+0000","last_became_active":"2024-04-24T23:34:54.366426+0000","last_became_peered":"2024-04-24T23:34:54.366426+0000","last_unstale":"2024-04-25T00:00:44.968398+0000","last_undegraded":"2024-04-25T00:00:44.968398+0000","last_fullsized":"2024-04-25T00:00:44.968398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2561,"log_dups_size":0,"ondisk_log_size":2561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:47.035356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"27'7909","reported_seq":7946,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:05.230406+0000","last_change":"2024-04-24T23:34:53.359140+0000","last_active":"2024-04-25T00:00:05.230406+0000","last_peered":"2024-04-25T00:00:05.230406+0000","last_clean":"2024-04-25T00:00:05.230406+0000","last_became_active":"2024-04-24T23:34:53.358841+0000","last_became_peered":"2024-04-24T23:34:53.358841+0000","last_unstale":"2024-04-25T00:00:05.230406+0000","last_undegraded":"2024-04-25T00:00:05.230406+0000","last_fullsized":"2024-04-25T00:00:05.230406+0000","mapping_epoch":18,"log_start":"27'500","ondisk_log_start":"27'500","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":7409,"log_dups_size":0,"ondisk_log_size":7409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:41.422591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7908,"num_write_kb":58475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"27'9659","reported_seq":10064,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:44.950505+0000","last_change":"2024-04-24T23:34:58.400128+0000","last_active":"2024-04-25T00:00:44.950505+0000","last_peered":"2024-04-25T00:00:44.950505+0000","last_clean":"2024-04-25T00:00:44.950505+0000","last_became_active":"2024-04-24T23:34:58.399375+0000","last_became_peered":"2024-04-24T23:34:58.399375+0000","last_unstale":"2024-04-25T00:00:44.950505+0000","last_undegraded":"2024-04-25T00:00:44.950505+0000","last_fullsized":"2024-04-25T00:00:44.950505+0000","mapping_epoch":23,"log_start":"27'2456","ondisk_log_start":"27'2456","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7203,"log_dups_size":0,"ondisk_log_size":7203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:53.409265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6924224,"num_objects":35,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":403,"num_read_kb":102362,"num_write":6939,"num_write_kb":969056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"27'2625","reported_seq":2660,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:44.403956+0000","last_change":"2024-04-24T23:34:54.365012+0000","last_active":"2024-04-25T00:00:44.403956+0000","last_peered":"2024-04-25T00:00:44.403956+0000","last_clean":"2024-04-25T00:00:44.403956+0000","last_became_active":"2024-04-24T23:34:54.364744+0000","last_became_peered":"2024-04-24T23:34:54.364744+0000","last_unstale":"2024-04-25T00:00:44.403956+0000","last_undegraded":"2024-04-25T00:00:44.403956+0000","last_fullsized":"2024-04-25T00:00:44.403956+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2625,"log_dups_size":0,"ondisk_log_size":2625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:35.774122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"27'4020","reported_seq":4057,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-24T23:58:44.283649+0000","last_change":"2024-04-24T23:34:53.360843+0000","last_active":"2024-04-24T23:58:44.283649+0000","last_peered":"2024-04-24T23:58:44.283649+0000","last_clean":"2024-04-24T23:58:44.283649+0000","last_became_active":"2024-04-24T23:34:53.360570+0000","last_became_peered":"2024-04-24T23:34:53.360570+0000","last_unstale":"2024-04-24T23:58:44.283649+0000","last_undegraded":"2024-04-24T23:58:44.283649+0000","last_fullsized":"2024-04-24T23:58:44.283649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":4020,"log_dups_size":0,"ondisk_log_size":4020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:21.089467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4020,"num_write_kb":27320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"27'9575","reported_seq":9994,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:45.249126+0000","last_change":"2024-04-24T23:34:58.731835+0000","last_active":"2024-04-25T00:00:45.249126+0000","last_peered":"2024-04-25T00:00:45.249126+0000","last_clean":"2024-04-25T00:00:45.249126+0000","last_became_active":"2024-04-24T23:34:58.731485+0000","last_became_peered":"2024-04-24T23:34:58.731485+0000","last_unstale":"2024-04-25T00:00:45.249126+0000","last_undegraded":"2024-04-25T00:00:45.249126+0000","last_fullsized":"2024-04-25T00:00:45.249126+0000","mapping_epoch":23,"log_start":"27'2216","ondisk_log_start":"27'2216","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7359,"log_dups_size":0,"ondisk_log_size":7359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:56.609480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5803439,"num_objects":17,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":392,"num_read_kb":99568,"num_write":6958,"num_write_kb":902680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"27'2530","reported_seq":2565,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:44.799276+0000","last_change":"2024-04-24T23:34:54.362447+0000","last_active":"2024-04-25T00:00:44.799276+0000","last_peered":"2024-04-25T00:00:44.799276+0000","last_clean":"2024-04-25T00:00:44.799276+0000","last_became_active":"2024-04-24T23:34:54.362141+0000","last_became_peered":"2024-04-24T23:34:54.362141+0000","last_unstale":"2024-04-25T00:00:44.799276+0000","last_undegraded":"2024-04-25T00:00:44.799276+0000","last_fullsized":"2024-04-25T00:00:44.799276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2530,"log_dups_size":0,"ondisk_log_size":2530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:51.227333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"27'2570","reported_seq":2606,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.182556+0000","last_change":"2024-04-24T23:34:53.366452+0000","last_active":"2024-04-25T00:00:47.182556+0000","last_peered":"2024-04-25T00:00:47.182556+0000","last_clean":"2024-04-25T00:00:47.182556+0000","last_became_active":"2024-04-24T23:34:53.366110+0000","last_became_peered":"2024-04-24T23:34:53.366110+0000","last_unstale":"2024-04-25T00:00:47.182556+0000","last_undegraded":"2024-04-25T00:00:47.182556+0000","last_fullsized":"2024-04-25T00:00:47.182556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":2570,"log_dups_size":0,"ondisk_log_size":2570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:14.716741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2576,"num_write_kb":22186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"27'9154","reported_seq":9327,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:46.615785+0000","last_change":"2024-04-24T23:34:58.402849+0000","last_active":"2024-04-25T00:00:46.615785+0000","last_peered":"2024-04-25T00:00:46.615785+0000","last_clean":"2024-04-25T00:00:46.615785+0000","last_became_active":"2024-04-24T23:34:58.402365+0000","last_became_peered":"2024-04-24T23:34:58.402365+0000","last_unstale":"2024-04-25T00:00:46.615785+0000","last_undegraded":"2024-04-25T00:00:46.615785+0000","last_fullsized":"2024-04-25T00:00:46.615785+0000","mapping_epoch":23,"log_start":"27'1949","ondisk_log_start":"27'1949","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7205,"log_dups_size":0,"ondisk_log_size":7205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:07:32.822427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6020711,"num_objects":43,"num_object_clones":0,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":151,"num_read_kb":38354,"num_write":6506,"num_write_kb":977639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"27'2514","reported_seq":2549,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:49.836629+0000","last_change":"2024-04-24T23:34:54.370300+0000","last_active":"2024-04-25T00:00:49.836629+0000","last_peered":"2024-04-25T00:00:49.836629+0000","last_clean":"2024-04-25T00:00:49.836629+0000","last_became_active":"2024-04-24T23:34:54.369835+0000","last_became_peered":"2024-04-24T23:34:54.369835+0000","last_unstale":"2024-04-25T00:00:49.836629+0000","last_undegraded":"2024-04-25T00:00:49.836629+0000","last_fullsized":"2024-04-25T00:00:49.836629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2514,"log_dups_size":0,"ondisk_log_size":2514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:42.199856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"27'4478","reported_seq":4516,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:01.006427+0000","last_change":"2024-04-24T23:34:53.366110+0000","last_active":"2024-04-25T00:00:01.006427+0000","last_peered":"2024-04-25T00:00:01.006427+0000","last_clean":"2024-04-25T00:00:01.006427+0000","last_became_active":"2024-04-24T23:34:53.365783+0000","last_became_peered":"2024-04-24T23:34:53.365783+0000","last_unstale":"2024-04-25T00:00:01.006427+0000","last_undegraded":"2024-04-25T00:00:01.006427+0000","last_fullsized":"2024-04-25T00:00:01.006427+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":4478,"log_dups_size":0,"ondisk_log_size":4478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:41.422591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":4488,"num_write_kb":39872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"27'9186","reported_seq":9406,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:46.594939+0000","last_change":"2024-04-24T23:34:58.728191+0000","last_active":"2024-04-25T00:00:46.594939+0000","last_peered":"2024-04-25T00:00:46.594939+0000","last_clean":"2024-04-25T00:00:46.594939+0000","last_became_active":"2024-04-24T23:34:58.727653+0000","last_became_peered":"2024-04-24T23:34:58.727653+0000","last_unstale":"2024-04-25T00:00:46.594939+0000","last_undegraded":"2024-04-25T00:00:46.594939+0000","last_fullsized":"2024-04-25T00:00:46.594939+0000","mapping_epoch":23,"log_start":"27'2006","ondisk_log_start":"27'2006","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7180,"log_dups_size":0,"ondisk_log_size":7180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:30.299011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5124378,"num_objects":31,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":190,"num_read_kb":48260,"num_write":6538,"num_write_kb":840404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"27'2471","reported_seq":2506,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:44.965107+0000","last_change":"2024-04-24T23:34:54.368278+0000","last_active":"2024-04-25T00:00:44.965107+0000","last_peered":"2024-04-25T00:00:44.965107+0000","last_clean":"2024-04-25T00:00:44.965107+0000","last_became_active":"2024-04-24T23:34:54.367991+0000","last_became_peered":"2024-04-24T23:34:54.367991+0000","last_unstale":"2024-04-25T00:00:44.965107+0000","last_undegraded":"2024-04-25T00:00:44.965107+0000","last_fullsized":"2024-04-25T00:00:44.965107+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2471,"log_dups_size":0,"ondisk_log_size":2471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:49:31.018719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"27'7464","reported_seq":8126,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:43.649617+0000","last_change":"2024-04-24T23:34:53.364189+0000","last_active":"2024-04-25T00:00:43.649617+0000","last_peered":"2024-04-25T00:00:43.649617+0000","last_clean":"2024-04-25T00:00:43.649617+0000","last_became_active":"2024-04-24T23:34:53.363842+0000","last_became_peered":"2024-04-24T23:34:53.363842+0000","last_unstale":"2024-04-25T00:00:43.649617+0000","last_undegraded":"2024-04-25T00:00:43.649617+0000","last_fullsized":"2024-04-25T00:00:43.649617+0000","mapping_epoch":18,"log_start":"27'100","ondisk_log_start":"27'100","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":7364,"log_dups_size":0,"ondisk_log_size":7364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:12.258211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":626,"num_read_kb":4509,"num_write":10302,"num_write_kb":61529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"27'8867","reported_seq":9175,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:45.755441+0000","last_change":"2024-04-24T23:34:58.729450+0000","last_active":"2024-04-25T00:00:45.755441+0000","last_peered":"2024-04-25T00:00:45.755441+0000","last_clean":"2024-04-25T00:00:45.755441+0000","last_became_active":"2024-04-24T23:34:58.729082+0000","last_became_peered":"2024-04-24T23:34:58.729082+0000","last_unstale":"2024-04-25T00:00:45.755441+0000","last_undegraded":"2024-04-25T00:00:45.755441+0000","last_fullsized":"2024-04-25T00:00:45.755441+0000","mapping_epoch":23,"log_start":"27'2609","ondisk_log_start":"27'2609","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":6258,"log_dups_size":0,"ondisk_log_size":6258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:45.372053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3433546,"num_objects":14,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":278,"num_read_kb":70612,"num_write":6185,"num_write_kb":897704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"27'2575","reported_seq":2611,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:44.888358+0000","last_change":"2024-04-24T23:34:54.357246+0000","last_active":"2024-04-25T00:00:44.888358+0000","last_peered":"2024-04-25T00:00:44.888358+0000","last_clean":"2024-04-25T00:00:44.888358+0000","last_became_active":"2024-04-24T23:34:54.356560+0000","last_became_peered":"2024-04-24T23:34:54.356560+0000","last_unstale":"2024-04-25T00:00:44.888358+0000","last_undegraded":"2024-04-25T00:00:44.888358+0000","last_fullsized":"2024-04-25T00:00:44.888358+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2575,"log_dups_size":0,"ondisk_log_size":2575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:42.199856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"27'6261","reported_seq":6298,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.213097+0000","last_change":"2024-04-24T23:34:53.366296+0000","last_active":"2024-04-25T00:00:47.213097+0000","last_peered":"2024-04-25T00:00:47.213097+0000","last_clean":"2024-04-25T00:00:47.213097+0000","last_became_active":"2024-04-24T23:34:53.365636+0000","last_became_peered":"2024-04-24T23:34:53.365636+0000","last_unstale":"2024-04-25T00:00:47.213097+0000","last_undegraded":"2024-04-25T00:00:47.213097+0000","last_fullsized":"2024-04-25T00:00:47.213097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":6261,"log_dups_size":0,"ondisk_log_size":6261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:50.222015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":6274,"num_write_kb":49208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"27'9152","reported_seq":9686,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:48.941905+0000","last_change":"2024-04-24T23:34:58.728271+0000","last_active":"2024-04-25T00:00:48.941905+0000","last_peered":"2024-04-25T00:00:48.941905+0000","last_clean":"2024-04-25T00:00:48.941905+0000","last_became_active":"2024-04-24T23:34:58.727885+0000","last_became_peered":"2024-04-24T23:34:58.727885+0000","last_unstale":"2024-04-25T00:00:48.941905+0000","last_undegraded":"2024-04-25T00:00:48.941905+0000","last_fullsized":"2024-04-25T00:00:48.941905+0000","mapping_epoch":23,"log_start":"27'1968","ondisk_log_start":"27'1968","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7184,"log_dups_size":0,"ondisk_log_size":7184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:49.461271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":504,"num_read_kb":128016,"num_write":6457,"num_write_kb":842362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"27'2479","reported_seq":2515,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.782154+0000","last_change":"2024-04-24T23:34:54.372245+0000","last_active":"2024-04-25T00:00:47.782154+0000","last_peered":"2024-04-25T00:00:47.782154+0000","last_clean":"2024-04-25T00:00:47.782154+0000","last_became_active":"2024-04-24T23:34:54.371947+0000","last_became_peered":"2024-04-24T23:34:54.371947+0000","last_unstale":"2024-04-25T00:00:47.782154+0000","last_undegraded":"2024-04-25T00:00:47.782154+0000","last_fullsized":"2024-04-25T00:00:47.782154+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2479,"log_dups_size":0,"ondisk_log_size":2479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:47:52.460508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"27'7217","reported_seq":7255,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:22.697628+0000","last_change":"2024-04-24T23:34:53.360352+0000","last_active":"2024-04-25T00:00:22.697628+0000","last_peered":"2024-04-25T00:00:22.697628+0000","last_clean":"2024-04-25T00:00:22.697628+0000","last_became_active":"2024-04-24T23:34:53.360082+0000","last_became_peered":"2024-04-24T23:34:53.360082+0000","last_unstale":"2024-04-25T00:00:22.697628+0000","last_undegraded":"2024-04-25T00:00:22.697628+0000","last_fullsized":"2024-04-25T00:00:22.697628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":7217,"log_dups_size":0,"ondisk_log_size":7217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:50.222015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":7215,"num_write_kb":70161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"27'9096","reported_seq":9280,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:45.286121+0000","last_change":"2024-04-24T23:34:58.404176+0000","last_active":"2024-04-25T00:00:45.286121+0000","last_peered":"2024-04-25T00:00:45.286121+0000","last_clean":"2024-04-25T00:00:45.286121+0000","last_became_active":"2024-04-24T23:34:58.403735+0000","last_became_peered":"2024-04-24T23:34:58.403735+0000","last_unstale":"2024-04-25T00:00:45.286121+0000","last_undegraded":"2024-04-25T00:00:45.286121+0000","last_fullsized":"2024-04-25T00:00:45.286121+0000","mapping_epoch":23,"log_start":"27'1709","ondisk_log_start":"27'1709","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7387,"log_dups_size":0,"ondisk_log_size":7387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:49.461271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3287360,"num_objects":22,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":155,"num_read_kb":39370,"num_write":6428,"num_write_kb":922251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"27'2510","reported_seq":2546,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:44.883767+0000","last_change":"2024-04-24T23:34:54.367638+0000","last_active":"2024-04-25T00:00:44.883767+0000","last_peered":"2024-04-25T00:00:44.883767+0000","last_clean":"2024-04-25T00:00:44.883767+0000","last_became_active":"2024-04-24T23:34:54.367355+0000","last_became_peered":"2024-04-24T23:34:54.367355+0000","last_unstale":"2024-04-25T00:00:44.883767+0000","last_undegraded":"2024-04-25T00:00:44.883767+0000","last_fullsized":"2024-04-25T00:00:44.883767+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2510,"log_dups_size":0,"ondisk_log_size":2510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:15.722059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"27'8389","reported_seq":8427,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.171711+0000","last_change":"2024-04-24T23:34:53.356111+0000","last_active":"2024-04-25T00:00:47.171711+0000","last_peered":"2024-04-25T00:00:47.171711+0000","last_clean":"2024-04-25T00:00:47.171711+0000","last_became_active":"2024-04-24T23:34:53.355710+0000","last_became_peered":"2024-04-24T23:34:53.355710+0000","last_unstale":"2024-04-25T00:00:47.171711+0000","last_undegraded":"2024-04-25T00:00:47.171711+0000","last_fullsized":"2024-04-25T00:00:47.171711+0000","mapping_epoch":18,"log_start":"27'600","ondisk_log_start":"27'600","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":7789,"log_dups_size":0,"ondisk_log_size":7789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:21.089467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":8401,"num_write_kb":63182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"27'9604","reported_seq":9968,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:45.056118+0000","last_change":"2024-04-24T23:34:58.403016+0000","last_active":"2024-04-25T00:00:45.056118+0000","last_peered":"2024-04-25T00:00:45.056118+0000","last_clean":"2024-04-25T00:00:45.056118+0000","last_became_active":"2024-04-24T23:34:58.402598+0000","last_became_peered":"2024-04-24T23:34:58.402598+0000","last_unstale":"2024-04-25T00:00:45.056118+0000","last_undegraded":"2024-04-25T00:00:45.056118+0000","last_fullsized":"2024-04-25T00:00:45.056118+0000","mapping_epoch":23,"log_start":"27'2193","ondisk_log_start":"27'2193","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7411,"log_dups_size":0,"ondisk_log_size":7411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:47:56.497734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2988236,"num_objects":18,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":334,"num_read_kb":84836,"num_write":6810,"num_write_kb":885456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"27'2530","reported_seq":2566,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:44.936154+0000","last_change":"2024-04-24T23:34:54.356601+0000","last_active":"2024-04-25T00:00:44.936154+0000","last_peered":"2024-04-25T00:00:44.936154+0000","last_clean":"2024-04-25T00:00:44.936154+0000","last_became_active":"2024-04-24T23:34:54.356326+0000","last_became_peered":"2024-04-24T23:34:54.356326+0000","last_unstale":"2024-04-25T00:00:44.936154+0000","last_undegraded":"2024-04-25T00:00:44.936154+0000","last_fullsized":"2024-04-25T00:00:44.936154+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2530,"log_dups_size":0,"ondisk_log_size":2530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:47.446459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"27'7177","reported_seq":8144,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.164180+0000","last_change":"2024-04-24T23:34:53.359238+0000","last_active":"2024-04-25T00:00:47.164180+0000","last_peered":"2024-04-25T00:00:47.164180+0000","last_clean":"2024-04-25T00:00:47.164180+0000","last_became_active":"2024-04-24T23:34:53.358937+0000","last_became_peered":"2024-04-24T23:34:53.358937+0000","last_unstale":"2024-04-25T00:00:47.164180+0000","last_undegraded":"2024-04-25T00:00:47.164180+0000","last_fullsized":"2024-04-25T00:00:47.164180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":7177,"log_dups_size":0,"ondisk_log_size":7177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:50.222015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":929,"num_read_kb":4700,"num_write":7175,"num_write_kb":61982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"27'8943","reported_seq":9533,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:45.360737+0000","last_change":"2024-04-24T23:34:58.403258+0000","last_active":"2024-04-25T00:00:45.360737+0000","last_peered":"2024-04-25T00:00:45.360737+0000","last_clean":"2024-04-25T00:00:45.360737+0000","last_became_active":"2024-04-24T23:34:58.402569+0000","last_became_peered":"2024-04-24T23:34:58.402569+0000","last_unstale":"2024-04-25T00:00:45.360737+0000","last_undegraded":"2024-04-25T00:00:45.360737+0000","last_fullsized":"2024-04-25T00:00:45.360737+0000","mapping_epoch":23,"log_start":"27'1538","ondisk_log_start":"27'1538","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7405,"log_dups_size":0,"ondisk_log_size":7405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:32:08.782070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6134362,"num_objects":21,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":564,"num_read_kb":143256,"num_write":6220,"num_write_kb":909081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"27'2480","reported_seq":2516,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:49.530584+0000","last_change":"2024-04-24T23:34:54.377543+0000","last_active":"2024-04-25T00:00:49.530584+0000","last_peered":"2024-04-25T00:00:49.530584+0000","last_clean":"2024-04-25T00:00:49.530584+0000","last_became_active":"2024-04-24T23:34:54.377228+0000","last_became_peered":"2024-04-24T23:34:54.377228+0000","last_unstale":"2024-04-25T00:00:49.530584+0000","last_undegraded":"2024-04-25T00:00:49.530584+0000","last_fullsized":"2024-04-25T00:00:49.530584+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2480,"log_dups_size":0,"ondisk_log_size":2480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:42.255038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"27'7206","reported_seq":7243,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.222770+0000","last_change":"2024-04-24T23:34:53.362675+0000","last_active":"2024-04-25T00:00:47.222770+0000","last_peered":"2024-04-25T00:00:47.222770+0000","last_clean":"2024-04-25T00:00:47.222770+0000","last_became_active":"2024-04-24T23:34:53.362425+0000","last_became_peered":"2024-04-24T23:34:53.362425+0000","last_unstale":"2024-04-25T00:00:47.222770+0000","last_undegraded":"2024-04-25T00:00:47.222770+0000","last_fullsized":"2024-04-25T00:00:47.222770+0000","mapping_epoch":18,"log_start":"27'300","ondisk_log_start":"27'300","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":6906,"log_dups_size":0,"ondisk_log_size":6906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:21.089467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":7208,"num_write_kb":58030,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"27'9329","reported_seq":9851,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:49.782889+0000","last_change":"2024-04-24T23:34:58.728573+0000","last_active":"2024-04-25T00:00:49.782889+0000","last_peered":"2024-04-25T00:00:49.782889+0000","last_clean":"2024-04-25T00:00:49.782889+0000","last_became_active":"2024-04-24T23:34:58.727284+0000","last_became_peered":"2024-04-24T23:34:58.727284+0000","last_unstale":"2024-04-25T00:00:49.782889+0000","last_undegraded":"2024-04-25T00:00:49.782889+0000","last_fullsized":"2024-04-25T00:00:49.782889+0000","mapping_epoch":23,"log_start":"27'2114","ondisk_log_start":"27'2114","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7215,"log_dups_size":0,"ondisk_log_size":7215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:32:08.782070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":493,"num_read_kb":125222,"num_write":6691,"num_write_kb":1110316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"27'2568","reported_seq":2604,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:44.518051+0000","last_change":"2024-04-24T23:34:54.382820+0000","last_active":"2024-04-25T00:00:44.518051+0000","last_peered":"2024-04-25T00:00:44.518051+0000","last_clean":"2024-04-25T00:00:44.518051+0000","last_became_active":"2024-04-24T23:34:54.382564+0000","last_became_peered":"2024-04-24T23:34:54.382564+0000","last_unstale":"2024-04-25T00:00:44.518051+0000","last_undegraded":"2024-04-25T00:00:44.518051+0000","last_fullsized":"2024-04-25T00:00:44.518051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2568,"log_dups_size":0,"ondisk_log_size":2568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:45.622611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"27'5189","reported_seq":5227,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.187361+0000","last_change":"2024-04-24T23:34:53.357743+0000","last_active":"2024-04-25T00:00:47.187361+0000","last_peered":"2024-04-25T00:00:47.187361+0000","last_clean":"2024-04-25T00:00:47.187361+0000","last_became_active":"2024-04-24T23:34:53.357456+0000","last_became_peered":"2024-04-24T23:34:53.357456+0000","last_unstale":"2024-04-25T00:00:47.187361+0000","last_undegraded":"2024-04-25T00:00:47.187361+0000","last_fullsized":"2024-04-25T00:00:47.187361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":5189,"log_dups_size":0,"ondisk_log_size":5189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:57:14.381644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5191,"num_write_kb":40339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":193,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-24T23:56:58.615940+0000","last_change":"2024-04-24T23:34:49.778966+0000","last_active":"2024-04-24T23:56:58.615940+0000","last_peered":"2024-04-24T23:56:58.615940+0000","last_clean":"2024-04-24T23:56:58.615940+0000","last_became_active":"2024-04-24T23:34:49.778083+0000","last_became_peered":"2024-04-24T23:34:49.778083+0000","last_unstale":"2024-04-24T23:56:58.615940+0000","last_undegraded":"2024-04-24T23:56:58.615940+0000","last_fullsized":"2024-04-24T23:56:58.615940+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:48.587078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:48.587078+0000","last_clean_scrub_stamp":"2024-04-24T23:34:48.587078+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:46:59.471089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"27'8813","reported_seq":9330,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:44.690274+0000","last_change":"2024-04-24T23:34:58.729779+0000","last_active":"2024-04-25T00:00:44.690274+0000","last_peered":"2024-04-25T00:00:44.690274+0000","last_clean":"2024-04-25T00:00:44.690274+0000","last_became_active":"2024-04-24T23:34:58.729370+0000","last_became_peered":"2024-04-24T23:34:58.729370+0000","last_unstale":"2024-04-25T00:00:44.690274+0000","last_undegraded":"2024-04-25T00:00:44.690274+0000","last_fullsized":"2024-04-25T00:00:44.690274+0000","mapping_epoch":23,"log_start":"27'902","ondisk_log_start":"27'902","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7911,"log_dups_size":0,"ondisk_log_size":7911,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:30.299011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2352473,"num_objects":16,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":487,"num_read_kb":123698,"num_write":6192,"num_write_kb":909586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"27'2518","reported_seq":2552,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:48.969863+0000","last_change":"2024-04-24T23:34:54.384106+0000","last_active":"2024-04-25T00:00:48.969863+0000","last_peered":"2024-04-25T00:00:48.969863+0000","last_clean":"2024-04-25T00:00:48.969863+0000","last_became_active":"2024-04-24T23:34:54.383589+0000","last_became_peered":"2024-04-24T23:34:54.383589+0000","last_unstale":"2024-04-25T00:00:48.969863+0000","last_undegraded":"2024-04-25T00:00:48.969863+0000","last_fullsized":"2024-04-25T00:00:48.969863+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2518,"log_dups_size":0,"ondisk_log_size":2518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:41:48.207542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"27'8231","reported_seq":8268,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.191600+0000","last_change":"2024-04-24T23:34:53.354077+0000","last_active":"2024-04-25T00:00:47.191600+0000","last_peered":"2024-04-25T00:00:47.191600+0000","last_clean":"2024-04-25T00:00:47.191600+0000","last_became_active":"2024-04-24T23:34:53.353804+0000","last_became_peered":"2024-04-24T23:34:53.353804+0000","last_unstale":"2024-04-25T00:00:47.191600+0000","last_undegraded":"2024-04-25T00:00:47.191600+0000","last_fullsized":"2024-04-25T00:00:47.191600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":8231,"log_dups_size":0,"ondisk_log_size":8231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:41.422591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8225,"num_write_kb":58738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"27'9206","reported_seq":9393,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:46.047173+0000","last_change":"2024-04-24T23:34:58.728722+0000","last_active":"2024-04-25T00:00:46.047173+0000","last_peered":"2024-04-25T00:00:46.047173+0000","last_clean":"2024-04-25T00:00:46.047173+0000","last_became_active":"2024-04-24T23:34:58.727649+0000","last_became_peered":"2024-04-24T23:34:58.727649+0000","last_unstale":"2024-04-25T00:00:46.047173+0000","last_undegraded":"2024-04-25T00:00:46.047173+0000","last_fullsized":"2024-04-25T00:00:46.047173+0000","mapping_epoch":23,"log_start":"27'1207","ondisk_log_start":"27'1207","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7999,"log_dups_size":0,"ondisk_log_size":7999,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:41:52.244767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5413297,"num_objects":32,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":158,"num_read_kb":40132,"num_write":6534,"num_write_kb":960960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"27'2542","reported_seq":2577,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.151045+0000","last_change":"2024-04-24T23:34:54.373371+0000","last_active":"2024-04-25T00:00:47.151045+0000","last_peered":"2024-04-25T00:00:47.151045+0000","last_clean":"2024-04-25T00:00:47.151045+0000","last_became_active":"2024-04-24T23:34:54.372553+0000","last_became_peered":"2024-04-24T23:34:54.372553+0000","last_unstale":"2024-04-25T00:00:47.151045+0000","last_undegraded":"2024-04-25T00:00:47.151045+0000","last_fullsized":"2024-04-25T00:00:47.151045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2542,"log_dups_size":0,"ondisk_log_size":2542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:26.261786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"27'8389","reported_seq":8426,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:40.547701+0000","last_change":"2024-04-24T23:34:53.353505+0000","last_active":"2024-04-25T00:00:40.547701+0000","last_peered":"2024-04-25T00:00:40.547701+0000","last_clean":"2024-04-25T00:00:40.547701+0000","last_became_active":"2024-04-24T23:34:53.353186+0000","last_became_peered":"2024-04-24T23:34:53.353186+0000","last_unstale":"2024-04-25T00:00:40.547701+0000","last_undegraded":"2024-04-25T00:00:40.547701+0000","last_fullsized":"2024-04-25T00:00:40.547701+0000","mapping_epoch":18,"log_start":"27'1000","ondisk_log_start":"27'1000","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":7389,"log_dups_size":0,"ondisk_log_size":7389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:42.205997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388678,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8388,"num_write_kb":58880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"27'9582","reported_seq":9937,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:48.891190+0000","last_change":"2024-04-24T23:34:58.400988+0000","last_active":"2024-04-25T00:00:48.891190+0000","last_peered":"2024-04-25T00:00:48.891190+0000","last_clean":"2024-04-25T00:00:48.891190+0000","last_became_active":"2024-04-24T23:34:58.400549+0000","last_became_peered":"2024-04-24T23:34:58.400549+0000","last_unstale":"2024-04-25T00:00:48.891190+0000","last_undegraded":"2024-04-25T00:00:48.891190+0000","last_fullsized":"2024-04-25T00:00:48.891190+0000","mapping_epoch":23,"log_start":"27'2356","ondisk_log_start":"27'2356","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7226,"log_dups_size":0,"ondisk_log_size":7226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:47:56.497734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":326,"num_read_kb":82804,"num_write":6847,"num_write_kb":886980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"27'2418","reported_seq":2452,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:48.956305+0000","last_change":"2024-04-24T23:34:54.375234+0000","last_active":"2024-04-25T00:00:48.956305+0000","last_peered":"2024-04-25T00:00:48.956305+0000","last_clean":"2024-04-25T00:00:48.956305+0000","last_became_active":"2024-04-24T23:34:54.374963+0000","last_became_peered":"2024-04-24T23:34:54.374963+0000","last_unstale":"2024-04-25T00:00:48.956305+0000","last_undegraded":"2024-04-25T00:00:48.956305+0000","last_fullsized":"2024-04-25T00:00:48.956305+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2418,"log_dups_size":0,"ondisk_log_size":2418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:47:52.460508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"27'8937","reported_seq":8974,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.146208+0000","last_change":"2024-04-24T23:34:53.360531+0000","last_active":"2024-04-25T00:00:47.146208+0000","last_peered":"2024-04-25T00:00:47.146208+0000","last_clean":"2024-04-25T00:00:47.146208+0000","last_became_active":"2024-04-24T23:34:53.360247+0000","last_became_peered":"2024-04-24T23:34:53.360247+0000","last_unstale":"2024-04-25T00:00:47.146208+0000","last_undegraded":"2024-04-25T00:00:47.146208+0000","last_fullsized":"2024-04-25T00:00:47.146208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":8937,"log_dups_size":0,"ondisk_log_size":8937,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:57:14.381644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8935,"num_write_kb":79464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"27'9249","reported_seq":10432,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:48.466460+0000","last_change":"2024-04-24T23:34:58.729798+0000","last_active":"2024-04-25T00:00:48.466460+0000","last_peered":"2024-04-25T00:00:48.466460+0000","last_clean":"2024-04-25T00:00:48.466460+0000","last_became_active":"2024-04-24T23:34:58.729345+0000","last_became_peered":"2024-04-24T23:34:58.729345+0000","last_unstale":"2024-04-25T00:00:48.466460+0000","last_undegraded":"2024-04-25T00:00:48.466460+0000","last_fullsized":"2024-04-25T00:00:48.466460+0000","mapping_epoch":23,"log_start":"27'1527","ondisk_log_start":"27'1527","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7722,"log_dups_size":0,"ondisk_log_size":7722,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:06.670518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":427171,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1154,"num_read_kb":293116,"num_write":6507,"num_write_kb":1022776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"27'2518","reported_seq":2552,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:49.157303+0000","last_change":"2024-04-24T23:34:54.377593+0000","last_active":"2024-04-25T00:00:49.157303+0000","last_peered":"2024-04-25T00:00:49.157303+0000","last_clean":"2024-04-25T00:00:49.157303+0000","last_became_active":"2024-04-24T23:34:54.377270+0000","last_became_peered":"2024-04-24T23:34:54.377270+0000","last_unstale":"2024-04-25T00:00:49.157303+0000","last_undegraded":"2024-04-25T00:00:49.157303+0000","last_fullsized":"2024-04-25T00:00:49.157303+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2518,"log_dups_size":0,"ondisk_log_size":2518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:26.261786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"27'4549","reported_seq":4586,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:08.704934+0000","last_change":"2024-04-24T23:34:53.352981+0000","last_active":"2024-04-25T00:00:08.704934+0000","last_peered":"2024-04-25T00:00:08.704934+0000","last_clean":"2024-04-25T00:00:08.704934+0000","last_became_active":"2024-04-24T23:34:53.352669+0000","last_became_peered":"2024-04-24T23:34:53.352669+0000","last_unstale":"2024-04-25T00:00:08.704934+0000","last_undegraded":"2024-04-25T00:00:08.704934+0000","last_fullsized":"2024-04-25T00:00:08.704934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":4549,"log_dups_size":0,"ondisk_log_size":4549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:09:49.550050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4539,"num_write_kb":43853,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"27'9026","reported_seq":9792,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:46.587640+0000","last_change":"2024-04-24T23:35:43.529974+0000","last_active":"2024-04-25T00:00:46.587640+0000","last_peered":"2024-04-25T00:00:46.587640+0000","last_clean":"2024-04-25T00:00:46.587640+0000","last_became_active":"2024-04-24T23:35:43.529375+0000","last_became_peered":"2024-04-24T23:35:43.529375+0000","last_unstale":"2024-04-25T00:00:46.587640+0000","last_undegraded":"2024-04-25T00:00:46.587640+0000","last_fullsized":"2024-04-25T00:00:46.587640+0000","mapping_epoch":26,"log_start":"27'1812","ondisk_log_start":"27'1812","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7214,"log_dups_size":0,"ondisk_log_size":7214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:48.286783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4258061,"num_objects":31,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":744,"num_read_kb":188976,"num_write":6336,"num_write_kb":872530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3,6],"acting":[4,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"27'2587","reported_seq":2621,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:48.619362+0000","last_change":"2024-04-24T23:34:54.376800+0000","last_active":"2024-04-25T00:00:48.619362+0000","last_peered":"2024-04-25T00:00:48.619362+0000","last_clean":"2024-04-25T00:00:48.619362+0000","last_became_active":"2024-04-24T23:34:54.376512+0000","last_became_peered":"2024-04-24T23:34:54.376512+0000","last_unstale":"2024-04-25T00:00:48.619362+0000","last_undegraded":"2024-04-25T00:00:48.619362+0000","last_fullsized":"2024-04-25T00:00:48.619362+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2587,"log_dups_size":0,"ondisk_log_size":2587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:49.048141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"27'3996","reported_seq":4033,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.147213+0000","last_change":"2024-04-24T23:34:53.355307+0000","last_active":"2024-04-25T00:00:47.147213+0000","last_peered":"2024-04-25T00:00:47.147213+0000","last_clean":"2024-04-25T00:00:47.147213+0000","last_became_active":"2024-04-24T23:34:53.354988+0000","last_became_peered":"2024-04-24T23:34:53.354988+0000","last_unstale":"2024-04-25T00:00:47.147213+0000","last_undegraded":"2024-04-25T00:00:47.147213+0000","last_fullsized":"2024-04-25T00:00:47.147213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":3996,"log_dups_size":0,"ondisk_log_size":3996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:39.457116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4001,"num_write_kb":39473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"27'9856","reported_seq":10228,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.120845+0000","last_change":"2024-04-24T23:34:58.399188+0000","last_active":"2024-04-25T00:00:50.120845+0000","last_peered":"2024-04-25T00:00:50.120845+0000","last_clean":"2024-04-25T00:00:50.120845+0000","last_became_active":"2024-04-24T23:34:58.398457+0000","last_became_peered":"2024-04-24T23:34:58.398457+0000","last_unstale":"2024-04-25T00:00:50.120845+0000","last_undegraded":"2024-04-25T00:00:50.120845+0000","last_fullsized":"2024-04-25T00:00:50.120845+0000","mapping_epoch":23,"log_start":"27'2619","ondisk_log_start":"27'2619","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7237,"log_dups_size":0,"ondisk_log_size":7237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:50:21.658540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":344,"num_read_kb":87376,"num_write":7128,"num_write_kb":1170154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"27'2449","reported_seq":2484,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:44.996971+0000","last_change":"2024-04-24T23:34:54.372150+0000","last_active":"2024-04-25T00:00:44.996971+0000","last_peered":"2024-04-25T00:00:44.996971+0000","last_clean":"2024-04-25T00:00:44.996971+0000","last_became_active":"2024-04-24T23:34:54.371703+0000","last_became_peered":"2024-04-24T23:34:54.371703+0000","last_unstale":"2024-04-25T00:00:44.996971+0000","last_undegraded":"2024-04-25T00:00:44.996971+0000","last_fullsized":"2024-04-25T00:00:44.996971+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2449,"log_dups_size":0,"ondisk_log_size":2449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:15.477061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"27'4711","reported_seq":4748,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:25.348241+0000","last_change":"2024-04-24T23:34:53.363787+0000","last_active":"2024-04-25T00:00:25.348241+0000","last_peered":"2024-04-25T00:00:25.348241+0000","last_clean":"2024-04-25T00:00:25.348241+0000","last_became_active":"2024-04-24T23:34:53.363515+0000","last_became_peered":"2024-04-24T23:34:53.363515+0000","last_unstale":"2024-04-25T00:00:25.348241+0000","last_undegraded":"2024-04-25T00:00:25.348241+0000","last_fullsized":"2024-04-25T00:00:25.348241+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":4711,"log_dups_size":0,"ondisk_log_size":4711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:50.222015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4704,"num_write_kb":43976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"27'9316","reported_seq":9703,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:46.062715+0000","last_change":"2024-04-24T23:34:58.728420+0000","last_active":"2024-04-25T00:00:46.062715+0000","last_peered":"2024-04-25T00:00:46.062715+0000","last_clean":"2024-04-25T00:00:46.062715+0000","last_became_active":"2024-04-24T23:34:58.727425+0000","last_became_peered":"2024-04-24T23:34:58.727425+0000","last_unstale":"2024-04-25T00:00:46.062715+0000","last_undegraded":"2024-04-25T00:00:46.062715+0000","last_fullsized":"2024-04-25T00:00:46.062715+0000","mapping_epoch":23,"log_start":"27'3035","ondisk_log_start":"27'3035","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":6281,"log_dups_size":0,"ondisk_log_size":6281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:19:14.487689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4376076,"num_objects":25,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":367,"num_read_kb":93218,"num_write":6599,"num_write_kb":922243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"27'2557","reported_seq":2591,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.325385+0000","last_change":"2024-04-24T23:34:54.379296+0000","last_active":"2024-04-25T00:00:47.325385+0000","last_peered":"2024-04-25T00:00:47.325385+0000","last_clean":"2024-04-25T00:00:47.325385+0000","last_became_active":"2024-04-24T23:34:54.379016+0000","last_became_peered":"2024-04-24T23:34:54.379016+0000","last_unstale":"2024-04-25T00:00:47.325385+0000","last_undegraded":"2024-04-25T00:00:47.325385+0000","last_fullsized":"2024-04-25T00:00:47.325385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2557,"log_dups_size":0,"ondisk_log_size":2557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:37.323551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"27'2767","reported_seq":2804,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.170814+0000","last_change":"2024-04-24T23:34:53.360210+0000","last_active":"2024-04-25T00:00:47.170814+0000","last_peered":"2024-04-25T00:00:47.170814+0000","last_clean":"2024-04-25T00:00:47.170814+0000","last_became_active":"2024-04-24T23:34:53.359581+0000","last_became_peered":"2024-04-24T23:34:53.359581+0000","last_unstale":"2024-04-25T00:00:47.170814+0000","last_undegraded":"2024-04-25T00:00:47.170814+0000","last_fullsized":"2024-04-25T00:00:47.170814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":2767,"log_dups_size":0,"ondisk_log_size":2767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:14.716741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2773,"num_write_kb":30370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"27'9043","reported_seq":9471,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:45.335278+0000","last_change":"2024-04-24T23:34:58.728491+0000","last_active":"2024-04-25T00:00:45.335278+0000","last_peered":"2024-04-25T00:00:45.335278+0000","last_clean":"2024-04-25T00:00:45.335278+0000","last_became_active":"2024-04-24T23:34:58.727686+0000","last_became_peered":"2024-04-24T23:34:58.727686+0000","last_unstale":"2024-04-25T00:00:45.335278+0000","last_undegraded":"2024-04-25T00:00:45.335278+0000","last_fullsized":"2024-04-25T00:00:45.335278+0000","mapping_epoch":23,"log_start":"27'2730","ondisk_log_start":"27'2730","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":6313,"log_dups_size":0,"ondisk_log_size":6313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:46.282712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4413096,"num_objects":43,"num_object_clones":0,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":441,"num_read_kb":112014,"num_write":6490,"num_write_kb":907150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"27'2526","reported_seq":2561,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.023712+0000","last_change":"2024-04-24T23:34:54.372009+0000","last_active":"2024-04-25T00:00:50.023712+0000","last_peered":"2024-04-25T00:00:50.023712+0000","last_clean":"2024-04-25T00:00:50.023712+0000","last_became_active":"2024-04-24T23:34:54.371720+0000","last_became_peered":"2024-04-24T23:34:54.371720+0000","last_unstale":"2024-04-25T00:00:50.023712+0000","last_undegraded":"2024-04-25T00:00:50.023712+0000","last_fullsized":"2024-04-25T00:00:50.023712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2526,"log_dups_size":0,"ondisk_log_size":2526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:04:07.487367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"27'6531","reported_seq":6567,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:35.064277+0000","last_change":"2024-04-24T23:34:53.353738+0000","last_active":"2024-04-25T00:00:35.064277+0000","last_peered":"2024-04-25T00:00:35.064277+0000","last_clean":"2024-04-25T00:00:35.064277+0000","last_became_active":"2024-04-24T23:34:53.353452+0000","last_became_peered":"2024-04-24T23:34:53.353452+0000","last_unstale":"2024-04-25T00:00:35.064277+0000","last_undegraded":"2024-04-25T00:00:35.064277+0000","last_fullsized":"2024-04-25T00:00:35.064277+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":6531,"log_dups_size":0,"ondisk_log_size":6531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:33:43.693095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971554,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6522,"num_write_kb":57515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"27'9599","reported_seq":10048,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:48.575361+0000","last_change":"2024-04-24T23:34:58.728503+0000","last_active":"2024-04-25T00:00:48.575361+0000","last_peered":"2024-04-25T00:00:48.575361+0000","last_clean":"2024-04-25T00:00:48.575361+0000","last_became_active":"2024-04-24T23:34:58.727921+0000","last_became_peered":"2024-04-24T23:34:58.727921+0000","last_unstale":"2024-04-25T00:00:48.575361+0000","last_undegraded":"2024-04-25T00:00:48.575361+0000","last_fullsized":"2024-04-25T00:00:48.575361+0000","mapping_epoch":23,"log_start":"27'1833","ondisk_log_start":"27'1833","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7766,"log_dups_size":0,"ondisk_log_size":7766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:19:40.928966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1518653,"num_objects":16,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":421,"num_read_kb":106934,"num_write":6985,"num_write_kb":1058941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"27'2469","reported_seq":2502,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:48.895019+0000","last_change":"2024-04-24T23:34:54.368532+0000","last_active":"2024-04-25T00:00:48.895019+0000","last_peered":"2024-04-25T00:00:48.895019+0000","last_clean":"2024-04-25T00:00:48.895019+0000","last_became_active":"2024-04-24T23:34:54.367903+0000","last_became_peered":"2024-04-24T23:34:54.367903+0000","last_unstale":"2024-04-25T00:00:48.895019+0000","last_undegraded":"2024-04-25T00:00:48.895019+0000","last_fullsized":"2024-04-25T00:00:48.895019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2469,"log_dups_size":0,"ondisk_log_size":2469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:49.372040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'9563","reported_seq":10200,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:34.614071+0000","last_change":"2024-04-24T23:34:53.362758+0000","last_active":"2024-04-25T00:00:34.614071+0000","last_peered":"2024-04-25T00:00:34.614071+0000","last_clean":"2024-04-25T00:00:34.614071+0000","last_became_active":"2024-04-24T23:34:53.362230+0000","last_became_peered":"2024-04-24T23:34:53.362230+0000","last_unstale":"2024-04-25T00:00:34.614071+0000","last_undegraded":"2024-04-25T00:00:34.614071+0000","last_fullsized":"2024-04-25T00:00:34.614071+0000","mapping_epoch":18,"log_start":"27'1600","ondisk_log_start":"27'1600","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":7963,"log_dups_size":0,"ondisk_log_size":7963,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:14.716741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389280,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":602,"num_read_kb":4494,"num_write":9563,"num_write_kb":76044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"27'8959","reported_seq":9364,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:45.247476+0000","last_change":"2024-04-24T23:34:58.400881+0000","last_active":"2024-04-25T00:00:45.247476+0000","last_peered":"2024-04-25T00:00:45.247476+0000","last_clean":"2024-04-25T00:00:45.247476+0000","last_became_active":"2024-04-24T23:34:58.400429+0000","last_became_peered":"2024-04-24T23:34:58.400429+0000","last_unstale":"2024-04-25T00:00:45.247476+0000","last_undegraded":"2024-04-25T00:00:45.247476+0000","last_fullsized":"2024-04-25T00:00:45.247476+0000","mapping_epoch":23,"log_start":"27'1625","ondisk_log_start":"27'1625","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7334,"log_dups_size":0,"ondisk_log_size":7334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:50:21.658540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1794492,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":377,"num_read_kb":95758,"num_write":6289,"num_write_kb":953618,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"27'2530","reported_seq":2564,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:44.880833+0000","last_change":"2024-04-24T23:34:54.373603+0000","last_active":"2024-04-25T00:00:44.880833+0000","last_peered":"2024-04-25T00:00:44.880833+0000","last_clean":"2024-04-25T00:00:44.880833+0000","last_became_active":"2024-04-24T23:34:54.373347+0000","last_became_peered":"2024-04-24T23:34:54.373347+0000","last_unstale":"2024-04-25T00:00:44.880833+0000","last_undegraded":"2024-04-25T00:00:44.880833+0000","last_fullsized":"2024-04-25T00:00:44.880833+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2530,"log_dups_size":0,"ondisk_log_size":2530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:13.263529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"27'5921","reported_seq":5957,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:42.665448+0000","last_change":"2024-04-24T23:34:53.354673+0000","last_active":"2024-04-25T00:00:42.665448+0000","last_peered":"2024-04-25T00:00:42.665448+0000","last_clean":"2024-04-25T00:00:42.665448+0000","last_became_active":"2024-04-24T23:34:53.354394+0000","last_became_peered":"2024-04-24T23:34:53.354394+0000","last_unstale":"2024-04-25T00:00:42.665448+0000","last_undegraded":"2024-04-25T00:00:42.665448+0000","last_fullsized":"2024-04-25T00:00:42.665448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":5921,"log_dups_size":0,"ondisk_log_size":5921,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:14.716741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5922,"num_write_kb":41234,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"27'9078","reported_seq":9483,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:46.574937+0000","last_change":"2024-04-24T23:34:58.397739+0000","last_active":"2024-04-25T00:00:46.574937+0000","last_peered":"2024-04-25T00:00:46.574937+0000","last_clean":"2024-04-25T00:00:46.574937+0000","last_became_active":"2024-04-24T23:34:58.397353+0000","last_became_peered":"2024-04-24T23:34:58.397353+0000","last_unstale":"2024-04-25T00:00:46.574937+0000","last_undegraded":"2024-04-25T00:00:46.574937+0000","last_fullsized":"2024-04-25T00:00:46.574937+0000","mapping_epoch":23,"log_start":"27'1901","ondisk_log_start":"27'1901","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7177,"log_dups_size":0,"ondisk_log_size":7177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:55.672101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4144570,"num_objects":28,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":378,"num_read_kb":96012,"num_write":6395,"num_write_kb":974866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"27'2564","reported_seq":2598,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:49.547142+0000","last_change":"2024-04-24T23:34:54.383348+0000","last_active":"2024-04-25T00:00:49.547142+0000","last_peered":"2024-04-25T00:00:49.547142+0000","last_clean":"2024-04-25T00:00:49.547142+0000","last_became_active":"2024-04-24T23:34:54.383082+0000","last_became_peered":"2024-04-24T23:34:54.383082+0000","last_unstale":"2024-04-25T00:00:49.547142+0000","last_undegraded":"2024-04-25T00:00:49.547142+0000","last_fullsized":"2024-04-25T00:00:49.547142+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2564,"log_dups_size":0,"ondisk_log_size":2564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:56:17.763159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"27'7101","reported_seq":7137,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.223449+0000","last_change":"2024-04-24T23:34:53.364540+0000","last_active":"2024-04-25T00:00:47.223449+0000","last_peered":"2024-04-25T00:00:47.223449+0000","last_clean":"2024-04-25T00:00:47.223449+0000","last_became_active":"2024-04-24T23:34:53.364259+0000","last_became_peered":"2024-04-24T23:34:53.364259+0000","last_unstale":"2024-04-25T00:00:47.223449+0000","last_undegraded":"2024-04-25T00:00:47.223449+0000","last_fullsized":"2024-04-25T00:00:47.223449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":7101,"log_dups_size":0,"ondisk_log_size":7101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:41.422591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":7102,"num_write_kb":53865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"27'8979","reported_seq":9068,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:45.779269+0000","last_change":"2024-04-24T23:34:58.405865+0000","last_active":"2024-04-25T00:00:45.779269+0000","last_peered":"2024-04-25T00:00:45.779269+0000","last_clean":"2024-04-25T00:00:45.779269+0000","last_became_active":"2024-04-24T23:34:58.405032+0000","last_became_peered":"2024-04-24T23:34:58.405032+0000","last_unstale":"2024-04-25T00:00:45.779269+0000","last_undegraded":"2024-04-25T00:00:45.779269+0000","last_fullsized":"2024-04-25T00:00:45.779269+0000","mapping_epoch":23,"log_start":"27'2712","ondisk_log_start":"27'2712","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":6267,"log_dups_size":0,"ondisk_log_size":6267,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:54.012389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4968479,"num_objects":25,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":69,"num_read_kb":17526,"num_write":6345,"num_write_kb":965302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"27'2534","reported_seq":2568,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:48.509523+0000","last_change":"2024-04-24T23:34:54.383631+0000","last_active":"2024-04-25T00:00:48.509523+0000","last_peered":"2024-04-25T00:00:48.509523+0000","last_clean":"2024-04-25T00:00:48.509523+0000","last_became_active":"2024-04-24T23:34:54.383354+0000","last_became_peered":"2024-04-24T23:34:54.383354+0000","last_unstale":"2024-04-25T00:00:48.509523+0000","last_undegraded":"2024-04-25T00:00:48.509523+0000","last_fullsized":"2024-04-25T00:00:48.509523+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2534,"log_dups_size":0,"ondisk_log_size":2534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:50:17.621315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"27'6415","reported_seq":7297,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-24T23:59:16.514501+0000","last_change":"2024-04-24T23:34:53.362409+0000","last_active":"2024-04-24T23:59:16.514501+0000","last_peered":"2024-04-24T23:59:16.514501+0000","last_clean":"2024-04-24T23:59:16.514501+0000","last_became_active":"2024-04-24T23:34:53.362103+0000","last_became_peered":"2024-04-24T23:34:53.362103+0000","last_unstale":"2024-04-24T23:59:16.514501+0000","last_undegraded":"2024-04-24T23:59:16.514501+0000","last_fullsized":"2024-04-24T23:59:16.514501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":6415,"log_dups_size":0,"ondisk_log_size":6415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:57:14.381644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":847,"num_read_kb":4657,"num_write":6414,"num_write_kb":49337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"27'9358","reported_seq":9642,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:45.950273+0000","last_change":"2024-04-24T23:34:58.730088+0000","last_active":"2024-04-25T00:00:45.950273+0000","last_peered":"2024-04-25T00:00:45.950273+0000","last_clean":"2024-04-25T00:00:45.950273+0000","last_became_active":"2024-04-24T23:34:58.729217+0000","last_became_peered":"2024-04-24T23:34:58.729217+0000","last_unstale":"2024-04-25T00:00:45.950273+0000","last_undegraded":"2024-04-25T00:00:45.950273+0000","last_fullsized":"2024-04-25T00:00:45.950273+0000","mapping_epoch":23,"log_start":"27'3013","ondisk_log_start":"27'3013","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":6345,"log_dups_size":0,"ondisk_log_size":6345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:12:14.243770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1226372,"num_objects":22,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":259,"num_read_kb":65786,"num_write":6570,"num_write_kb":1055419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"27'9147","reported_seq":9276,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:49.915079+0000","last_change":"2024-04-24T23:34:58.728892+0000","last_active":"2024-04-25T00:00:49.915079+0000","last_peered":"2024-04-25T00:00:49.915079+0000","last_clean":"2024-04-25T00:00:49.915079+0000","last_became_active":"2024-04-24T23:34:58.727914+0000","last_became_peered":"2024-04-24T23:34:58.727914+0000","last_unstale":"2024-04-25T00:00:49.915079+0000","last_undegraded":"2024-04-25T00:00:49.915079+0000","last_fullsized":"2024-04-25T00:00:49.915079+0000","mapping_epoch":23,"log_start":"27'1931","ondisk_log_start":"27'1931","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7216,"log_dups_size":0,"ondisk_log_size":7216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:56.609480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":99,"num_read_kb":25146,"num_write":6551,"num_write_kb":920441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"27'2477","reported_seq":2511,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:44.578170+0000","last_change":"2024-04-24T23:34:54.363973+0000","last_active":"2024-04-25T00:00:44.578170+0000","last_peered":"2024-04-25T00:00:44.578170+0000","last_clean":"2024-04-25T00:00:44.578170+0000","last_became_active":"2024-04-24T23:34:54.363724+0000","last_became_peered":"2024-04-24T23:34:54.363724+0000","last_unstale":"2024-04-25T00:00:44.578170+0000","last_undegraded":"2024-04-25T00:00:44.578170+0000","last_fullsized":"2024-04-25T00:00:44.578170+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2477,"log_dups_size":0,"ondisk_log_size":2477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:03:24.745895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"27'9687","reported_seq":9723,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.373826+0000","last_change":"2024-04-24T23:34:53.360041+0000","last_active":"2024-04-25T00:00:47.373826+0000","last_peered":"2024-04-25T00:00:47.373826+0000","last_clean":"2024-04-25T00:00:47.373826+0000","last_became_active":"2024-04-24T23:34:53.359707+0000","last_became_peered":"2024-04-24T23:34:53.359707+0000","last_unstale":"2024-04-25T00:00:47.373826+0000","last_undegraded":"2024-04-25T00:00:47.373826+0000","last_fullsized":"2024-04-25T00:00:47.373826+0000","mapping_epoch":18,"log_start":"27'1900","ondisk_log_start":"27'1900","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":7787,"log_dups_size":0,"ondisk_log_size":7787,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:50.222015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9677,"num_write_kb":67972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"27'9562","reported_seq":10651,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:46.082792+0000","last_change":"2024-04-24T23:35:43.834517+0000","last_active":"2024-04-25T00:00:46.082792+0000","last_peered":"2024-04-25T00:00:46.082792+0000","last_clean":"2024-04-25T00:00:46.082792+0000","last_became_active":"2024-04-24T23:35:43.834066+0000","last_became_peered":"2024-04-24T23:35:43.834066+0000","last_unstale":"2024-04-25T00:00:46.082792+0000","last_undegraded":"2024-04-25T00:00:46.082792+0000","last_fullsized":"2024-04-25T00:00:46.082792+0000","mapping_epoch":26,"log_start":"27'1608","ondisk_log_start":"27'1608","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7954,"log_dups_size":0,"ondisk_log_size":7954,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:48.286783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5985156,"num_objects":38,"num_object_clones":0,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":1054,"num_read_kb":267716,"num_write":6861,"num_write_kb":899615,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2,0],"acting":[6,5,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"27'2568","reported_seq":2602,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:44.370129+0000","last_change":"2024-04-24T23:34:54.369729+0000","last_active":"2024-04-25T00:00:44.370129+0000","last_peered":"2024-04-25T00:00:44.370129+0000","last_clean":"2024-04-25T00:00:44.370129+0000","last_became_active":"2024-04-24T23:34:54.369155+0000","last_became_peered":"2024-04-24T23:34:54.369155+0000","last_unstale":"2024-04-25T00:00:44.370129+0000","last_undegraded":"2024-04-25T00:00:44.370129+0000","last_fullsized":"2024-04-25T00:00:44.370129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2568,"log_dups_size":0,"ondisk_log_size":2568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:46:28.181996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"27'4840","reported_seq":4876,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-24T23:59:32.942342+0000","last_change":"2024-04-24T23:34:53.365328+0000","last_active":"2024-04-24T23:59:32.942342+0000","last_peered":"2024-04-24T23:59:32.942342+0000","last_clean":"2024-04-24T23:59:32.942342+0000","last_became_active":"2024-04-24T23:34:53.365029+0000","last_became_peered":"2024-04-24T23:34:53.365029+0000","last_unstale":"2024-04-24T23:59:32.942342+0000","last_undegraded":"2024-04-24T23:59:32.942342+0000","last_fullsized":"2024-04-24T23:59:32.942342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":4840,"log_dups_size":0,"ondisk_log_size":4840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:48:30.961870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4845,"num_write_kb":44132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"27'9225","reported_seq":9261,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-24T23:59:35.593651+0000","last_change":"2024-04-24T23:34:53.365123+0000","last_active":"2024-04-24T23:59:35.593651+0000","last_peered":"2024-04-24T23:59:35.593651+0000","last_clean":"2024-04-24T23:59:35.593651+0000","last_became_active":"2024-04-24T23:34:53.364821+0000","last_became_peered":"2024-04-24T23:34:53.364821+0000","last_unstale":"2024-04-24T23:59:35.593651+0000","last_undegraded":"2024-04-24T23:59:35.593651+0000","last_fullsized":"2024-04-24T23:59:35.593651+0000","mapping_epoch":18,"log_start":"27'1300","ondisk_log_start":"27'1300","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":7925,"log_dups_size":0,"ondisk_log_size":7925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:39.457116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9217,"num_write_kb":71599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"27'2524","reported_seq":2558,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.139570+0000","last_change":"2024-04-24T23:34:54.376037+0000","last_active":"2024-04-25T00:00:50.139570+0000","last_peered":"2024-04-25T00:00:50.139570+0000","last_clean":"2024-04-25T00:00:50.139570+0000","last_became_active":"2024-04-24T23:34:54.375689+0000","last_became_peered":"2024-04-24T23:34:54.375689+0000","last_unstale":"2024-04-25T00:00:50.139570+0000","last_undegraded":"2024-04-25T00:00:50.139570+0000","last_fullsized":"2024-04-25T00:00:50.139570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2524,"log_dups_size":0,"ondisk_log_size":2524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:47.446459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"27'9696","reported_seq":9920,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:48.257507+0000","last_change":"2024-04-24T23:35:43.834538+0000","last_active":"2024-04-25T00:00:48.257507+0000","last_peered":"2024-04-25T00:00:48.257507+0000","last_clean":"2024-04-25T00:00:48.257507+0000","last_became_active":"2024-04-24T23:35:43.834061+0000","last_became_peered":"2024-04-24T23:35:43.834061+0000","last_unstale":"2024-04-25T00:00:48.257507+0000","last_undegraded":"2024-04-25T00:00:48.257507+0000","last_fullsized":"2024-04-25T00:00:48.257507+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":9696,"log_dups_size":0,"ondisk_log_size":9696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:41:52.244767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3304017,"num_objects":33,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":189,"num_read_kb":48006,"num_write":6965,"num_write_kb":973944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3,5],"acting":[2,0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"27'7146","reported_seq":7182,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-24T23:58:12.312669+0000","last_change":"2024-04-24T23:34:53.363066+0000","last_active":"2024-04-24T23:58:12.312669+0000","last_peered":"2024-04-24T23:58:12.312669+0000","last_clean":"2024-04-24T23:58:12.312669+0000","last_became_active":"2024-04-24T23:34:53.362548+0000","last_became_peered":"2024-04-24T23:34:53.362548+0000","last_unstale":"2024-04-24T23:58:12.312669+0000","last_undegraded":"2024-04-24T23:58:12.312669+0000","last_fullsized":"2024-04-24T23:58:12.312669+0000","mapping_epoch":18,"log_start":"27'800","ondisk_log_start":"27'800","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":6346,"log_dups_size":0,"ondisk_log_size":6346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:21:13.338964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7133,"num_write_kb":66117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"27'2549","reported_seq":2583,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:44.471789+0000","last_change":"2024-04-24T23:34:54.365704+0000","last_active":"2024-04-25T00:00:44.471789+0000","last_peered":"2024-04-25T00:00:44.471789+0000","last_clean":"2024-04-25T00:00:44.471789+0000","last_became_active":"2024-04-24T23:34:54.364814+0000","last_became_peered":"2024-04-24T23:34:54.364814+0000","last_unstale":"2024-04-25T00:00:44.471789+0000","last_undegraded":"2024-04-25T00:00:44.471789+0000","last_fullsized":"2024-04-25T00:00:44.471789+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2549,"log_dups_size":0,"ondisk_log_size":2549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:22.094785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"27'9291","reported_seq":9497,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:45.850805+0000","last_change":"2024-04-24T23:34:58.407186+0000","last_active":"2024-04-25T00:00:45.850805+0000","last_peered":"2024-04-25T00:00:45.850805+0000","last_clean":"2024-04-25T00:00:45.850805+0000","last_became_active":"2024-04-24T23:34:58.406811+0000","last_became_peered":"2024-04-24T23:34:58.406811+0000","last_unstale":"2024-04-25T00:00:45.850805+0000","last_undegraded":"2024-04-25T00:00:45.850805+0000","last_fullsized":"2024-04-25T00:00:45.850805+0000","mapping_epoch":23,"log_start":"27'3012","ondisk_log_start":"27'3012","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":6279,"log_dups_size":0,"ondisk_log_size":6279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:55.861555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6226351,"num_objects":17,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":178,"num_read_kb":45212,"num_write":6534,"num_write_kb":1041117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"27'2439","reported_seq":2475,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:44.473291+0000","last_change":"2024-04-24T23:34:54.375142+0000","last_active":"2024-04-25T00:00:44.473291+0000","last_peered":"2024-04-25T00:00:44.473291+0000","last_clean":"2024-04-25T00:00:44.473291+0000","last_became_active":"2024-04-24T23:34:54.374844+0000","last_became_peered":"2024-04-24T23:34:54.374844+0000","last_unstale":"2024-04-25T00:00:44.473291+0000","last_undegraded":"2024-04-25T00:00:44.473291+0000","last_fullsized":"2024-04-25T00:00:44.473291+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2439,"log_dups_size":0,"ondisk_log_size":2439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:29:07.593498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"27'2583","reported_seq":2619,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:48.181599+0000","last_change":"2024-04-24T23:34:54.383574+0000","last_active":"2024-04-25T00:00:48.181599+0000","last_peered":"2024-04-25T00:00:48.181599+0000","last_clean":"2024-04-25T00:00:48.181599+0000","last_became_active":"2024-04-24T23:34:54.383253+0000","last_became_peered":"2024-04-24T23:34:54.383253+0000","last_unstale":"2024-04-25T00:00:48.181599+0000","last_undegraded":"2024-04-25T00:00:48.181599+0000","last_fullsized":"2024-04-25T00:00:48.181599+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2583,"log_dups_size":0,"ondisk_log_size":2583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:43.211314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"27'2465","reported_seq":2487,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.944438+0000","last_change":"2024-04-24T23:35:43.526600+0000","last_active":"2024-04-25T00:00:47.944438+0000","last_peered":"2024-04-25T00:00:47.944438+0000","last_clean":"2024-04-25T00:00:47.944438+0000","last_became_active":"2024-04-24T23:35:43.525929+0000","last_became_peered":"2024-04-24T23:35:43.525929+0000","last_unstale":"2024-04-25T00:00:47.944438+0000","last_undegraded":"2024-04-25T00:00:47.944438+0000","last_fullsized":"2024-04-25T00:00:47.944438+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2465,"log_dups_size":0,"ondisk_log_size":2465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:49.372040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"27'2575","reported_seq":2611,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:48.937515+0000","last_change":"2024-04-24T23:34:54.366271+0000","last_active":"2024-04-25T00:00:48.937515+0000","last_peered":"2024-04-25T00:00:48.937515+0000","last_clean":"2024-04-25T00:00:48.937515+0000","last_became_active":"2024-04-24T23:34:54.366005+0000","last_became_peered":"2024-04-24T23:34:54.366005+0000","last_unstale":"2024-04-25T00:00:48.937515+0000","last_undegraded":"2024-04-25T00:00:48.937515+0000","last_fullsized":"2024-04-25T00:00:48.937515+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2575,"log_dups_size":0,"ondisk_log_size":2575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:03:24.745895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"27'2559","reported_seq":2594,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:44.819878+0000","last_change":"2024-04-24T23:34:54.360529+0000","last_active":"2024-04-25T00:00:44.819878+0000","last_peered":"2024-04-25T00:00:44.819878+0000","last_clean":"2024-04-25T00:00:44.819878+0000","last_became_active":"2024-04-24T23:34:54.360266+0000","last_became_peered":"2024-04-24T23:34:54.360266+0000","last_unstale":"2024-04-25T00:00:44.819878+0000","last_undegraded":"2024-04-25T00:00:44.819878+0000","last_fullsized":"2024-04-25T00:00:44.819878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2559,"log_dups_size":0,"ondisk_log_size":2559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:49:47.246193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"27'2432","reported_seq":2467,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:44.680703+0000","last_change":"2024-04-24T23:34:54.365744+0000","last_active":"2024-04-25T00:00:44.680703+0000","last_peered":"2024-04-25T00:00:44.680703+0000","last_clean":"2024-04-25T00:00:44.680703+0000","last_became_active":"2024-04-24T23:34:54.365493+0000","last_became_peered":"2024-04-24T23:34:54.365493+0000","last_unstale":"2024-04-25T00:00:44.680703+0000","last_undegraded":"2024-04-25T00:00:44.680703+0000","last_fullsized":"2024-04-25T00:00:44.680703+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2432,"log_dups_size":0,"ondisk_log_size":2432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:15.477061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"27'2562","reported_seq":2597,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:44.988811+0000","last_change":"2024-04-24T23:34:54.379642+0000","last_active":"2024-04-25T00:00:44.988811+0000","last_peered":"2024-04-25T00:00:44.988811+0000","last_clean":"2024-04-25T00:00:44.988811+0000","last_became_active":"2024-04-24T23:34:54.378568+0000","last_became_peered":"2024-04-24T23:34:54.378568+0000","last_unstale":"2024-04-25T00:00:44.988811+0000","last_undegraded":"2024-04-25T00:00:44.988811+0000","last_fullsized":"2024-04-25T00:00:44.988811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2562,"log_dups_size":0,"ondisk_log_size":2562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:42.199856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"27'2530","reported_seq":2565,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:45.015897+0000","last_change":"2024-04-24T23:34:54.363700+0000","last_active":"2024-04-25T00:00:45.015897+0000","last_peered":"2024-04-25T00:00:45.015897+0000","last_clean":"2024-04-25T00:00:45.015897+0000","last_became_active":"2024-04-24T23:34:54.363439+0000","last_became_peered":"2024-04-24T23:34:54.363439+0000","last_unstale":"2024-04-25T00:00:45.015897+0000","last_undegraded":"2024-04-25T00:00:45.015897+0000","last_fullsized":"2024-04-25T00:00:45.015897+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2530,"log_dups_size":0,"ondisk_log_size":2530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:28:56.048178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"27'2531","reported_seq":2566,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:48.430106+0000","last_change":"2024-04-24T23:34:54.375460+0000","last_active":"2024-04-25T00:00:48.430106+0000","last_peered":"2024-04-25T00:00:48.430106+0000","last_clean":"2024-04-25T00:00:48.430106+0000","last_became_active":"2024-04-24T23:34:54.375123+0000","last_became_peered":"2024-04-24T23:34:54.375123+0000","last_unstale":"2024-04-25T00:00:48.430106+0000","last_undegraded":"2024-04-25T00:00:48.430106+0000","last_fullsized":"2024-04-25T00:00:48.430106+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2531,"log_dups_size":0,"ondisk_log_size":2531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:18.207267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"27'2496","reported_seq":2531,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:44.958661+0000","last_change":"2024-04-24T23:34:54.369509+0000","last_active":"2024-04-25T00:00:44.958661+0000","last_peered":"2024-04-25T00:00:44.958661+0000","last_clean":"2024-04-25T00:00:44.958661+0000","last_became_active":"2024-04-24T23:34:54.369219+0000","last_became_peered":"2024-04-24T23:34:54.369219+0000","last_unstale":"2024-04-25T00:00:44.958661+0000","last_undegraded":"2024-04-25T00:00:44.958661+0000","last_fullsized":"2024-04-25T00:00:44.958661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2496,"log_dups_size":0,"ondisk_log_size":2496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:09:37.649503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"27'2551","reported_seq":2572,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:48.387420+0000","last_change":"2024-04-24T23:35:43.933911+0000","last_active":"2024-04-25T00:00:48.387420+0000","last_peered":"2024-04-25T00:00:48.387420+0000","last_clean":"2024-04-25T00:00:48.387420+0000","last_became_active":"2024-04-24T23:35:43.933686+0000","last_became_peered":"2024-04-24T23:35:43.933686+0000","last_unstale":"2024-04-25T00:00:48.387420+0000","last_undegraded":"2024-04-25T00:00:48.387420+0000","last_fullsized":"2024-04-25T00:00:48.387420+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2551,"log_dups_size":0,"ondisk_log_size":2551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:26.261786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2b","version":"27'2585","reported_seq":2620,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:48.808345+0000","last_change":"2024-04-24T23:34:54.371052+0000","last_active":"2024-04-25T00:00:48.808345+0000","last_peered":"2024-04-25T00:00:48.808345+0000","last_clean":"2024-04-25T00:00:48.808345+0000","last_became_active":"2024-04-24T23:34:54.370793+0000","last_became_peered":"2024-04-24T23:34:54.370793+0000","last_unstale":"2024-04-25T00:00:48.808345+0000","last_undegraded":"2024-04-25T00:00:48.808345+0000","last_fullsized":"2024-04-25T00:00:48.808345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2585,"log_dups_size":0,"ondisk_log_size":2585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:29:07.593498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"27'2564","reported_seq":2599,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:45.080910+0000","last_change":"2024-04-24T23:34:54.377190+0000","last_active":"2024-04-25T00:00:45.080910+0000","last_peered":"2024-04-25T00:00:45.080910+0000","last_clean":"2024-04-25T00:00:45.080910+0000","last_became_active":"2024-04-24T23:34:54.376940+0000","last_became_peered":"2024-04-24T23:34:54.376940+0000","last_unstale":"2024-04-25T00:00:45.080910+0000","last_undegraded":"2024-04-25T00:00:45.080910+0000","last_fullsized":"2024-04-25T00:00:45.080910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2564,"log_dups_size":0,"ondisk_log_size":2564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:03:24.745895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"27'2570","reported_seq":2604,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:49.071176+0000","last_change":"2024-04-24T23:34:54.368770+0000","last_active":"2024-04-25T00:00:49.071176+0000","last_peered":"2024-04-25T00:00:49.071176+0000","last_clean":"2024-04-25T00:00:49.071176+0000","last_became_active":"2024-04-24T23:34:54.368228+0000","last_became_peered":"2024-04-24T23:34:54.368228+0000","last_unstale":"2024-04-25T00:00:49.071176+0000","last_undegraded":"2024-04-25T00:00:49.071176+0000","last_fullsized":"2024-04-25T00:00:49.071176+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2570,"log_dups_size":0,"ondisk_log_size":2570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:51.634876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"27'2510","reported_seq":2545,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.391247+0000","last_change":"2024-04-24T23:34:54.373038+0000","last_active":"2024-04-25T00:00:47.391247+0000","last_peered":"2024-04-25T00:00:47.391247+0000","last_clean":"2024-04-25T00:00:47.391247+0000","last_became_active":"2024-04-24T23:34:54.372766+0000","last_became_peered":"2024-04-24T23:34:54.372766+0000","last_unstale":"2024-04-25T00:00:47.391247+0000","last_undegraded":"2024-04-25T00:00:47.391247+0000","last_fullsized":"2024-04-25T00:00:47.391247+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2510,"log_dups_size":0,"ondisk_log_size":2510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:13.263529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"27'2577","reported_seq":2612,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.872643+0000","last_change":"2024-04-24T23:34:54.383733+0000","last_active":"2024-04-25T00:00:47.872643+0000","last_peered":"2024-04-25T00:00:47.872643+0000","last_clean":"2024-04-25T00:00:47.872643+0000","last_became_active":"2024-04-24T23:34:54.383463+0000","last_became_peered":"2024-04-24T23:34:54.383463+0000","last_unstale":"2024-04-25T00:00:47.872643+0000","last_undegraded":"2024-04-25T00:00:47.872643+0000","last_fullsized":"2024-04-25T00:00:47.872643+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2577,"log_dups_size":0,"ondisk_log_size":2577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:51.634876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"27'2508","reported_seq":2543,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:48.402598+0000","last_change":"2024-04-24T23:34:54.380959+0000","last_active":"2024-04-25T00:00:48.402598+0000","last_peered":"2024-04-25T00:00:48.402598+0000","last_clean":"2024-04-25T00:00:48.402598+0000","last_became_active":"2024-04-24T23:34:54.380702+0000","last_became_peered":"2024-04-24T23:34:54.380702+0000","last_unstale":"2024-04-25T00:00:48.402598+0000","last_undegraded":"2024-04-25T00:00:48.402598+0000","last_fullsized":"2024-04-25T00:00:48.402598+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2508,"log_dups_size":0,"ondisk_log_size":2508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:45.424046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"27'2625","reported_seq":2660,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:44.836426+0000","last_change":"2024-04-24T23:34:54.358461+0000","last_active":"2024-04-25T00:00:44.836426+0000","last_peered":"2024-04-25T00:00:44.836426+0000","last_clean":"2024-04-25T00:00:44.836426+0000","last_became_active":"2024-04-24T23:34:54.358203+0000","last_became_peered":"2024-04-24T23:34:54.358203+0000","last_unstale":"2024-04-25T00:00:44.836426+0000","last_undegraded":"2024-04-25T00:00:44.836426+0000","last_fullsized":"2024-04-25T00:00:44.836426+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2625,"log_dups_size":0,"ondisk_log_size":2625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:18.207267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"27'2489","reported_seq":2524,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:49.786683+0000","last_change":"2024-04-24T23:34:54.367860+0000","last_active":"2024-04-25T00:00:49.786683+0000","last_peered":"2024-04-25T00:00:49.786683+0000","last_clean":"2024-04-25T00:00:49.786683+0000","last_became_active":"2024-04-24T23:34:54.367595+0000","last_became_peered":"2024-04-24T23:34:54.367595+0000","last_unstale":"2024-04-25T00:00:49.786683+0000","last_undegraded":"2024-04-25T00:00:49.786683+0000","last_fullsized":"2024-04-25T00:00:49.786683+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2489,"log_dups_size":0,"ondisk_log_size":2489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:40:40.928386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"27'2462","reported_seq":2482,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:45.113749+0000","last_change":"2024-04-24T23:35:43.529991+0000","last_active":"2024-04-25T00:00:45.113749+0000","last_peered":"2024-04-25T00:00:45.113749+0000","last_clean":"2024-04-25T00:00:45.113749+0000","last_became_active":"2024-04-24T23:35:43.529631+0000","last_became_peered":"2024-04-24T23:35:43.529631+0000","last_unstale":"2024-04-25T00:00:45.113749+0000","last_undegraded":"2024-04-25T00:00:45.113749+0000","last_fullsized":"2024-04-25T00:00:45.113749+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2462,"log_dups_size":0,"ondisk_log_size":2462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:40:39.166560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.34","version":"27'2422","reported_seq":2456,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:44.924916+0000","last_change":"2024-04-24T23:34:54.377573+0000","last_active":"2024-04-25T00:00:44.924916+0000","last_peered":"2024-04-25T00:00:44.924916+0000","last_clean":"2024-04-25T00:00:44.924916+0000","last_became_active":"2024-04-24T23:34:54.377277+0000","last_became_peered":"2024-04-24T23:34:54.377277+0000","last_unstale":"2024-04-25T00:00:44.924916+0000","last_undegraded":"2024-04-25T00:00:44.924916+0000","last_fullsized":"2024-04-25T00:00:44.924916+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2422,"log_dups_size":0,"ondisk_log_size":2422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:43.211314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"27'2466","reported_seq":2500,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:44.929859+0000","last_change":"2024-04-24T23:34:54.388134+0000","last_active":"2024-04-25T00:00:44.929859+0000","last_peered":"2024-04-25T00:00:44.929859+0000","last_clean":"2024-04-25T00:00:44.929859+0000","last_became_active":"2024-04-24T23:34:54.387781+0000","last_became_peered":"2024-04-24T23:34:54.387781+0000","last_unstale":"2024-04-25T00:00:44.929859+0000","last_undegraded":"2024-04-25T00:00:44.929859+0000","last_fullsized":"2024-04-25T00:00:44.929859+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2466,"log_dups_size":0,"ondisk_log_size":2466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:56:17.763159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"27'2585","reported_seq":2619,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:44.928263+0000","last_change":"2024-04-24T23:34:54.388182+0000","last_active":"2024-04-25T00:00:44.928263+0000","last_peered":"2024-04-25T00:00:44.928263+0000","last_clean":"2024-04-25T00:00:44.928263+0000","last_became_active":"2024-04-24T23:34:54.387913+0000","last_became_peered":"2024-04-24T23:34:54.387913+0000","last_unstale":"2024-04-25T00:00:44.928263+0000","last_undegraded":"2024-04-25T00:00:44.928263+0000","last_fullsized":"2024-04-25T00:00:44.928263+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2585,"log_dups_size":0,"ondisk_log_size":2585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:18.207267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"27'2556","reported_seq":2589,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.919316+0000","last_change":"2024-04-24T23:34:54.383303+0000","last_active":"2024-04-25T00:00:47.919316+0000","last_peered":"2024-04-25T00:00:47.919316+0000","last_clean":"2024-04-25T00:00:47.919316+0000","last_became_active":"2024-04-24T23:34:54.383024+0000","last_became_peered":"2024-04-24T23:34:54.383024+0000","last_unstale":"2024-04-25T00:00:47.919316+0000","last_undegraded":"2024-04-25T00:00:47.919316+0000","last_fullsized":"2024-04-25T00:00:47.919316+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2556,"log_dups_size":0,"ondisk_log_size":2556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:45.424046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"27'2492","reported_seq":2526,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:44.928077+0000","last_change":"2024-04-24T23:34:54.382803+0000","last_active":"2024-04-25T00:00:44.928077+0000","last_peered":"2024-04-25T00:00:44.928077+0000","last_clean":"2024-04-25T00:00:44.928077+0000","last_became_active":"2024-04-24T23:34:54.382482+0000","last_became_peered":"2024-04-24T23:34:54.382482+0000","last_unstale":"2024-04-25T00:00:44.928077+0000","last_undegraded":"2024-04-25T00:00:44.928077+0000","last_fullsized":"2024-04-25T00:00:44.928077+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2492,"log_dups_size":0,"ondisk_log_size":2492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:27:33.721584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"27'2525","reported_seq":2558,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:49.069349+0000","last_change":"2024-04-24T23:34:54.369167+0000","last_active":"2024-04-25T00:00:49.069349+0000","last_peered":"2024-04-25T00:00:49.069349+0000","last_clean":"2024-04-25T00:00:49.069349+0000","last_became_active":"2024-04-24T23:34:54.368663+0000","last_became_peered":"2024-04-24T23:34:54.368663+0000","last_unstale":"2024-04-25T00:00:49.069349+0000","last_undegraded":"2024-04-25T00:00:49.069349+0000","last_fullsized":"2024-04-25T00:00:49.069349+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2525,"log_dups_size":0,"ondisk_log_size":2525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:18.207267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"27'2452","reported_seq":2485,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:48.608029+0000","last_change":"2024-04-24T23:34:54.380345+0000","last_active":"2024-04-25T00:00:48.608029+0000","last_peered":"2024-04-25T00:00:48.608029+0000","last_clean":"2024-04-25T00:00:48.608029+0000","last_became_active":"2024-04-24T23:34:54.380052+0000","last_became_peered":"2024-04-24T23:34:54.380052+0000","last_unstale":"2024-04-25T00:00:48.608029+0000","last_undegraded":"2024-04-25T00:00:48.608029+0000","last_fullsized":"2024-04-25T00:00:48.608029+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2452,"log_dups_size":0,"ondisk_log_size":2452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:42.199856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"27'2497","reported_seq":2530,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:49.181531+0000","last_change":"2024-04-24T23:34:54.365114+0000","last_active":"2024-04-25T00:00:49.181531+0000","last_peered":"2024-04-25T00:00:49.181531+0000","last_clean":"2024-04-25T00:00:49.181531+0000","last_became_active":"2024-04-24T23:34:54.364605+0000","last_became_peered":"2024-04-24T23:34:54.364605+0000","last_unstale":"2024-04-25T00:00:49.181531+0000","last_undegraded":"2024-04-25T00:00:49.181531+0000","last_fullsized":"2024-04-25T00:00:49.181531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2497,"log_dups_size":0,"ondisk_log_size":2497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:07:28.785202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"27'2544","reported_seq":2566,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.971798+0000","last_change":"2024-04-24T23:35:43.525630+0000","last_active":"2024-04-25T00:00:47.971798+0000","last_peered":"2024-04-25T00:00:47.971798+0000","last_clean":"2024-04-25T00:00:47.971798+0000","last_became_active":"2024-04-24T23:35:43.525191+0000","last_became_peered":"2024-04-24T23:35:43.525191+0000","last_unstale":"2024-04-25T00:00:47.971798+0000","last_undegraded":"2024-04-25T00:00:47.971798+0000","last_fullsized":"2024-04-25T00:00:47.971798+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2544,"log_dups_size":0,"ondisk_log_size":2544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:07:28.785202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"27'2608","reported_seq":2644,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.023985+0000","last_change":"2024-04-24T23:34:54.366079+0000","last_active":"2024-04-25T00:00:50.023985+0000","last_peered":"2024-04-25T00:00:50.023985+0000","last_clean":"2024-04-25T00:00:50.023985+0000","last_became_active":"2024-04-24T23:34:54.365814+0000","last_became_peered":"2024-04-24T23:34:54.365814+0000","last_unstale":"2024-04-25T00:00:50.023985+0000","last_undegraded":"2024-04-25T00:00:50.023985+0000","last_fullsized":"2024-04-25T00:00:50.023985+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2608,"log_dups_size":0,"ondisk_log_size":2608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:18.207267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"27'2449","reported_seq":2485,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:44.815128+0000","last_change":"2024-04-24T23:34:54.368976+0000","last_active":"2024-04-25T00:00:44.815128+0000","last_peered":"2024-04-25T00:00:44.815128+0000","last_clean":"2024-04-25T00:00:44.815128+0000","last_became_active":"2024-04-24T23:34:54.368663+0000","last_became_peered":"2024-04-24T23:34:54.368663+0000","last_unstale":"2024-04-25T00:00:44.815128+0000","last_undegraded":"2024-04-25T00:00:44.815128+0000","last_fullsized":"2024-04-25T00:00:44.815128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2449,"log_dups_size":0,"ondisk_log_size":2449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:56:17.763159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"27'2453","reported_seq":2488,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:49.117795+0000","last_change":"2024-04-24T23:34:54.373810+0000","last_active":"2024-04-25T00:00:49.117795+0000","last_peered":"2024-04-25T00:00:49.117795+0000","last_clean":"2024-04-25T00:00:49.117795+0000","last_became_active":"2024-04-24T23:34:54.373531+0000","last_became_peered":"2024-04-24T23:34:54.373531+0000","last_unstale":"2024-04-25T00:00:49.117795+0000","last_undegraded":"2024-04-25T00:00:49.117795+0000","last_fullsized":"2024-04-25T00:00:49.117795+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2453,"log_dups_size":0,"ondisk_log_size":2453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:50:17.621315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":111785901,"num_objects":673,"num_object_clones":0,"num_object_copies":2692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":673,"num_whiteouts":0,"num_read":12310,"num_read_kb":3126740,"num_write":210948,"num_write_kb":30633802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24567808,"data_stored":221478912,"data_compressed":10295227,"data_compressed_allocated":16637952,"data_compressed_original":213549056,"omap_allocated":0,"internal_metadata":0},"log_size":230878,"ondisk_log_size":230878,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":114,"num_object_clones":0,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1006,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161566,"ondisk_log_size":161566,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":233539226,"num_objects":127,"num_object_clones":0,"num_object_copies":254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":127,"num_whiteouts":0,"num_read":3152,"num_read_kb":19253,"num_write":210043,"num_write_kb":1679720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":63,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":287272960,"data_stored":469797651,"data_compressed":55273429,"data_compressed_allocated":80859136,"data_compressed_original":263442432,"omap_allocated":0,"internal_metadata":0},"log_size":195599,"ondisk_log_size":195599,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65248,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542458,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":963152,"kb_used_data":29704,"kb_used_omap":1,"kb_used_meta":933438,"kb_avail":93408688,"statfs":{"total":96636764160,"available":95650496512,"internally_reserved":0,"allocated":30416896,"data_stored":56977813,"data_compressed":5989303,"data_compressed_allocated":9019392,"data_compressed_original":35758080,"omap_allocated":1590,"internal_metadata":955840970},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,9,8,11,10,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542456,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1121104,"kb_used_data":43332,"kb_used_omap":1,"kb_used_meta":1077758,"kb_avail":93250736,"statfs":{"total":96636764160,"available":95488753664,"internally_reserved":0,"allocated":44371968,"data_stored":102701999,"data_compressed":9657573,"data_compressed_allocated":14041088,"data_compressed_original":72548352,"omap_allocated":1594,"internal_metadata":1103624646},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542457,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":978456,"kb_used_data":27024,"kb_used_omap":1,"kb_used_meta":951422,"kb_avail":93393384,"statfs":{"total":96636764160,"available":95634825216,"internally_reserved":0,"allocated":27672576,"data_stored":58276474,"data_compressed":5800819,"data_compressed_allocated":8785920,"data_compressed_original":39563264,"omap_allocated":1595,"internal_metadata":974256581},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542456,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":919584,"kb_used_data":44376,"kb_used_omap":1,"kb_used_meta":875198,"kb_avail":93452256,"statfs":{"total":96636764160,"available":95695110144,"internally_reserved":0,"allocated":45441024,"data_stored":102992542,"data_compressed":9448676,"data_compressed_allocated":13889536,"data_compressed_original":71622656,"omap_allocated":1595,"internal_metadata":896203205},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,12,12,26,43,46,253],"upper_bound":1024},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542458,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":663352,"kb_used_data":36592,"kb_used_omap":1,"kb_used_meta":626750,"kb_avail":93708488,"statfs":{"total":96636764160,"available":95957491712,"internally_reserved":0,"allocated":37470208,"data_stored":74025504,"data_compressed":7080739,"data_compressed_allocated":10862592,"data_compressed_original":47587328,"omap_allocated":1595,"internal_metadata":641792453},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542457,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1233860,"kb_used_data":43564,"kb_used_omap":1,"kb_used_meta":1190270,"kb_avail":93137980,"statfs":{"total":96636764160,"available":95373291520,"internally_reserved":0,"allocated":44609536,"data_stored":108677824,"data_compressed":9380730,"data_compressed_allocated":13918208,"data_compressed_original":78163968,"omap_allocated":1595,"internal_metadata":1218836933},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542458,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":823368,"kb_used_data":34304,"kb_used_omap":1,"kb_used_meta":789054,"kb_avail":93548472,"statfs":{"total":96636764160,"available":95793635328,"internally_reserved":0,"allocated":35127296,"data_stored":63717002,"data_compressed":7508106,"data_compressed_allocated":10883072,"data_compressed_original":39657472,"omap_allocated":1595,"internal_metadata":807991749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4],"upper_bound":16},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575162,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1146004,"kb_used_data":45964,"kb_used_omap":1,"kb_used_meta":1100030,"kb_avail":93225836,"statfs":{"total":96636764160,"available":95463256064,"internally_reserved":0,"allocated":47067136,"data_stored":117951222,"data_compressed":10359157,"data_compressed_allocated":15155200,"data_compressed_original":86216704,"omap_allocated":1593,"internal_metadata":1126431175},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32624,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32624,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":34275328,"data_stored":59417162,"data_compressed":7311941,"data_compressed_allocated":10522624,"data_compressed_original":35676160,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40267776,"data_stored":62914720,"data_compressed":7244862,"data_compressed_allocated":10850304,"data_compressed_original":33505280,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":34680832,"data_stored":54525952,"data_compressed":6190216,"data_compressed_allocated":9224192,"data_compressed_original":29069312,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":41914368,"data_stored":69131938,"data_compressed":8117877,"data_compressed_allocated":11812864,"data_compressed_original":39038976,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":41668608,"data_stored":67108990,"data_compressed":7785480,"data_compressed_allocated":11411456,"data_compressed_original":36864000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":25022464,"data_stored":41943130,"data_compressed":5058379,"data_compressed_allocated":7299072,"data_compressed_original":24223744,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40677376,"data_stored":67789199,"data_compressed":8032433,"data_compressed_allocated":11632640,"data_compressed_original":38752256,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28766208,"data_stored":46966560,"data_compressed":5532241,"data_compressed_allocated":8105984,"data_compressed_original":26312704,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":385024,"data_stored":2519040,"data_compressed":113842,"data_compressed_allocated":229376,"data_compressed_original":2363392,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4562944,"data_stored":46469120,"data_compressed":2170165,"data_compressed_allocated":3207168,"data_compressed_original":45113344,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2228224,"data_stored":16973824,"data_compressed":779748,"data_compressed_allocated":1454080,"data_compressed_original":16199680,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5431296,"data_stored":53047296,"data_compressed":2469993,"data_compressed_allocated":3739648,"data_compressed_original":51355648,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4603904,"data_stored":42082304,"data_compressed":1962538,"data_compressed_allocated":3190784,"data_compressed_original":40669184,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1859584,"data_stored":12443648,"data_compressed":567456,"data_compressed_allocated":1142784,"data_compressed_original":11726848,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4456448,"data_stored":40402944,"data_compressed":1886828,"data_compressed_allocated":2969600,"data_compressed_original":38916096,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1040384,"data_stored":7540736,"data_compressed":344657,"data_compressed_allocated":704512,"data_compressed_original":7204864,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:00:52.056 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T00:00:52.056 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:00:52.193 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.194+0000 7f649f110640 1 Processor -- start 2024-04-25T00:00:52.194 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.194+0000 7f649f110640 1 -- start start 2024-04-25T00:00:52.195 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.194+0000 7f649f110640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6498152b40 0x7f64980c5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:52.196 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.194+0000 7f649f110640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f64980be360 0x7f6498155350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:52.196 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.194+0000 7f649f110640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64980c8bb0 con 0x7f64980c5b70 2024-04-25T00:00:52.196 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.194+0000 7f649f110640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f64980c8e90 con 0x7f64980be360 2024-04-25T00:00:52.196 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.194+0000 7f649f110640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6498159480 con 0x7f6498152b40 2024-04-25T00:00:52.196 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.195+0000 7f649d6ad640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f64980c5b70 0x7f6498155890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:55252/0 (socket says 172.21.15.157:55252) 2024-04-25T00:00:52.196 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.195+0000 7f649d6ad640 1 -- 172.21.15.157:0/2511582662 learned_addr learned my addr 172.21.15.157:0/2511582662 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-25T00:00:52.196 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.195+0000 7f649ceac640 1 --2- 172.21.15.157:0/2511582662 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6498152b40 0x7f64980c5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:52.196 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.195+0000 7f6497fff640 1 --2- 172.21.15.157:0/2511582662 >> v2:172.21.15.169:3300/0 conn(0x7f64980be360 0x7f6498155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:52.196 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.195+0000 7f64977fe640 1 -- 172.21.15.157:0/2511582662 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 873457037 0 0) 0x7f64980c8bb0 con 0x7f64980c5b70 2024-04-25T00:00:52.196 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.195+0000 7f64977fe640 1 -- 172.21.15.157:0/2511582662 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6488003520 con 0x7f64980c5b70 2024-04-25T00:00:52.196 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.195+0000 7f649ceac640 1 -- 172.21.15.157:0/2511582662 >> v2:172.21.15.169:3300/0 conn(0x7f64980be360 msgr2=0x7f6498155350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:52.196 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.195+0000 7f649ceac640 1 --2- 172.21.15.157:0/2511582662 >> v2:172.21.15.169:3300/0 conn(0x7f64980be360 0x7f6498155350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:52.196 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.195+0000 7f649ceac640 1 -- 172.21.15.157:0/2511582662 >> v1:172.21.15.157:6789/0 conn(0x7f64980c5b70 legacy=0x7f6498155890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:52.197 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.195+0000 7f649ceac640 1 -- 172.21.15.157:0/2511582662 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6498159760 con 0x7f6498152b40 2024-04-25T00:00:52.197 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.195+0000 7f6497fff640 1 --2- 172.21.15.157:0/2511582662 >> v2:172.21.15.169:3300/0 conn(0x7f64980be360 0x7f6498155350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:00:52.197 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.196+0000 7f649ceac640 1 --2- 172.21.15.157:0/2511582662 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6498152b40 0x7f64980c5630 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f648c0097a0 tx=0x7f648c02a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63f4d2b92c9b4340 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:52.197 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.196+0000 7f64977fe640 1 -- 172.21.15.157:0/2511582662 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648c037070 con 0x7f6498152b40 2024-04-25T00:00:52.197 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.196+0000 7f64977fe640 1 -- 172.21.15.157:0/2511582662 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f648c02adb0 con 0x7f6498152b40 2024-04-25T00:00:52.197 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.196+0000 7f64977fe640 1 -- 172.21.15.157:0/2511582662 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648c032640 con 0x7f6498152b40 2024-04-25T00:00:52.198 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.197+0000 7f649f110640 1 -- 172.21.15.157:0/2511582662 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6498152b40 msgr2=0x7f64980c5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:52.198 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.197+0000 7f649f110640 1 --2- 172.21.15.157:0/2511582662 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6498152b40 0x7f64980c5630 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f648c0097a0 tx=0x7f648c02a060 comp rx=0 tx=0).stop 2024-04-25T00:00:52.199 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.197+0000 7f649f110640 1 -- 172.21.15.157:0/2511582662 shutdown_connections 2024-04-25T00:00:52.199 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.197+0000 7f649f110640 1 --2- 172.21.15.157:0/2511582662 >> v2:172.21.15.169:3300/0 conn(0x7f64980be360 0x7f6498155350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:52.199 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.197+0000 7f649f110640 1 --2- 172.21.15.157:0/2511582662 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6498152b40 0x7f64980c5630 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:52.199 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.197+0000 7f649f110640 1 -- 172.21.15.157:0/2511582662 >> 172.21.15.157:0/2511582662 conn(0x7f64980672c0 msgr2=0x7f64980676d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:52.199 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.197+0000 7f649f110640 1 -- 172.21.15.157:0/2511582662 shutdown_connections 2024-04-25T00:00:52.199 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.197+0000 7f649f110640 1 -- 172.21.15.157:0/2511582662 wait complete. 2024-04-25T00:00:52.199 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.198+0000 7f649f110640 1 Processor -- start 2024-04-25T00:00:52.199 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.199+0000 7f649f110640 1 -- start start 2024-04-25T00:00:52.202 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.200+0000 7f649f110640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64980be360 0x7f6498147a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:52.202 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.200+0000 7f649f110640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64980c5b70 0x7f64980ab7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:52.203 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.200+0000 7f649f110640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6498152b40 0x7f64981ee640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:52.203 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.200+0000 7f649f110640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f64980caa00 con 0x7f6498152b40 2024-04-25T00:00:52.203 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.200+0000 7f649f110640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f64980cb210 con 0x7f64980be360 2024-04-25T00:00:52.203 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.200+0000 7f649f110640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f64980cba40 con 0x7f64980c5b70 2024-04-25T00:00:52.203 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.200+0000 7f649d6ad640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6498152b40 0x7f64981ee640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:52.203 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.200+0000 7f649d6ad640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6498152b40 0x7f64981ee640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51992/0 (socket says 172.21.15.157:51992) 2024-04-25T00:00:52.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.200+0000 7f649d6ad640 1 -- 172.21.15.157:0/1063276416 learned_addr learned my addr 172.21.15.157:0/1063276416 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:00:52.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.200+0000 7f649ceac640 1 --2- 172.21.15.157:0/1063276416 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64980be360 0x7f6498147a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:52.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.200+0000 7f6497fff640 1 --2- 172.21.15.157:0/1063276416 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64980c5b70 0x7f64980ab7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:52.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.200+0000 7f649d6ad640 1 -- 172.21.15.157:0/1063276416 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64980c5b70 msgr2=0x7f64980ab7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:52.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.200+0000 7f649d6ad640 1 --2- 172.21.15.157:0/1063276416 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64980c5b70 0x7f64980ab7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:52.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.200+0000 7f649d6ad640 1 -- 172.21.15.157:0/1063276416 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64980be360 msgr2=0x7f6498147a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:52.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.200+0000 7f649d6ad640 1 --2- 172.21.15.157:0/1063276416 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64980be360 0x7f6498147a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:52.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.200+0000 7f649d6ad640 1 -- 172.21.15.157:0/1063276416 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64980aa760 con 0x7f6498152b40 2024-04-25T00:00:52.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.200+0000 7f649d6ad640 1 --2- 172.21.15.157:0/1063276416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6498152b40 0x7f64981ee640 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f649000cb40 tx=0x7f649000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:52.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.200+0000 7f64957fa640 1 -- 172.21.15.157:0/1063276416 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6490017070 con 0x7f6498152b40 2024-04-25T00:00:52.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.200+0000 7f64957fa640 1 -- 172.21.15.157:0/1063276416 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f649000aca0 con 0x7f6498152b40 2024-04-25T00:00:52.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.200+0000 7f649f110640 1 -- 172.21.15.157:0/1063276416 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64980aaa40 con 0x7f6498152b40 2024-04-25T00:00:52.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.200+0000 7f64957fa640 1 -- 172.21.15.157:0/1063276416 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6490012620 con 0x7f6498152b40 2024-04-25T00:00:52.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.201+0000 7f649f110640 1 -- 172.21.15.157:0/1063276416 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64980ab000 con 0x7f6498152b40 2024-04-25T00:00:52.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.202+0000 7f649f110640 1 -- 172.21.15.157:0/1063276416 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64980c18a0 con 0x7f6498152b40 2024-04-25T00:00:52.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.202+0000 7f64957fa640 1 -- 172.21.15.157:0/1063276416 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f649001a070 con 0x7f6498152b40 2024-04-25T00:00:52.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.202+0000 7f64957fa640 1 --2- 172.21.15.157:0/1063276416 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f647c06be30 0x7f647c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:52.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.202+0000 7f64957fa640 1 -- 172.21.15.157:0/1063276416 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f649008af60 con 0x7f6498152b40 2024-04-25T00:00:52.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.203+0000 7f649ceac640 1 --2- 172.21.15.157:0/1063276416 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f647c06be30 0x7f647c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:52.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.203+0000 7f649ceac640 1 --2- 172.21.15.157:0/1063276416 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f647c06be30 0x7f647c06e2f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f648c034000 tx=0x7f648c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:52.206 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.207+0000 7f64957fa640 1 -- 172.21.15.157:0/1063276416 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6490058f50 con 0x7f6498152b40 2024-04-25T00:00:52.386 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.386+0000 7f649f110640 1 -- 172.21.15.157:0/1063276416 --> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f649806d3f0 con 0x7f647c06be30 2024-04-25T00:00:52.400 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.401+0000 7f64957fa640 1 -- 172.21.15.157:0/1063276416 <== mgr.4105 v2:172.21.15.169:6832/662378289 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+336477 (secure 0 0 0) 0x7f649806d3f0 con 0x7f647c06be30 2024-04-25T00:00:52.404 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-25T00:00:52.406 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2024-04-25T00:00:52.408 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.405+0000 7f649f110640 1 -- 172.21.15.157:0/1063276416 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f647c06be30 msgr2=0x7f647c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:52.408 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.405+0000 7f649f110640 1 --2- 172.21.15.157:0/1063276416 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f647c06be30 0x7f647c06e2f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f648c034000 tx=0x7f648c02e000 comp rx=0 tx=0).stop 2024-04-25T00:00:52.408 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.405+0000 7f649f110640 1 -- 172.21.15.157:0/1063276416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6498152b40 msgr2=0x7f64981ee640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:52.408 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.405+0000 7f649f110640 1 --2- 172.21.15.157:0/1063276416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6498152b40 0x7f64981ee640 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f649000cb40 tx=0x7f649000a060 comp rx=0 tx=0).stop 2024-04-25T00:00:52.408 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.405+0000 7f649f110640 1 -- 172.21.15.157:0/1063276416 shutdown_connections 2024-04-25T00:00:52.408 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.405+0000 7f649f110640 1 --2- 172.21.15.157:0/1063276416 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f647c06be30 0x7f647c06e2f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:52.408 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.405+0000 7f649f110640 1 --2- 172.21.15.157:0/1063276416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6498152b40 0x7f64981ee640 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:52.408 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.405+0000 7f649f110640 1 --2- 172.21.15.157:0/1063276416 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64980c5b70 0x7f64980ab7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:52.408 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.405+0000 7f649f110640 1 --2- 172.21.15.157:0/1063276416 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64980be360 0x7f6498147a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:52.408 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.405+0000 7f649f110640 1 -- 172.21.15.157:0/1063276416 >> 172.21.15.157:0/1063276416 conn(0x7f64980672c0 msgr2=0x7f649814ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:52.408 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.405+0000 7f649f110640 1 -- 172.21.15.157:0/1063276416 shutdown_connections 2024-04-25T00:00:52.409 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.405+0000 7f649f110640 1 -- 172.21.15.157:0/1063276416 wait complete. 2024-04-25T00:00:52.416 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":803,"stamp":"2024-04-25T00:00:52.173103+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":242685187,"num_objects":169,"num_object_clones":0,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":169,"num_whiteouts":0,"num_read":15578,"num_read_kb":3146093,"num_write":423823,"num_write_kb":32349121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":588545,"ondisk_log_size":588545,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":7876448,"kb_used_data":293600,"kb_used_omap":12,"kb_used_meta":7582771,"kb_avail":747098272,"statfs":{"total":773094113280,"available":765028630528,"internally_reserved":0,"allocated":300646400,"data_stored":551555133,"data_compressed":59320411,"data_compressed_allocated":88268800,"data_compressed_original":340598784,"omap_allocated":12752,"internal_metadata":7764758064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,9,11,15,19,19,14,10,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-227078932,"num_objects":-1406,"num_object_clones":0,"num_object_copies":-4328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1406,"num_whiteouts":0,"num_read":106,"num_read_kb":22333,"num_write":4745,"num_write_kb":88285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-113,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008314"},"pg_stats":[{"pgid":"2.1f","version":"27'2678","reported_seq":2716,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.212495+0000","last_change":"2024-04-24T23:34:53.362409+0000","last_active":"2024-04-25T00:00:47.212495+0000","last_peered":"2024-04-25T00:00:47.212495+0000","last_clean":"2024-04-25T00:00:47.212495+0000","last_became_active":"2024-04-24T23:34:53.362104+0000","last_became_peered":"2024-04-24T23:34:53.362104+0000","last_unstale":"2024-04-25T00:00:47.212495+0000","last_undegraded":"2024-04-25T00:00:47.212495+0000","last_fullsized":"2024-04-25T00:00:47.212495+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":2678,"log_dups_size":0,"ondisk_log_size":2678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:53.159219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":2687,"num_write_kb":18218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"27'2551","reported_seq":2587,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.968603+0000","last_change":"2024-04-24T23:34:54.373560+0000","last_active":"2024-04-25T00:00:50.968603+0000","last_peered":"2024-04-25T00:00:50.968603+0000","last_clean":"2024-04-25T00:00:50.968603+0000","last_became_active":"2024-04-24T23:34:54.373289+0000","last_became_peered":"2024-04-24T23:34:54.373289+0000","last_unstale":"2024-04-25T00:00:50.968603+0000","last_undegraded":"2024-04-25T00:00:50.968603+0000","last_fullsized":"2024-04-25T00:00:50.968603+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2551,"log_dups_size":0,"ondisk_log_size":2551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:31.447899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"27'9462","reported_seq":9809,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:49.915720+0000","last_change":"2024-04-24T23:34:58.730165+0000","last_active":"2024-04-25T00:00:49.915720+0000","last_peered":"2024-04-25T00:00:49.915720+0000","last_clean":"2024-04-25T00:00:49.915720+0000","last_became_active":"2024-04-24T23:34:58.729816+0000","last_became_peered":"2024-04-24T23:34:58.729816+0000","last_unstale":"2024-04-25T00:00:49.915720+0000","last_undegraded":"2024-04-25T00:00:49.915720+0000","last_fullsized":"2024-04-25T00:00:49.915720+0000","mapping_epoch":23,"log_start":"27'3200","ondisk_log_start":"27'3200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":6262,"log_dups_size":0,"ondisk_log_size":6262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:02:02.335135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":319,"num_read_kb":81026,"num_write":6761,"num_write_kb":1031213,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"27'6592","reported_seq":6630,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.215933+0000","last_change":"2024-04-24T23:34:53.361650+0000","last_active":"2024-04-25T00:00:47.215933+0000","last_peered":"2024-04-25T00:00:47.215933+0000","last_clean":"2024-04-25T00:00:47.215933+0000","last_became_active":"2024-04-24T23:34:53.361366+0000","last_became_peered":"2024-04-24T23:34:53.361366+0000","last_unstale":"2024-04-25T00:00:47.215933+0000","last_undegraded":"2024-04-25T00:00:47.215933+0000","last_fullsized":"2024-04-25T00:00:47.215933+0000","mapping_epoch":18,"log_start":"27'300","ondisk_log_start":"27'300","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":6292,"log_dups_size":0,"ondisk_log_size":6292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:56:15.784223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6593,"num_write_kb":53610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"27'2521","reported_seq":2557,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:49.128653+0000","last_change":"2024-04-24T23:34:54.383602+0000","last_active":"2024-04-25T00:00:49.128653+0000","last_peered":"2024-04-25T00:00:49.128653+0000","last_clean":"2024-04-25T00:00:49.128653+0000","last_became_active":"2024-04-24T23:34:54.383160+0000","last_became_peered":"2024-04-24T23:34:54.383160+0000","last_unstale":"2024-04-25T00:00:49.128653+0000","last_undegraded":"2024-04-25T00:00:49.128653+0000","last_fullsized":"2024-04-25T00:00:49.128653+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2521,"log_dups_size":0,"ondisk_log_size":2521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:15.477061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"27'9228","reported_seq":9425,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.815605+0000","last_change":"2024-04-24T23:34:58.404416+0000","last_active":"2024-04-25T00:00:50.815605+0000","last_peered":"2024-04-25T00:00:50.815605+0000","last_clean":"2024-04-25T00:00:50.815605+0000","last_became_active":"2024-04-24T23:34:58.403984+0000","last_became_peered":"2024-04-24T23:34:58.403984+0000","last_unstale":"2024-04-25T00:00:50.815605+0000","last_undegraded":"2024-04-25T00:00:50.815605+0000","last_fullsized":"2024-04-25T00:00:50.815605+0000","mapping_epoch":23,"log_start":"27'2060","ondisk_log_start":"27'2060","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7168,"log_dups_size":0,"ondisk_log_size":7168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:46.237082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":169,"num_read_kb":42926,"num_write":6485,"num_write_kb":1085292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"27'10053","reported_seq":10208,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:49.138658+0000","last_change":"2024-04-24T23:34:53.366730+0000","last_active":"2024-04-25T00:00:49.138658+0000","last_peered":"2024-04-25T00:00:49.138658+0000","last_clean":"2024-04-25T00:00:49.138658+0000","last_became_active":"2024-04-24T23:34:53.366430+0000","last_became_peered":"2024-04-24T23:34:53.366430+0000","last_unstale":"2024-04-25T00:00:49.138658+0000","last_undegraded":"2024-04-25T00:00:49.138658+0000","last_fullsized":"2024-04-25T00:00:49.138658+0000","mapping_epoch":18,"log_start":"27'3200","ondisk_log_start":"27'3200","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":6853,"log_dups_size":0,"ondisk_log_size":6853,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:41.422591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5022938,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":120,"num_read_kb":893,"num_write":10045,"num_write_kb":77054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"27'2616","reported_seq":2650,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.280419+0000","last_change":"2024-04-24T23:34:54.373592+0000","last_active":"2024-04-25T00:00:47.280419+0000","last_peered":"2024-04-25T00:00:47.280419+0000","last_clean":"2024-04-25T00:00:47.280419+0000","last_became_active":"2024-04-24T23:34:54.372844+0000","last_became_peered":"2024-04-24T23:34:54.372844+0000","last_unstale":"2024-04-25T00:00:47.280419+0000","last_undegraded":"2024-04-25T00:00:47.280419+0000","last_fullsized":"2024-04-25T00:00:47.280419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2616,"log_dups_size":0,"ondisk_log_size":2616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:49.372040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"27'9403","reported_seq":10019,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.785455+0000","last_change":"2024-04-24T23:34:58.399224+0000","last_active":"2024-04-25T00:00:50.785455+0000","last_peered":"2024-04-25T00:00:50.785455+0000","last_clean":"2024-04-25T00:00:50.785455+0000","last_became_active":"2024-04-24T23:34:58.397817+0000","last_became_peered":"2024-04-24T23:34:58.397817+0000","last_unstale":"2024-04-25T00:00:50.785455+0000","last_undegraded":"2024-04-25T00:00:50.785455+0000","last_fullsized":"2024-04-25T00:00:50.785455+0000","mapping_epoch":23,"log_start":"27'2212","ondisk_log_start":"27'2212","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7191,"log_dups_size":0,"ondisk_log_size":7191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:53.085366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":588,"num_read_kb":149352,"num_write":6710,"num_write_kb":891395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"27'5158","reported_seq":5193,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.195334+0000","last_change":"2024-04-24T23:34:53.362994+0000","last_active":"2024-04-25T00:00:47.195334+0000","last_peered":"2024-04-25T00:00:47.195334+0000","last_clean":"2024-04-25T00:00:47.195334+0000","last_became_active":"2024-04-24T23:34:53.362337+0000","last_became_peered":"2024-04-24T23:34:53.362337+0000","last_unstale":"2024-04-25T00:00:47.195334+0000","last_undegraded":"2024-04-25T00:00:47.195334+0000","last_fullsized":"2024-04-25T00:00:47.195334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":5158,"log_dups_size":0,"ondisk_log_size":5158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:12.258211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":5165,"num_write_kb":44324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"27'2532","reported_seq":2566,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.851337+0000","last_change":"2024-04-24T23:34:54.379218+0000","last_active":"2024-04-25T00:00:47.851337+0000","last_peered":"2024-04-25T00:00:47.851337+0000","last_clean":"2024-04-25T00:00:47.851337+0000","last_became_active":"2024-04-24T23:34:54.376019+0000","last_became_peered":"2024-04-24T23:34:54.376019+0000","last_unstale":"2024-04-25T00:00:47.851337+0000","last_undegraded":"2024-04-25T00:00:47.851337+0000","last_fullsized":"2024-04-25T00:00:47.851337+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2532,"log_dups_size":0,"ondisk_log_size":2532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:41:48.207542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"27'8765","reported_seq":9265,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.714347+0000","last_change":"2024-04-24T23:34:58.397152+0000","last_active":"2024-04-25T00:00:50.714347+0000","last_peered":"2024-04-25T00:00:50.714347+0000","last_clean":"2024-04-25T00:00:50.714347+0000","last_became_active":"2024-04-24T23:34:58.396658+0000","last_became_peered":"2024-04-24T23:34:58.396658+0000","last_unstale":"2024-04-25T00:00:50.714347+0000","last_undegraded":"2024-04-25T00:00:50.714347+0000","last_fullsized":"2024-04-25T00:00:50.714347+0000","mapping_epoch":23,"log_start":"27'1609","ondisk_log_start":"27'1609","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7156,"log_dups_size":0,"ondisk_log_size":7156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:41.360776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":472,"num_read_kb":119888,"num_write":5982,"num_write_kb":849049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"27'2561","reported_seq":2596,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.975199+0000","last_change":"2024-04-24T23:34:54.377859+0000","last_active":"2024-04-25T00:00:50.975199+0000","last_peered":"2024-04-25T00:00:50.975199+0000","last_clean":"2024-04-25T00:00:50.975199+0000","last_became_active":"2024-04-24T23:34:54.377576+0000","last_became_peered":"2024-04-24T23:34:54.377576+0000","last_unstale":"2024-04-25T00:00:50.975199+0000","last_undegraded":"2024-04-25T00:00:50.975199+0000","last_fullsized":"2024-04-25T00:00:50.975199+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2561,"log_dups_size":0,"ondisk_log_size":2561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:47.446459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"27'6834","reported_seq":6870,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.226938+0000","last_change":"2024-04-24T23:34:53.360903+0000","last_active":"2024-04-25T00:00:47.226938+0000","last_peered":"2024-04-25T00:00:47.226938+0000","last_clean":"2024-04-25T00:00:47.226938+0000","last_became_active":"2024-04-24T23:34:53.360635+0000","last_became_peered":"2024-04-24T23:34:53.360635+0000","last_unstale":"2024-04-25T00:00:47.226938+0000","last_undegraded":"2024-04-25T00:00:47.226938+0000","last_fullsized":"2024-04-25T00:00:47.226938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":6834,"log_dups_size":0,"ondisk_log_size":6834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:42.205997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6840,"num_write_kb":53782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"27'10213","reported_seq":10495,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.805890+0000","last_change":"2024-04-24T23:34:58.407146+0000","last_active":"2024-04-25T00:00:50.805890+0000","last_peered":"2024-04-25T00:00:50.805890+0000","last_clean":"2024-04-25T00:00:50.805890+0000","last_became_active":"2024-04-24T23:34:58.406844+0000","last_became_peered":"2024-04-24T23:34:58.406844+0000","last_unstale":"2024-04-25T00:00:50.805890+0000","last_undegraded":"2024-04-25T00:00:50.805890+0000","last_fullsized":"2024-04-25T00:00:50.805890+0000","mapping_epoch":23,"log_start":"27'3041","ondisk_log_start":"27'3041","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7172,"log_dups_size":0,"ondisk_log_size":7172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:41:52.244767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":253,"num_read_kb":64262,"num_write":7465,"num_write_kb":1030177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"27'2579","reported_seq":2614,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.977518+0000","last_change":"2024-04-24T23:34:54.366724+0000","last_active":"2024-04-25T00:00:50.977518+0000","last_peered":"2024-04-25T00:00:50.977518+0000","last_clean":"2024-04-25T00:00:50.977518+0000","last_became_active":"2024-04-24T23:34:54.366426+0000","last_became_peered":"2024-04-24T23:34:54.366426+0000","last_unstale":"2024-04-25T00:00:50.977518+0000","last_undegraded":"2024-04-25T00:00:50.977518+0000","last_fullsized":"2024-04-25T00:00:50.977518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2579,"log_dups_size":0,"ondisk_log_size":2579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:47.035356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"27'7913","reported_seq":7950,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.199526+0000","last_change":"2024-04-24T23:34:53.359140+0000","last_active":"2024-04-25T00:00:47.199526+0000","last_peered":"2024-04-25T00:00:47.199526+0000","last_clean":"2024-04-25T00:00:47.199526+0000","last_became_active":"2024-04-24T23:34:53.358841+0000","last_became_peered":"2024-04-24T23:34:53.358841+0000","last_unstale":"2024-04-25T00:00:47.199526+0000","last_undegraded":"2024-04-25T00:00:47.199526+0000","last_fullsized":"2024-04-25T00:00:47.199526+0000","mapping_epoch":18,"log_start":"27'500","ondisk_log_start":"27'500","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":7413,"log_dups_size":0,"ondisk_log_size":7413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:41.422591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7912,"num_write_kb":58475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"27'9714","reported_seq":10146,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.696313+0000","last_change":"2024-04-24T23:34:58.400128+0000","last_active":"2024-04-25T00:00:50.696313+0000","last_peered":"2024-04-25T00:00:50.696313+0000","last_clean":"2024-04-25T00:00:50.696313+0000","last_became_active":"2024-04-24T23:34:58.399375+0000","last_became_peered":"2024-04-24T23:34:58.399375+0000","last_unstale":"2024-04-25T00:00:50.696313+0000","last_undegraded":"2024-04-25T00:00:50.696313+0000","last_fullsized":"2024-04-25T00:00:50.696313+0000","mapping_epoch":23,"log_start":"27'2556","ondisk_log_start":"27'2556","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7158,"log_dups_size":0,"ondisk_log_size":7158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:53.409265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":403,"num_read_kb":102362,"num_write":6994,"num_write_kb":969161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"27'2649","reported_seq":2684,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.992088+0000","last_change":"2024-04-24T23:34:54.365012+0000","last_active":"2024-04-25T00:00:50.992088+0000","last_peered":"2024-04-25T00:00:50.992088+0000","last_clean":"2024-04-25T00:00:50.992088+0000","last_became_active":"2024-04-24T23:34:54.364744+0000","last_became_peered":"2024-04-24T23:34:54.364744+0000","last_unstale":"2024-04-25T00:00:50.992088+0000","last_undegraded":"2024-04-25T00:00:50.992088+0000","last_fullsized":"2024-04-25T00:00:50.992088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2649,"log_dups_size":0,"ondisk_log_size":2649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:35.774122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"27'4022","reported_seq":4059,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.118666+0000","last_change":"2024-04-24T23:34:53.360843+0000","last_active":"2024-04-25T00:00:47.118666+0000","last_peered":"2024-04-25T00:00:47.118666+0000","last_clean":"2024-04-25T00:00:47.118666+0000","last_became_active":"2024-04-24T23:34:53.360570+0000","last_became_peered":"2024-04-24T23:34:53.360570+0000","last_unstale":"2024-04-25T00:00:47.118666+0000","last_undegraded":"2024-04-25T00:00:47.118666+0000","last_fullsized":"2024-04-25T00:00:47.118666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":4022,"log_dups_size":0,"ondisk_log_size":4022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:21.089467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4022,"num_write_kb":27320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"27'9656","reported_seq":10077,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.006967+0000","last_change":"2024-04-24T23:34:58.731835+0000","last_active":"2024-04-25T00:00:50.006967+0000","last_peered":"2024-04-25T00:00:50.006967+0000","last_clean":"2024-04-25T00:00:50.006967+0000","last_became_active":"2024-04-24T23:34:58.731485+0000","last_became_peered":"2024-04-24T23:34:58.731485+0000","last_unstale":"2024-04-25T00:00:50.006967+0000","last_undegraded":"2024-04-25T00:00:50.006967+0000","last_fullsized":"2024-04-25T00:00:50.006967+0000","mapping_epoch":23,"log_start":"27'2316","ondisk_log_start":"27'2316","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7340,"log_dups_size":0,"ondisk_log_size":7340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:56.609480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":392,"num_read_kb":99568,"num_write":7039,"num_write_kb":904216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"27'2554","reported_seq":2589,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:51.027913+0000","last_change":"2024-04-24T23:34:54.362447+0000","last_active":"2024-04-25T00:00:51.027913+0000","last_peered":"2024-04-25T00:00:51.027913+0000","last_clean":"2024-04-25T00:00:51.027913+0000","last_became_active":"2024-04-24T23:34:54.362141+0000","last_became_peered":"2024-04-24T23:34:54.362141+0000","last_unstale":"2024-04-25T00:00:51.027913+0000","last_undegraded":"2024-04-25T00:00:51.027913+0000","last_fullsized":"2024-04-25T00:00:51.027913+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2554,"log_dups_size":0,"ondisk_log_size":2554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:51.227333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"27'2570","reported_seq":2606,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.182556+0000","last_change":"2024-04-24T23:34:53.366452+0000","last_active":"2024-04-25T00:00:47.182556+0000","last_peered":"2024-04-25T00:00:47.182556+0000","last_clean":"2024-04-25T00:00:47.182556+0000","last_became_active":"2024-04-24T23:34:53.366110+0000","last_became_peered":"2024-04-24T23:34:53.366110+0000","last_unstale":"2024-04-25T00:00:47.182556+0000","last_undegraded":"2024-04-25T00:00:47.182556+0000","last_fullsized":"2024-04-25T00:00:47.182556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":2570,"log_dups_size":0,"ondisk_log_size":2570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:14.716741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2576,"num_write_kb":22186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"27'9201","reported_seq":9381,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.195641+0000","last_change":"2024-04-24T23:34:58.402849+0000","last_active":"2024-04-25T00:00:50.195641+0000","last_peered":"2024-04-25T00:00:50.195641+0000","last_clean":"2024-04-25T00:00:50.195641+0000","last_became_active":"2024-04-24T23:34:58.402365+0000","last_became_peered":"2024-04-24T23:34:58.402365+0000","last_unstale":"2024-04-25T00:00:50.195641+0000","last_undegraded":"2024-04-25T00:00:50.195641+0000","last_fullsized":"2024-04-25T00:00:50.195641+0000","mapping_epoch":23,"log_start":"27'2049","ondisk_log_start":"27'2049","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7152,"log_dups_size":0,"ondisk_log_size":7152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:07:32.822427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":151,"num_read_kb":38354,"num_write":6553,"num_write_kb":977677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"27'2514","reported_seq":2549,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:49.836629+0000","last_change":"2024-04-24T23:34:54.370300+0000","last_active":"2024-04-25T00:00:49.836629+0000","last_peered":"2024-04-25T00:00:49.836629+0000","last_clean":"2024-04-25T00:00:49.836629+0000","last_became_active":"2024-04-24T23:34:54.369835+0000","last_became_peered":"2024-04-24T23:34:54.369835+0000","last_unstale":"2024-04-25T00:00:49.836629+0000","last_undegraded":"2024-04-25T00:00:49.836629+0000","last_fullsized":"2024-04-25T00:00:49.836629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2514,"log_dups_size":0,"ondisk_log_size":2514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:42.199856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"27'4484","reported_seq":4522,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.180061+0000","last_change":"2024-04-24T23:34:53.366110+0000","last_active":"2024-04-25T00:00:47.180061+0000","last_peered":"2024-04-25T00:00:47.180061+0000","last_clean":"2024-04-25T00:00:47.180061+0000","last_became_active":"2024-04-24T23:34:53.365783+0000","last_became_peered":"2024-04-24T23:34:53.365783+0000","last_unstale":"2024-04-25T00:00:47.180061+0000","last_undegraded":"2024-04-25T00:00:47.180061+0000","last_fullsized":"2024-04-25T00:00:47.180061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":4484,"log_dups_size":0,"ondisk_log_size":4484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:41.422591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":4494,"num_write_kb":39872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"27'9221","reported_seq":9441,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:49.970638+0000","last_change":"2024-04-24T23:34:58.728191+0000","last_active":"2024-04-25T00:00:49.970638+0000","last_peered":"2024-04-25T00:00:49.970638+0000","last_clean":"2024-04-25T00:00:49.970638+0000","last_became_active":"2024-04-24T23:34:58.727653+0000","last_became_peered":"2024-04-24T23:34:58.727653+0000","last_unstale":"2024-04-25T00:00:49.970638+0000","last_undegraded":"2024-04-25T00:00:49.970638+0000","last_fullsized":"2024-04-25T00:00:49.970638+0000","mapping_epoch":23,"log_start":"27'2006","ondisk_log_start":"27'2006","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7215,"log_dups_size":0,"ondisk_log_size":7215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:30.299011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":190,"num_read_kb":48260,"num_write":6573,"num_write_kb":840416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"27'2508","reported_seq":2543,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:51.015541+0000","last_change":"2024-04-24T23:34:54.368278+0000","last_active":"2024-04-25T00:00:51.015541+0000","last_peered":"2024-04-25T00:00:51.015541+0000","last_clean":"2024-04-25T00:00:51.015541+0000","last_became_active":"2024-04-24T23:34:54.367991+0000","last_became_peered":"2024-04-24T23:34:54.367991+0000","last_unstale":"2024-04-25T00:00:51.015541+0000","last_undegraded":"2024-04-25T00:00:51.015541+0000","last_fullsized":"2024-04-25T00:00:51.015541+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2508,"log_dups_size":0,"ondisk_log_size":2508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:49:31.018719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"27'7474","reported_seq":8137,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:48.653456+0000","last_change":"2024-04-24T23:34:53.364189+0000","last_active":"2024-04-25T00:00:48.653456+0000","last_peered":"2024-04-25T00:00:48.653456+0000","last_clean":"2024-04-25T00:00:48.653456+0000","last_became_active":"2024-04-24T23:34:53.363842+0000","last_became_peered":"2024-04-24T23:34:53.363842+0000","last_unstale":"2024-04-25T00:00:48.653456+0000","last_undegraded":"2024-04-25T00:00:48.653456+0000","last_fullsized":"2024-04-25T00:00:48.653456+0000","mapping_epoch":18,"log_start":"27'100","ondisk_log_start":"27'100","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":7374,"log_dups_size":0,"ondisk_log_size":7374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:12.258211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":626,"num_read_kb":4509,"num_write":10319,"num_write_kb":61532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"27'8920","reported_seq":9228,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:49.631097+0000","last_change":"2024-04-24T23:34:58.729450+0000","last_active":"2024-04-25T00:00:49.631097+0000","last_peered":"2024-04-25T00:00:49.631097+0000","last_clean":"2024-04-25T00:00:49.631097+0000","last_became_active":"2024-04-24T23:34:58.729082+0000","last_became_peered":"2024-04-24T23:34:58.729082+0000","last_unstale":"2024-04-25T00:00:49.631097+0000","last_undegraded":"2024-04-25T00:00:49.631097+0000","last_fullsized":"2024-04-25T00:00:49.631097+0000","mapping_epoch":23,"log_start":"27'2609","ondisk_log_start":"27'2609","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":6311,"log_dups_size":0,"ondisk_log_size":6311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:45.372053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":278,"num_read_kb":70612,"num_write":6238,"num_write_kb":899702,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"27'2592","reported_seq":2628,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.255599+0000","last_change":"2024-04-24T23:34:54.357246+0000","last_active":"2024-04-25T00:00:50.255599+0000","last_peered":"2024-04-25T00:00:50.255599+0000","last_clean":"2024-04-25T00:00:50.255599+0000","last_became_active":"2024-04-24T23:34:54.356560+0000","last_became_peered":"2024-04-24T23:34:54.356560+0000","last_unstale":"2024-04-25T00:00:50.255599+0000","last_undegraded":"2024-04-25T00:00:50.255599+0000","last_fullsized":"2024-04-25T00:00:50.255599+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2592,"log_dups_size":0,"ondisk_log_size":2592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:42.199856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"27'6261","reported_seq":6298,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.213097+0000","last_change":"2024-04-24T23:34:53.366296+0000","last_active":"2024-04-25T00:00:47.213097+0000","last_peered":"2024-04-25T00:00:47.213097+0000","last_clean":"2024-04-25T00:00:47.213097+0000","last_became_active":"2024-04-24T23:34:53.365636+0000","last_became_peered":"2024-04-24T23:34:53.365636+0000","last_unstale":"2024-04-25T00:00:47.213097+0000","last_undegraded":"2024-04-25T00:00:47.213097+0000","last_fullsized":"2024-04-25T00:00:47.213097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":6261,"log_dups_size":0,"ondisk_log_size":6261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:50.222015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":6274,"num_write_kb":49208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"27'9152","reported_seq":9686,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:48.941905+0000","last_change":"2024-04-24T23:34:58.728271+0000","last_active":"2024-04-25T00:00:48.941905+0000","last_peered":"2024-04-25T00:00:48.941905+0000","last_clean":"2024-04-25T00:00:48.941905+0000","last_became_active":"2024-04-24T23:34:58.727885+0000","last_became_peered":"2024-04-24T23:34:58.727885+0000","last_unstale":"2024-04-25T00:00:48.941905+0000","last_undegraded":"2024-04-25T00:00:48.941905+0000","last_fullsized":"2024-04-25T00:00:48.941905+0000","mapping_epoch":23,"log_start":"27'1968","ondisk_log_start":"27'1968","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7184,"log_dups_size":0,"ondisk_log_size":7184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:49.461271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":504,"num_read_kb":128016,"num_write":6457,"num_write_kb":842362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"27'2479","reported_seq":2515,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.782154+0000","last_change":"2024-04-24T23:34:54.372245+0000","last_active":"2024-04-25T00:00:47.782154+0000","last_peered":"2024-04-25T00:00:47.782154+0000","last_clean":"2024-04-25T00:00:47.782154+0000","last_became_active":"2024-04-24T23:34:54.371947+0000","last_became_peered":"2024-04-24T23:34:54.371947+0000","last_unstale":"2024-04-25T00:00:47.782154+0000","last_undegraded":"2024-04-25T00:00:47.782154+0000","last_fullsized":"2024-04-25T00:00:47.782154+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2479,"log_dups_size":0,"ondisk_log_size":2479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:47:52.460508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"27'7223","reported_seq":7261,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:48.404668+0000","last_change":"2024-04-24T23:34:53.360352+0000","last_active":"2024-04-25T00:00:48.404668+0000","last_peered":"2024-04-25T00:00:48.404668+0000","last_clean":"2024-04-25T00:00:48.404668+0000","last_became_active":"2024-04-24T23:34:53.360082+0000","last_became_peered":"2024-04-24T23:34:53.360082+0000","last_unstale":"2024-04-25T00:00:48.404668+0000","last_undegraded":"2024-04-25T00:00:48.404668+0000","last_fullsized":"2024-04-25T00:00:48.404668+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":7223,"log_dups_size":0,"ondisk_log_size":7223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:50.222015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17473484,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":7221,"num_write_kb":70842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"27'9196","reported_seq":9381,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.407169+0000","last_change":"2024-04-24T23:34:58.404176+0000","last_active":"2024-04-25T00:00:50.407169+0000","last_peered":"2024-04-25T00:00:50.407169+0000","last_clean":"2024-04-25T00:00:50.407169+0000","last_became_active":"2024-04-24T23:34:58.403735+0000","last_became_peered":"2024-04-24T23:34:58.403735+0000","last_unstale":"2024-04-25T00:00:50.407169+0000","last_undegraded":"2024-04-25T00:00:50.407169+0000","last_fullsized":"2024-04-25T00:00:50.407169+0000","mapping_epoch":23,"log_start":"27'1819","ondisk_log_start":"27'1819","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7377,"log_dups_size":0,"ondisk_log_size":7377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:49.461271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":288778,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":155,"num_read_kb":39370,"num_write":6528,"num_write_kb":924891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"27'2542","reported_seq":2578,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:51.030774+0000","last_change":"2024-04-24T23:34:54.367638+0000","last_active":"2024-04-25T00:00:51.030774+0000","last_peered":"2024-04-25T00:00:51.030774+0000","last_clean":"2024-04-25T00:00:51.030774+0000","last_became_active":"2024-04-24T23:34:54.367355+0000","last_became_peered":"2024-04-24T23:34:54.367355+0000","last_unstale":"2024-04-25T00:00:51.030774+0000","last_undegraded":"2024-04-25T00:00:51.030774+0000","last_fullsized":"2024-04-25T00:00:51.030774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2542,"log_dups_size":0,"ondisk_log_size":2542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:15.722059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"27'8389","reported_seq":8427,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.171711+0000","last_change":"2024-04-24T23:34:53.356111+0000","last_active":"2024-04-25T00:00:47.171711+0000","last_peered":"2024-04-25T00:00:47.171711+0000","last_clean":"2024-04-25T00:00:47.171711+0000","last_became_active":"2024-04-24T23:34:53.355710+0000","last_became_peered":"2024-04-24T23:34:53.355710+0000","last_unstale":"2024-04-25T00:00:47.171711+0000","last_undegraded":"2024-04-25T00:00:47.171711+0000","last_fullsized":"2024-04-25T00:00:47.171711+0000","mapping_epoch":18,"log_start":"27'600","ondisk_log_start":"27'600","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":7789,"log_dups_size":0,"ondisk_log_size":7789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:21.089467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":8401,"num_write_kb":63182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"27'9675","reported_seq":10039,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.355841+0000","last_change":"2024-04-24T23:34:58.403016+0000","last_active":"2024-04-25T00:00:50.355841+0000","last_peered":"2024-04-25T00:00:50.355841+0000","last_clean":"2024-04-25T00:00:50.355841+0000","last_became_active":"2024-04-24T23:34:58.402598+0000","last_became_peered":"2024-04-24T23:34:58.402598+0000","last_unstale":"2024-04-25T00:00:50.355841+0000","last_undegraded":"2024-04-25T00:00:50.355841+0000","last_fullsized":"2024-04-25T00:00:50.355841+0000","mapping_epoch":23,"log_start":"27'2299","ondisk_log_start":"27'2299","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7376,"log_dups_size":0,"ondisk_log_size":7376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:47:56.497734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":148804,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":334,"num_read_kb":84836,"num_write":6881,"num_write_kb":886687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"27'2544","reported_seq":2580,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:49.516117+0000","last_change":"2024-04-24T23:34:54.356601+0000","last_active":"2024-04-25T00:00:49.516117+0000","last_peered":"2024-04-25T00:00:49.516117+0000","last_clean":"2024-04-25T00:00:49.516117+0000","last_became_active":"2024-04-24T23:34:54.356326+0000","last_became_peered":"2024-04-24T23:34:54.356326+0000","last_unstale":"2024-04-25T00:00:49.516117+0000","last_undegraded":"2024-04-25T00:00:49.516117+0000","last_fullsized":"2024-04-25T00:00:49.516117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2544,"log_dups_size":0,"ondisk_log_size":2544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:47.446459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"27'7177","reported_seq":8144,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.164180+0000","last_change":"2024-04-24T23:34:53.359238+0000","last_active":"2024-04-25T00:00:47.164180+0000","last_peered":"2024-04-25T00:00:47.164180+0000","last_clean":"2024-04-25T00:00:47.164180+0000","last_became_active":"2024-04-24T23:34:53.358937+0000","last_became_peered":"2024-04-24T23:34:53.358937+0000","last_unstale":"2024-04-25T00:00:47.164180+0000","last_undegraded":"2024-04-25T00:00:47.164180+0000","last_fullsized":"2024-04-25T00:00:47.164180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":7177,"log_dups_size":0,"ondisk_log_size":7177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:50.222015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":929,"num_read_kb":4700,"num_write":7175,"num_write_kb":61982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"27'9030","reported_seq":9623,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:49.568281+0000","last_change":"2024-04-24T23:34:58.403258+0000","last_active":"2024-04-25T00:00:49.568281+0000","last_peered":"2024-04-25T00:00:49.568281+0000","last_clean":"2024-04-25T00:00:49.568281+0000","last_became_active":"2024-04-24T23:34:58.402569+0000","last_became_peered":"2024-04-24T23:34:58.402569+0000","last_unstale":"2024-04-25T00:00:49.568281+0000","last_undegraded":"2024-04-25T00:00:49.568281+0000","last_fullsized":"2024-04-25T00:00:49.568281+0000","mapping_epoch":23,"log_start":"27'1638","ondisk_log_start":"27'1638","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7392,"log_dups_size":0,"ondisk_log_size":7392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:32:08.782070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":564,"num_read_kb":143256,"num_write":6307,"num_write_kb":911438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"27'2480","reported_seq":2516,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:49.530584+0000","last_change":"2024-04-24T23:34:54.377543+0000","last_active":"2024-04-25T00:00:49.530584+0000","last_peered":"2024-04-25T00:00:49.530584+0000","last_clean":"2024-04-25T00:00:49.530584+0000","last_became_active":"2024-04-24T23:34:54.377228+0000","last_became_peered":"2024-04-24T23:34:54.377228+0000","last_unstale":"2024-04-25T00:00:49.530584+0000","last_undegraded":"2024-04-25T00:00:49.530584+0000","last_fullsized":"2024-04-25T00:00:49.530584+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2480,"log_dups_size":0,"ondisk_log_size":2480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:42.255038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"27'7206","reported_seq":7243,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.222770+0000","last_change":"2024-04-24T23:34:53.362675+0000","last_active":"2024-04-25T00:00:47.222770+0000","last_peered":"2024-04-25T00:00:47.222770+0000","last_clean":"2024-04-25T00:00:47.222770+0000","last_became_active":"2024-04-24T23:34:53.362425+0000","last_became_peered":"2024-04-24T23:34:53.362425+0000","last_unstale":"2024-04-25T00:00:47.222770+0000","last_undegraded":"2024-04-25T00:00:47.222770+0000","last_fullsized":"2024-04-25T00:00:47.222770+0000","mapping_epoch":18,"log_start":"27'300","ondisk_log_start":"27'300","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":6906,"log_dups_size":0,"ondisk_log_size":6906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:21.089467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":7208,"num_write_kb":58030,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"27'9329","reported_seq":9851,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:49.782889+0000","last_change":"2024-04-24T23:34:58.728573+0000","last_active":"2024-04-25T00:00:49.782889+0000","last_peered":"2024-04-25T00:00:49.782889+0000","last_clean":"2024-04-25T00:00:49.782889+0000","last_became_active":"2024-04-24T23:34:58.727284+0000","last_became_peered":"2024-04-24T23:34:58.727284+0000","last_unstale":"2024-04-25T00:00:49.782889+0000","last_undegraded":"2024-04-25T00:00:49.782889+0000","last_fullsized":"2024-04-25T00:00:49.782889+0000","mapping_epoch":23,"log_start":"27'2114","ondisk_log_start":"27'2114","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7215,"log_dups_size":0,"ondisk_log_size":7215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:32:08.782070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":493,"num_read_kb":125222,"num_write":6691,"num_write_kb":1110316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"27'2597","reported_seq":2633,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:51.021903+0000","last_change":"2024-04-24T23:34:54.382820+0000","last_active":"2024-04-25T00:00:51.021903+0000","last_peered":"2024-04-25T00:00:51.021903+0000","last_clean":"2024-04-25T00:00:51.021903+0000","last_became_active":"2024-04-24T23:34:54.382564+0000","last_became_peered":"2024-04-24T23:34:54.382564+0000","last_unstale":"2024-04-25T00:00:51.021903+0000","last_undegraded":"2024-04-25T00:00:51.021903+0000","last_fullsized":"2024-04-25T00:00:51.021903+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2597,"log_dups_size":0,"ondisk_log_size":2597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:45.622611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"27'5189","reported_seq":5227,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.187361+0000","last_change":"2024-04-24T23:34:53.357743+0000","last_active":"2024-04-25T00:00:47.187361+0000","last_peered":"2024-04-25T00:00:47.187361+0000","last_clean":"2024-04-25T00:00:47.187361+0000","last_became_active":"2024-04-24T23:34:53.357456+0000","last_became_peered":"2024-04-24T23:34:53.357456+0000","last_unstale":"2024-04-25T00:00:47.187361+0000","last_undegraded":"2024-04-25T00:00:47.187361+0000","last_fullsized":"2024-04-25T00:00:47.187361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":5189,"log_dups_size":0,"ondisk_log_size":5189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:57:14.381644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5191,"num_write_kb":40339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":193,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-24T23:56:58.615940+0000","last_change":"2024-04-24T23:34:49.778966+0000","last_active":"2024-04-24T23:56:58.615940+0000","last_peered":"2024-04-24T23:56:58.615940+0000","last_clean":"2024-04-24T23:56:58.615940+0000","last_became_active":"2024-04-24T23:34:49.778083+0000","last_became_peered":"2024-04-24T23:34:49.778083+0000","last_unstale":"2024-04-24T23:56:58.615940+0000","last_undegraded":"2024-04-24T23:56:58.615940+0000","last_fullsized":"2024-04-24T23:56:58.615940+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:48.587078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:48.587078+0000","last_clean_scrub_stamp":"2024-04-24T23:34:48.587078+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:46:59.471089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"27'8889","reported_seq":9406,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:49.282602+0000","last_change":"2024-04-24T23:34:58.729779+0000","last_active":"2024-04-25T00:00:49.282602+0000","last_peered":"2024-04-25T00:00:49.282602+0000","last_clean":"2024-04-25T00:00:49.282602+0000","last_became_active":"2024-04-24T23:34:58.729370+0000","last_became_peered":"2024-04-24T23:34:58.729370+0000","last_unstale":"2024-04-25T00:00:49.282602+0000","last_undegraded":"2024-04-25T00:00:49.282602+0000","last_fullsized":"2024-04-25T00:00:49.282602+0000","mapping_epoch":23,"log_start":"27'902","ondisk_log_start":"27'902","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7987,"log_dups_size":0,"ondisk_log_size":7987,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:30.299011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":487,"num_read_kb":123698,"num_write":6268,"num_write_kb":912685,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"27'2518","reported_seq":2552,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:48.969863+0000","last_change":"2024-04-24T23:34:54.384106+0000","last_active":"2024-04-25T00:00:48.969863+0000","last_peered":"2024-04-25T00:00:48.969863+0000","last_clean":"2024-04-25T00:00:48.969863+0000","last_became_active":"2024-04-24T23:34:54.383589+0000","last_became_peered":"2024-04-24T23:34:54.383589+0000","last_unstale":"2024-04-25T00:00:48.969863+0000","last_undegraded":"2024-04-25T00:00:48.969863+0000","last_fullsized":"2024-04-25T00:00:48.969863+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2518,"log_dups_size":0,"ondisk_log_size":2518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:41:48.207542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"27'8231","reported_seq":8268,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.191600+0000","last_change":"2024-04-24T23:34:53.354077+0000","last_active":"2024-04-25T00:00:47.191600+0000","last_peered":"2024-04-25T00:00:47.191600+0000","last_clean":"2024-04-25T00:00:47.191600+0000","last_became_active":"2024-04-24T23:34:53.353804+0000","last_became_peered":"2024-04-24T23:34:53.353804+0000","last_unstale":"2024-04-25T00:00:47.191600+0000","last_undegraded":"2024-04-25T00:00:47.191600+0000","last_fullsized":"2024-04-25T00:00:47.191600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":8231,"log_dups_size":0,"ondisk_log_size":8231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:41.422591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8225,"num_write_kb":58738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"27'9238","reported_seq":9425,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:48.570191+0000","last_change":"2024-04-24T23:34:58.728722+0000","last_active":"2024-04-25T00:00:48.570191+0000","last_peered":"2024-04-25T00:00:48.570191+0000","last_clean":"2024-04-25T00:00:48.570191+0000","last_became_active":"2024-04-24T23:34:58.727649+0000","last_became_peered":"2024-04-24T23:34:58.727649+0000","last_unstale":"2024-04-25T00:00:48.570191+0000","last_undegraded":"2024-04-25T00:00:48.570191+0000","last_fullsized":"2024-04-25T00:00:48.570191+0000","mapping_epoch":23,"log_start":"27'1312","ondisk_log_start":"27'1312","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7926,"log_dups_size":0,"ondisk_log_size":7926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:41:52.244767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":158,"num_read_kb":40132,"num_write":6566,"num_write_kb":960960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"27'2542","reported_seq":2577,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.151045+0000","last_change":"2024-04-24T23:34:54.373371+0000","last_active":"2024-04-25T00:00:47.151045+0000","last_peered":"2024-04-25T00:00:47.151045+0000","last_clean":"2024-04-25T00:00:47.151045+0000","last_became_active":"2024-04-24T23:34:54.372553+0000","last_became_peered":"2024-04-24T23:34:54.372553+0000","last_unstale":"2024-04-25T00:00:47.151045+0000","last_undegraded":"2024-04-25T00:00:47.151045+0000","last_fullsized":"2024-04-25T00:00:47.151045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2542,"log_dups_size":0,"ondisk_log_size":2542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:26.261786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"27'8394","reported_seq":8431,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.298609+0000","last_change":"2024-04-24T23:34:53.353505+0000","last_active":"2024-04-25T00:00:47.298609+0000","last_peered":"2024-04-25T00:00:47.298609+0000","last_clean":"2024-04-25T00:00:47.298609+0000","last_became_active":"2024-04-24T23:34:53.353186+0000","last_became_peered":"2024-04-24T23:34:53.353186+0000","last_unstale":"2024-04-25T00:00:47.298609+0000","last_undegraded":"2024-04-25T00:00:47.298609+0000","last_fullsized":"2024-04-25T00:00:47.298609+0000","mapping_epoch":18,"log_start":"27'1000","ondisk_log_start":"27'1000","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":7394,"log_dups_size":0,"ondisk_log_size":7394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:42.205997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388678,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8393,"num_write_kb":58880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"27'9582","reported_seq":9937,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:48.891190+0000","last_change":"2024-04-24T23:34:58.400988+0000","last_active":"2024-04-25T00:00:48.891190+0000","last_peered":"2024-04-25T00:00:48.891190+0000","last_clean":"2024-04-25T00:00:48.891190+0000","last_became_active":"2024-04-24T23:34:58.400549+0000","last_became_peered":"2024-04-24T23:34:58.400549+0000","last_unstale":"2024-04-25T00:00:48.891190+0000","last_undegraded":"2024-04-25T00:00:48.891190+0000","last_fullsized":"2024-04-25T00:00:48.891190+0000","mapping_epoch":23,"log_start":"27'2356","ondisk_log_start":"27'2356","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7226,"log_dups_size":0,"ondisk_log_size":7226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:47:56.497734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":326,"num_read_kb":82804,"num_write":6847,"num_write_kb":886980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"27'2418","reported_seq":2452,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:48.956305+0000","last_change":"2024-04-24T23:34:54.375234+0000","last_active":"2024-04-25T00:00:48.956305+0000","last_peered":"2024-04-25T00:00:48.956305+0000","last_clean":"2024-04-25T00:00:48.956305+0000","last_became_active":"2024-04-24T23:34:54.374963+0000","last_became_peered":"2024-04-24T23:34:54.374963+0000","last_unstale":"2024-04-25T00:00:48.956305+0000","last_undegraded":"2024-04-25T00:00:48.956305+0000","last_fullsized":"2024-04-25T00:00:48.956305+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2418,"log_dups_size":0,"ondisk_log_size":2418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:47:52.460508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"27'8937","reported_seq":8974,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.146208+0000","last_change":"2024-04-24T23:34:53.360531+0000","last_active":"2024-04-25T00:00:47.146208+0000","last_peered":"2024-04-25T00:00:47.146208+0000","last_clean":"2024-04-25T00:00:47.146208+0000","last_became_active":"2024-04-24T23:34:53.360247+0000","last_became_peered":"2024-04-24T23:34:53.360247+0000","last_unstale":"2024-04-25T00:00:47.146208+0000","last_undegraded":"2024-04-25T00:00:47.146208+0000","last_fullsized":"2024-04-25T00:00:47.146208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":8937,"log_dups_size":0,"ondisk_log_size":8937,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:57:14.381644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8935,"num_write_kb":79464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"27'9249","reported_seq":10432,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:48.466460+0000","last_change":"2024-04-24T23:34:58.729798+0000","last_active":"2024-04-25T00:00:48.466460+0000","last_peered":"2024-04-25T00:00:48.466460+0000","last_clean":"2024-04-25T00:00:48.466460+0000","last_became_active":"2024-04-24T23:34:58.729345+0000","last_became_peered":"2024-04-24T23:34:58.729345+0000","last_unstale":"2024-04-25T00:00:48.466460+0000","last_undegraded":"2024-04-25T00:00:48.466460+0000","last_fullsized":"2024-04-25T00:00:48.466460+0000","mapping_epoch":23,"log_start":"27'1527","ondisk_log_start":"27'1527","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7722,"log_dups_size":0,"ondisk_log_size":7722,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:06.670518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":427171,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1154,"num_read_kb":293116,"num_write":6507,"num_write_kb":1022776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"27'2518","reported_seq":2552,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:49.157303+0000","last_change":"2024-04-24T23:34:54.377593+0000","last_active":"2024-04-25T00:00:49.157303+0000","last_peered":"2024-04-25T00:00:49.157303+0000","last_clean":"2024-04-25T00:00:49.157303+0000","last_became_active":"2024-04-24T23:34:54.377270+0000","last_became_peered":"2024-04-24T23:34:54.377270+0000","last_unstale":"2024-04-25T00:00:49.157303+0000","last_undegraded":"2024-04-25T00:00:49.157303+0000","last_fullsized":"2024-04-25T00:00:49.157303+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2518,"log_dups_size":0,"ondisk_log_size":2518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:26.261786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"27'4549","reported_seq":4586,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:08.704934+0000","last_change":"2024-04-24T23:34:53.352981+0000","last_active":"2024-04-25T00:00:08.704934+0000","last_peered":"2024-04-25T00:00:08.704934+0000","last_clean":"2024-04-25T00:00:08.704934+0000","last_became_active":"2024-04-24T23:34:53.352669+0000","last_became_peered":"2024-04-24T23:34:53.352669+0000","last_unstale":"2024-04-25T00:00:08.704934+0000","last_undegraded":"2024-04-25T00:00:08.704934+0000","last_fullsized":"2024-04-25T00:00:08.704934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":4549,"log_dups_size":0,"ondisk_log_size":4549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:09:49.550050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4539,"num_write_kb":43853,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"27'9062","reported_seq":9842,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.115773+0000","last_change":"2024-04-24T23:35:43.529974+0000","last_active":"2024-04-25T00:00:50.115773+0000","last_peered":"2024-04-25T00:00:50.115773+0000","last_clean":"2024-04-25T00:00:50.115773+0000","last_became_active":"2024-04-24T23:35:43.529375+0000","last_became_peered":"2024-04-24T23:35:43.529375+0000","last_unstale":"2024-04-25T00:00:50.115773+0000","last_undegraded":"2024-04-25T00:00:50.115773+0000","last_fullsized":"2024-04-25T00:00:50.115773+0000","mapping_epoch":26,"log_start":"27'1912","ondisk_log_start":"27'1912","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7150,"log_dups_size":0,"ondisk_log_size":7150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:48.286783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":744,"num_read_kb":188976,"num_write":6372,"num_write_kb":873040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3,6],"acting":[4,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"27'2587","reported_seq":2621,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:48.619362+0000","last_change":"2024-04-24T23:34:54.376800+0000","last_active":"2024-04-25T00:00:48.619362+0000","last_peered":"2024-04-25T00:00:48.619362+0000","last_clean":"2024-04-25T00:00:48.619362+0000","last_became_active":"2024-04-24T23:34:54.376512+0000","last_became_peered":"2024-04-24T23:34:54.376512+0000","last_unstale":"2024-04-25T00:00:48.619362+0000","last_undegraded":"2024-04-25T00:00:48.619362+0000","last_fullsized":"2024-04-25T00:00:48.619362+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2587,"log_dups_size":0,"ondisk_log_size":2587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:49.048141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"27'3996","reported_seq":4033,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.147213+0000","last_change":"2024-04-24T23:34:53.355307+0000","last_active":"2024-04-25T00:00:47.147213+0000","last_peered":"2024-04-25T00:00:47.147213+0000","last_clean":"2024-04-25T00:00:47.147213+0000","last_became_active":"2024-04-24T23:34:53.354988+0000","last_became_peered":"2024-04-24T23:34:53.354988+0000","last_unstale":"2024-04-25T00:00:47.147213+0000","last_undegraded":"2024-04-25T00:00:47.147213+0000","last_fullsized":"2024-04-25T00:00:47.147213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":3996,"log_dups_size":0,"ondisk_log_size":3996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:39.457116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4001,"num_write_kb":39473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"27'9856","reported_seq":10228,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.120845+0000","last_change":"2024-04-24T23:34:58.399188+0000","last_active":"2024-04-25T00:00:50.120845+0000","last_peered":"2024-04-25T00:00:50.120845+0000","last_clean":"2024-04-25T00:00:50.120845+0000","last_became_active":"2024-04-24T23:34:58.398457+0000","last_became_peered":"2024-04-24T23:34:58.398457+0000","last_unstale":"2024-04-25T00:00:50.120845+0000","last_undegraded":"2024-04-25T00:00:50.120845+0000","last_fullsized":"2024-04-25T00:00:50.120845+0000","mapping_epoch":23,"log_start":"27'2619","ondisk_log_start":"27'2619","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7237,"log_dups_size":0,"ondisk_log_size":7237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:50:21.658540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":344,"num_read_kb":87376,"num_write":7128,"num_write_kb":1170154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"27'2472","reported_seq":2507,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:51.026585+0000","last_change":"2024-04-24T23:34:54.372150+0000","last_active":"2024-04-25T00:00:51.026585+0000","last_peered":"2024-04-25T00:00:51.026585+0000","last_clean":"2024-04-25T00:00:51.026585+0000","last_became_active":"2024-04-24T23:34:54.371703+0000","last_became_peered":"2024-04-24T23:34:54.371703+0000","last_unstale":"2024-04-25T00:00:51.026585+0000","last_undegraded":"2024-04-25T00:00:51.026585+0000","last_fullsized":"2024-04-25T00:00:51.026585+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2472,"log_dups_size":0,"ondisk_log_size":2472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:15.477061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"27'4712","reported_seq":4749,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:42.661191+0000","last_change":"2024-04-24T23:34:53.363787+0000","last_active":"2024-04-25T00:00:42.661191+0000","last_peered":"2024-04-25T00:00:42.661191+0000","last_clean":"2024-04-25T00:00:42.661191+0000","last_became_active":"2024-04-24T23:34:53.363515+0000","last_became_peered":"2024-04-24T23:34:53.363515+0000","last_unstale":"2024-04-25T00:00:42.661191+0000","last_undegraded":"2024-04-25T00:00:42.661191+0000","last_fullsized":"2024-04-25T00:00:42.661191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":4712,"log_dups_size":0,"ondisk_log_size":4712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:50.222015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4705,"num_write_kb":43976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"27'9394","reported_seq":9789,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.521737+0000","last_change":"2024-04-24T23:34:58.728420+0000","last_active":"2024-04-25T00:00:50.521737+0000","last_peered":"2024-04-25T00:00:50.521737+0000","last_clean":"2024-04-25T00:00:50.521737+0000","last_became_active":"2024-04-24T23:34:58.727425+0000","last_became_peered":"2024-04-24T23:34:58.727425+0000","last_unstale":"2024-04-25T00:00:50.521737+0000","last_undegraded":"2024-04-25T00:00:50.521737+0000","last_fullsized":"2024-04-25T00:00:50.521737+0000","mapping_epoch":23,"log_start":"27'3135","ondisk_log_start":"27'3135","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":6259,"log_dups_size":0,"ondisk_log_size":6259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:19:14.487689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":367,"num_read_kb":93218,"num_write":6677,"num_write_kb":924431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"27'2557","reported_seq":2591,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.325385+0000","last_change":"2024-04-24T23:34:54.379296+0000","last_active":"2024-04-25T00:00:47.325385+0000","last_peered":"2024-04-25T00:00:47.325385+0000","last_clean":"2024-04-25T00:00:47.325385+0000","last_became_active":"2024-04-24T23:34:54.379016+0000","last_became_peered":"2024-04-24T23:34:54.379016+0000","last_unstale":"2024-04-25T00:00:47.325385+0000","last_undegraded":"2024-04-25T00:00:47.325385+0000","last_fullsized":"2024-04-25T00:00:47.325385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2557,"log_dups_size":0,"ondisk_log_size":2557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:37.323551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"27'2767","reported_seq":2804,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.170814+0000","last_change":"2024-04-24T23:34:53.360210+0000","last_active":"2024-04-25T00:00:47.170814+0000","last_peered":"2024-04-25T00:00:47.170814+0000","last_clean":"2024-04-25T00:00:47.170814+0000","last_became_active":"2024-04-24T23:34:53.359581+0000","last_became_peered":"2024-04-24T23:34:53.359581+0000","last_unstale":"2024-04-25T00:00:47.170814+0000","last_undegraded":"2024-04-25T00:00:47.170814+0000","last_fullsized":"2024-04-25T00:00:47.170814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":2767,"log_dups_size":0,"ondisk_log_size":2767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:14.716741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2773,"num_write_kb":30370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"27'9089","reported_seq":9559,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.720655+0000","last_change":"2024-04-24T23:34:58.728491+0000","last_active":"2024-04-25T00:00:50.720655+0000","last_peered":"2024-04-25T00:00:50.720655+0000","last_clean":"2024-04-25T00:00:50.720655+0000","last_became_active":"2024-04-24T23:34:58.727686+0000","last_became_peered":"2024-04-24T23:34:58.727686+0000","last_unstale":"2024-04-25T00:00:50.720655+0000","last_undegraded":"2024-04-25T00:00:50.720655+0000","last_fullsized":"2024-04-25T00:00:50.720655+0000","mapping_epoch":23,"log_start":"27'2830","ondisk_log_start":"27'2830","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":6259,"log_dups_size":0,"ondisk_log_size":6259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:46.282712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":441,"num_read_kb":112014,"num_write":6536,"num_write_kb":907183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"27'2526","reported_seq":2561,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.023712+0000","last_change":"2024-04-24T23:34:54.372009+0000","last_active":"2024-04-25T00:00:50.023712+0000","last_peered":"2024-04-25T00:00:50.023712+0000","last_clean":"2024-04-25T00:00:50.023712+0000","last_became_active":"2024-04-24T23:34:54.371720+0000","last_became_peered":"2024-04-24T23:34:54.371720+0000","last_unstale":"2024-04-25T00:00:50.023712+0000","last_undegraded":"2024-04-25T00:00:50.023712+0000","last_fullsized":"2024-04-25T00:00:50.023712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2526,"log_dups_size":0,"ondisk_log_size":2526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:04:07.487367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"27'6531","reported_seq":6567,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:35.064277+0000","last_change":"2024-04-24T23:34:53.353738+0000","last_active":"2024-04-25T00:00:35.064277+0000","last_peered":"2024-04-25T00:00:35.064277+0000","last_clean":"2024-04-25T00:00:35.064277+0000","last_became_active":"2024-04-24T23:34:53.353452+0000","last_became_peered":"2024-04-24T23:34:53.353452+0000","last_unstale":"2024-04-25T00:00:35.064277+0000","last_undegraded":"2024-04-25T00:00:35.064277+0000","last_fullsized":"2024-04-25T00:00:35.064277+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":6531,"log_dups_size":0,"ondisk_log_size":6531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:33:43.693095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971554,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6522,"num_write_kb":57515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"27'9599","reported_seq":10048,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:48.575361+0000","last_change":"2024-04-24T23:34:58.728503+0000","last_active":"2024-04-25T00:00:48.575361+0000","last_peered":"2024-04-25T00:00:48.575361+0000","last_clean":"2024-04-25T00:00:48.575361+0000","last_became_active":"2024-04-24T23:34:58.727921+0000","last_became_peered":"2024-04-24T23:34:58.727921+0000","last_unstale":"2024-04-25T00:00:48.575361+0000","last_undegraded":"2024-04-25T00:00:48.575361+0000","last_fullsized":"2024-04-25T00:00:48.575361+0000","mapping_epoch":23,"log_start":"27'1833","ondisk_log_start":"27'1833","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7766,"log_dups_size":0,"ondisk_log_size":7766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:19:40.928966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1518653,"num_objects":16,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":421,"num_read_kb":106934,"num_write":6985,"num_write_kb":1058941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"27'2469","reported_seq":2502,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:48.895019+0000","last_change":"2024-04-24T23:34:54.368532+0000","last_active":"2024-04-25T00:00:48.895019+0000","last_peered":"2024-04-25T00:00:48.895019+0000","last_clean":"2024-04-25T00:00:48.895019+0000","last_became_active":"2024-04-24T23:34:54.367903+0000","last_became_peered":"2024-04-24T23:34:54.367903+0000","last_unstale":"2024-04-25T00:00:48.895019+0000","last_undegraded":"2024-04-25T00:00:48.895019+0000","last_fullsized":"2024-04-25T00:00:48.895019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2469,"log_dups_size":0,"ondisk_log_size":2469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:49.372040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'9568","reported_seq":10206,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.204974+0000","last_change":"2024-04-24T23:34:53.362758+0000","last_active":"2024-04-25T00:00:47.204974+0000","last_peered":"2024-04-25T00:00:47.204974+0000","last_clean":"2024-04-25T00:00:47.204974+0000","last_became_active":"2024-04-24T23:34:53.362230+0000","last_became_peered":"2024-04-24T23:34:53.362230+0000","last_unstale":"2024-04-25T00:00:47.204974+0000","last_undegraded":"2024-04-25T00:00:47.204974+0000","last_fullsized":"2024-04-25T00:00:47.204974+0000","mapping_epoch":18,"log_start":"27'1600","ondisk_log_start":"27'1600","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":7968,"log_dups_size":0,"ondisk_log_size":7968,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:14.716741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389280,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":602,"num_read_kb":4494,"num_write":9568,"num_write_kb":76044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"27'9012","reported_seq":9418,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.318385+0000","last_change":"2024-04-24T23:34:58.400881+0000","last_active":"2024-04-25T00:00:50.318385+0000","last_peered":"2024-04-25T00:00:50.318385+0000","last_clean":"2024-04-25T00:00:50.318385+0000","last_became_active":"2024-04-24T23:34:58.400429+0000","last_became_peered":"2024-04-24T23:34:58.400429+0000","last_unstale":"2024-04-25T00:00:50.318385+0000","last_undegraded":"2024-04-25T00:00:50.318385+0000","last_fullsized":"2024-04-25T00:00:50.318385+0000","mapping_epoch":23,"log_start":"27'1625","ondisk_log_start":"27'1625","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7387,"log_dups_size":0,"ondisk_log_size":7387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:50:21.658540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":377,"num_read_kb":95758,"num_write":6342,"num_write_kb":955265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"27'2558","reported_seq":2593,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:51.032939+0000","last_change":"2024-04-24T23:34:54.373603+0000","last_active":"2024-04-25T00:00:51.032939+0000","last_peered":"2024-04-25T00:00:51.032939+0000","last_clean":"2024-04-25T00:00:51.032939+0000","last_became_active":"2024-04-24T23:34:54.373347+0000","last_became_peered":"2024-04-24T23:34:54.373347+0000","last_unstale":"2024-04-25T00:00:51.032939+0000","last_undegraded":"2024-04-25T00:00:51.032939+0000","last_fullsized":"2024-04-25T00:00:51.032939+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2558,"log_dups_size":0,"ondisk_log_size":2558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:13.263529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"27'5924","reported_seq":5960,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.204405+0000","last_change":"2024-04-24T23:34:53.354673+0000","last_active":"2024-04-25T00:00:47.204405+0000","last_peered":"2024-04-25T00:00:47.204405+0000","last_clean":"2024-04-25T00:00:47.204405+0000","last_became_active":"2024-04-24T23:34:53.354394+0000","last_became_peered":"2024-04-24T23:34:53.354394+0000","last_unstale":"2024-04-25T00:00:47.204405+0000","last_undegraded":"2024-04-25T00:00:47.204405+0000","last_fullsized":"2024-04-25T00:00:47.204405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":5924,"log_dups_size":0,"ondisk_log_size":5924,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:14.716741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5925,"num_write_kb":41235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"27'9157","reported_seq":9564,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.803421+0000","last_change":"2024-04-24T23:34:58.397739+0000","last_active":"2024-04-25T00:00:50.803421+0000","last_peered":"2024-04-25T00:00:50.803421+0000","last_clean":"2024-04-25T00:00:50.803421+0000","last_became_active":"2024-04-24T23:34:58.397353+0000","last_became_peered":"2024-04-24T23:34:58.397353+0000","last_unstale":"2024-04-25T00:00:50.803421+0000","last_undegraded":"2024-04-25T00:00:50.803421+0000","last_fullsized":"2024-04-25T00:00:50.803421+0000","mapping_epoch":23,"log_start":"27'2001","ondisk_log_start":"27'2001","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7156,"log_dups_size":0,"ondisk_log_size":7156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:55.672101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":378,"num_read_kb":96012,"num_write":6474,"num_write_kb":975733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"27'2564","reported_seq":2598,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:49.547142+0000","last_change":"2024-04-24T23:34:54.383348+0000","last_active":"2024-04-25T00:00:49.547142+0000","last_peered":"2024-04-25T00:00:49.547142+0000","last_clean":"2024-04-25T00:00:49.547142+0000","last_became_active":"2024-04-24T23:34:54.383082+0000","last_became_peered":"2024-04-24T23:34:54.383082+0000","last_unstale":"2024-04-25T00:00:49.547142+0000","last_undegraded":"2024-04-25T00:00:49.547142+0000","last_fullsized":"2024-04-25T00:00:49.547142+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2564,"log_dups_size":0,"ondisk_log_size":2564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:56:17.763159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"27'7101","reported_seq":7137,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.223449+0000","last_change":"2024-04-24T23:34:53.364540+0000","last_active":"2024-04-25T00:00:47.223449+0000","last_peered":"2024-04-25T00:00:47.223449+0000","last_clean":"2024-04-25T00:00:47.223449+0000","last_became_active":"2024-04-24T23:34:53.364259+0000","last_became_peered":"2024-04-24T23:34:53.364259+0000","last_unstale":"2024-04-25T00:00:47.223449+0000","last_undegraded":"2024-04-25T00:00:47.223449+0000","last_fullsized":"2024-04-25T00:00:47.223449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":7101,"log_dups_size":0,"ondisk_log_size":7101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:41.422591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":7102,"num_write_kb":53865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"27'9038","reported_seq":9136,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.008156+0000","last_change":"2024-04-24T23:34:58.405865+0000","last_active":"2024-04-25T00:00:50.008156+0000","last_peered":"2024-04-25T00:00:50.008156+0000","last_clean":"2024-04-25T00:00:50.008156+0000","last_became_active":"2024-04-24T23:34:58.405032+0000","last_became_peered":"2024-04-24T23:34:58.405032+0000","last_unstale":"2024-04-25T00:00:50.008156+0000","last_undegraded":"2024-04-25T00:00:50.008156+0000","last_fullsized":"2024-04-25T00:00:50.008156+0000","mapping_epoch":23,"log_start":"27'2712","ondisk_log_start":"27'2712","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":6326,"log_dups_size":0,"ondisk_log_size":6326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:54.012389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":17526,"num_write":6404,"num_write_kb":965810,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"27'2534","reported_seq":2568,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:48.509523+0000","last_change":"2024-04-24T23:34:54.383631+0000","last_active":"2024-04-25T00:00:48.509523+0000","last_peered":"2024-04-25T00:00:48.509523+0000","last_clean":"2024-04-25T00:00:48.509523+0000","last_became_active":"2024-04-24T23:34:54.383354+0000","last_became_peered":"2024-04-24T23:34:54.383354+0000","last_unstale":"2024-04-25T00:00:48.509523+0000","last_undegraded":"2024-04-25T00:00:48.509523+0000","last_fullsized":"2024-04-25T00:00:48.509523+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2534,"log_dups_size":0,"ondisk_log_size":2534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:50:17.621315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"27'6418","reported_seq":7300,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.177210+0000","last_change":"2024-04-24T23:34:53.362409+0000","last_active":"2024-04-25T00:00:47.177210+0000","last_peered":"2024-04-25T00:00:47.177210+0000","last_clean":"2024-04-25T00:00:47.177210+0000","last_became_active":"2024-04-24T23:34:53.362103+0000","last_became_peered":"2024-04-24T23:34:53.362103+0000","last_unstale":"2024-04-25T00:00:47.177210+0000","last_undegraded":"2024-04-25T00:00:47.177210+0000","last_fullsized":"2024-04-25T00:00:47.177210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":6418,"log_dups_size":0,"ondisk_log_size":6418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:57:14.381644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":847,"num_read_kb":4657,"num_write":6417,"num_write_kb":49337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"27'9416","reported_seq":9704,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.219596+0000","last_change":"2024-04-24T23:34:58.730088+0000","last_active":"2024-04-25T00:00:50.219596+0000","last_peered":"2024-04-25T00:00:50.219596+0000","last_clean":"2024-04-25T00:00:50.219596+0000","last_became_active":"2024-04-24T23:34:58.729217+0000","last_became_peered":"2024-04-24T23:34:58.729217+0000","last_unstale":"2024-04-25T00:00:50.219596+0000","last_undegraded":"2024-04-25T00:00:50.219596+0000","last_fullsized":"2024-04-25T00:00:50.219596+0000","mapping_epoch":23,"log_start":"27'3115","ondisk_log_start":"27'3115","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":6301,"log_dups_size":0,"ondisk_log_size":6301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:12:14.243770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":259,"num_read_kb":65786,"num_write":6628,"num_write_kb":1056336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"27'9147","reported_seq":9276,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:49.915079+0000","last_change":"2024-04-24T23:34:58.728892+0000","last_active":"2024-04-25T00:00:49.915079+0000","last_peered":"2024-04-25T00:00:49.915079+0000","last_clean":"2024-04-25T00:00:49.915079+0000","last_became_active":"2024-04-24T23:34:58.727914+0000","last_became_peered":"2024-04-24T23:34:58.727914+0000","last_unstale":"2024-04-25T00:00:49.915079+0000","last_undegraded":"2024-04-25T00:00:49.915079+0000","last_fullsized":"2024-04-25T00:00:49.915079+0000","mapping_epoch":23,"log_start":"27'1931","ondisk_log_start":"27'1931","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7216,"log_dups_size":0,"ondisk_log_size":7216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:56.609480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":99,"num_read_kb":25146,"num_write":6551,"num_write_kb":920441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"27'2492","reported_seq":2526,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:49.904315+0000","last_change":"2024-04-24T23:34:54.363973+0000","last_active":"2024-04-25T00:00:49.904315+0000","last_peered":"2024-04-25T00:00:49.904315+0000","last_clean":"2024-04-25T00:00:49.904315+0000","last_became_active":"2024-04-24T23:34:54.363724+0000","last_became_peered":"2024-04-24T23:34:54.363724+0000","last_unstale":"2024-04-25T00:00:49.904315+0000","last_undegraded":"2024-04-25T00:00:49.904315+0000","last_fullsized":"2024-04-25T00:00:49.904315+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2492,"log_dups_size":0,"ondisk_log_size":2492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:03:24.745895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"27'9687","reported_seq":9723,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.373826+0000","last_change":"2024-04-24T23:34:53.360041+0000","last_active":"2024-04-25T00:00:47.373826+0000","last_peered":"2024-04-25T00:00:47.373826+0000","last_clean":"2024-04-25T00:00:47.373826+0000","last_became_active":"2024-04-24T23:34:53.359707+0000","last_became_peered":"2024-04-24T23:34:53.359707+0000","last_unstale":"2024-04-25T00:00:47.373826+0000","last_undegraded":"2024-04-25T00:00:47.373826+0000","last_fullsized":"2024-04-25T00:00:47.373826+0000","mapping_epoch":18,"log_start":"27'1900","ondisk_log_start":"27'1900","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":7787,"log_dups_size":0,"ondisk_log_size":7787,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:50.222015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9677,"num_write_kb":67972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"27'9609","reported_seq":10700,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:49.203484+0000","last_change":"2024-04-24T23:35:43.834517+0000","last_active":"2024-04-25T00:00:49.203484+0000","last_peered":"2024-04-25T00:00:49.203484+0000","last_clean":"2024-04-25T00:00:49.203484+0000","last_became_active":"2024-04-24T23:35:43.834066+0000","last_became_peered":"2024-04-24T23:35:43.834066+0000","last_unstale":"2024-04-25T00:00:49.203484+0000","last_undegraded":"2024-04-25T00:00:49.203484+0000","last_fullsized":"2024-04-25T00:00:49.203484+0000","mapping_epoch":26,"log_start":"27'1708","ondisk_log_start":"27'1708","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7901,"log_dups_size":0,"ondisk_log_size":7901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:48.286783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1054,"num_read_kb":267716,"num_write":6908,"num_write_kb":899799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2,0],"acting":[6,5,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"27'2603","reported_seq":2637,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:51.008132+0000","last_change":"2024-04-24T23:34:54.369729+0000","last_active":"2024-04-25T00:00:51.008132+0000","last_peered":"2024-04-25T00:00:51.008132+0000","last_clean":"2024-04-25T00:00:51.008132+0000","last_became_active":"2024-04-24T23:34:54.369155+0000","last_became_peered":"2024-04-24T23:34:54.369155+0000","last_unstale":"2024-04-25T00:00:51.008132+0000","last_undegraded":"2024-04-25T00:00:51.008132+0000","last_fullsized":"2024-04-25T00:00:51.008132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2603,"log_dups_size":0,"ondisk_log_size":2603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:46:28.181996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"27'4844","reported_seq":4880,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.211524+0000","last_change":"2024-04-24T23:34:53.365328+0000","last_active":"2024-04-25T00:00:47.211524+0000","last_peered":"2024-04-25T00:00:47.211524+0000","last_clean":"2024-04-25T00:00:47.211524+0000","last_became_active":"2024-04-24T23:34:53.365029+0000","last_became_peered":"2024-04-24T23:34:53.365029+0000","last_unstale":"2024-04-25T00:00:47.211524+0000","last_undegraded":"2024-04-25T00:00:47.211524+0000","last_fullsized":"2024-04-25T00:00:47.211524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":4844,"log_dups_size":0,"ondisk_log_size":4844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:48:30.961870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4849,"num_write_kb":44132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"27'9228","reported_seq":9264,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.258606+0000","last_change":"2024-04-24T23:34:53.365123+0000","last_active":"2024-04-25T00:00:47.258606+0000","last_peered":"2024-04-25T00:00:47.258606+0000","last_clean":"2024-04-25T00:00:47.258606+0000","last_became_active":"2024-04-24T23:34:53.364821+0000","last_became_peered":"2024-04-24T23:34:53.364821+0000","last_unstale":"2024-04-25T00:00:47.258606+0000","last_undegraded":"2024-04-25T00:00:47.258606+0000","last_fullsized":"2024-04-25T00:00:47.258606+0000","mapping_epoch":18,"log_start":"27'1300","ondisk_log_start":"27'1300","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":7928,"log_dups_size":0,"ondisk_log_size":7928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:39.457116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9220,"num_write_kb":71599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"27'2524","reported_seq":2558,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.139570+0000","last_change":"2024-04-24T23:34:54.376037+0000","last_active":"2024-04-25T00:00:50.139570+0000","last_peered":"2024-04-25T00:00:50.139570+0000","last_clean":"2024-04-25T00:00:50.139570+0000","last_became_active":"2024-04-24T23:34:54.375689+0000","last_became_peered":"2024-04-24T23:34:54.375689+0000","last_unstale":"2024-04-25T00:00:50.139570+0000","last_undegraded":"2024-04-25T00:00:50.139570+0000","last_fullsized":"2024-04-25T00:00:50.139570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2524,"log_dups_size":0,"ondisk_log_size":2524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:47.446459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"27'9696","reported_seq":9920,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:48.257507+0000","last_change":"2024-04-24T23:35:43.834538+0000","last_active":"2024-04-25T00:00:48.257507+0000","last_peered":"2024-04-25T00:00:48.257507+0000","last_clean":"2024-04-25T00:00:48.257507+0000","last_became_active":"2024-04-24T23:35:43.834061+0000","last_became_peered":"2024-04-24T23:35:43.834061+0000","last_unstale":"2024-04-25T00:00:48.257507+0000","last_undegraded":"2024-04-25T00:00:48.257507+0000","last_fullsized":"2024-04-25T00:00:48.257507+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":9696,"log_dups_size":0,"ondisk_log_size":9696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:41:52.244767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3304017,"num_objects":33,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":189,"num_read_kb":48006,"num_write":6965,"num_write_kb":973944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3,5],"acting":[2,0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"27'7147","reported_seq":7183,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-24T23:59:59.858753+0000","last_change":"2024-04-24T23:34:53.363066+0000","last_active":"2024-04-24T23:59:59.858753+0000","last_peered":"2024-04-24T23:59:59.858753+0000","last_clean":"2024-04-24T23:59:59.858753+0000","last_became_active":"2024-04-24T23:34:53.362548+0000","last_became_peered":"2024-04-24T23:34:53.362548+0000","last_unstale":"2024-04-24T23:59:59.858753+0000","last_undegraded":"2024-04-24T23:59:59.858753+0000","last_fullsized":"2024-04-24T23:59:59.858753+0000","mapping_epoch":18,"log_start":"27'800","ondisk_log_start":"27'800","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":6347,"log_dups_size":0,"ondisk_log_size":6347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:21:13.338964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7134,"num_write_kb":66117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"27'2573","reported_seq":2607,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.973745+0000","last_change":"2024-04-24T23:34:54.365704+0000","last_active":"2024-04-25T00:00:50.973745+0000","last_peered":"2024-04-25T00:00:50.973745+0000","last_clean":"2024-04-25T00:00:50.973745+0000","last_became_active":"2024-04-24T23:34:54.364814+0000","last_became_peered":"2024-04-24T23:34:54.364814+0000","last_unstale":"2024-04-25T00:00:50.973745+0000","last_undegraded":"2024-04-25T00:00:50.973745+0000","last_fullsized":"2024-04-25T00:00:50.973745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2573,"log_dups_size":0,"ondisk_log_size":2573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:22.094785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"27'9398","reported_seq":9604,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.632365+0000","last_change":"2024-04-24T23:34:58.407186+0000","last_active":"2024-04-25T00:00:50.632365+0000","last_peered":"2024-04-25T00:00:50.632365+0000","last_clean":"2024-04-25T00:00:50.632365+0000","last_became_active":"2024-04-24T23:34:58.406811+0000","last_became_peered":"2024-04-24T23:34:58.406811+0000","last_unstale":"2024-04-25T00:00:50.632365+0000","last_undegraded":"2024-04-25T00:00:50.632365+0000","last_fullsized":"2024-04-25T00:00:50.632365+0000","mapping_epoch":23,"log_start":"27'3112","ondisk_log_start":"27'3112","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":6286,"log_dups_size":0,"ondisk_log_size":6286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:55.861555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":178,"num_read_kb":45212,"num_write":6641,"num_write_kb":1045709,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"27'2471","reported_seq":2507,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:51.031872+0000","last_change":"2024-04-24T23:34:54.375142+0000","last_active":"2024-04-25T00:00:51.031872+0000","last_peered":"2024-04-25T00:00:51.031872+0000","last_clean":"2024-04-25T00:00:51.031872+0000","last_became_active":"2024-04-24T23:34:54.374844+0000","last_became_peered":"2024-04-24T23:34:54.374844+0000","last_unstale":"2024-04-25T00:00:51.031872+0000","last_undegraded":"2024-04-25T00:00:51.031872+0000","last_fullsized":"2024-04-25T00:00:51.031872+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2471,"log_dups_size":0,"ondisk_log_size":2471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:29:07.593498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"27'2583","reported_seq":2619,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:48.181599+0000","last_change":"2024-04-24T23:34:54.383574+0000","last_active":"2024-04-25T00:00:48.181599+0000","last_peered":"2024-04-25T00:00:48.181599+0000","last_clean":"2024-04-25T00:00:48.181599+0000","last_became_active":"2024-04-24T23:34:54.383253+0000","last_became_peered":"2024-04-24T23:34:54.383253+0000","last_unstale":"2024-04-25T00:00:48.181599+0000","last_undegraded":"2024-04-25T00:00:48.181599+0000","last_fullsized":"2024-04-25T00:00:48.181599+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2583,"log_dups_size":0,"ondisk_log_size":2583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:43.211314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"27'2465","reported_seq":2487,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.944438+0000","last_change":"2024-04-24T23:35:43.526600+0000","last_active":"2024-04-25T00:00:47.944438+0000","last_peered":"2024-04-25T00:00:47.944438+0000","last_clean":"2024-04-25T00:00:47.944438+0000","last_became_active":"2024-04-24T23:35:43.525929+0000","last_became_peered":"2024-04-24T23:35:43.525929+0000","last_unstale":"2024-04-25T00:00:47.944438+0000","last_undegraded":"2024-04-25T00:00:47.944438+0000","last_fullsized":"2024-04-25T00:00:47.944438+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2465,"log_dups_size":0,"ondisk_log_size":2465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:49.372040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"27'2575","reported_seq":2611,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:48.937515+0000","last_change":"2024-04-24T23:34:54.366271+0000","last_active":"2024-04-25T00:00:48.937515+0000","last_peered":"2024-04-25T00:00:48.937515+0000","last_clean":"2024-04-25T00:00:48.937515+0000","last_became_active":"2024-04-24T23:34:54.366005+0000","last_became_peered":"2024-04-24T23:34:54.366005+0000","last_unstale":"2024-04-25T00:00:48.937515+0000","last_undegraded":"2024-04-25T00:00:48.937515+0000","last_fullsized":"2024-04-25T00:00:48.937515+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2575,"log_dups_size":0,"ondisk_log_size":2575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:03:24.745895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"27'2585","reported_seq":2621,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:51.019874+0000","last_change":"2024-04-24T23:34:54.360529+0000","last_active":"2024-04-25T00:00:51.019874+0000","last_peered":"2024-04-25T00:00:51.019874+0000","last_clean":"2024-04-25T00:00:51.019874+0000","last_became_active":"2024-04-24T23:34:54.360266+0000","last_became_peered":"2024-04-24T23:34:54.360266+0000","last_unstale":"2024-04-25T00:00:51.019874+0000","last_undegraded":"2024-04-25T00:00:51.019874+0000","last_fullsized":"2024-04-25T00:00:51.019874+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2585,"log_dups_size":0,"ondisk_log_size":2585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:49:47.246193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"27'2450","reported_seq":2485,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:49.282540+0000","last_change":"2024-04-24T23:34:54.365744+0000","last_active":"2024-04-25T00:00:49.282540+0000","last_peered":"2024-04-25T00:00:49.282540+0000","last_clean":"2024-04-25T00:00:49.282540+0000","last_became_active":"2024-04-24T23:34:54.365493+0000","last_became_peered":"2024-04-24T23:34:54.365493+0000","last_unstale":"2024-04-25T00:00:49.282540+0000","last_undegraded":"2024-04-25T00:00:49.282540+0000","last_fullsized":"2024-04-25T00:00:49.282540+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2450,"log_dups_size":0,"ondisk_log_size":2450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:15.477061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"27'2590","reported_seq":2625,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:51.032201+0000","last_change":"2024-04-24T23:34:54.379642+0000","last_active":"2024-04-25T00:00:51.032201+0000","last_peered":"2024-04-25T00:00:51.032201+0000","last_clean":"2024-04-25T00:00:51.032201+0000","last_became_active":"2024-04-24T23:34:54.378568+0000","last_became_peered":"2024-04-24T23:34:54.378568+0000","last_unstale":"2024-04-25T00:00:51.032201+0000","last_undegraded":"2024-04-25T00:00:51.032201+0000","last_fullsized":"2024-04-25T00:00:51.032201+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2590,"log_dups_size":0,"ondisk_log_size":2590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:42.199856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"27'2557","reported_seq":2592,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.983982+0000","last_change":"2024-04-24T23:34:54.363700+0000","last_active":"2024-04-25T00:00:50.983982+0000","last_peered":"2024-04-25T00:00:50.983982+0000","last_clean":"2024-04-25T00:00:50.983982+0000","last_became_active":"2024-04-24T23:34:54.363439+0000","last_became_peered":"2024-04-24T23:34:54.363439+0000","last_unstale":"2024-04-25T00:00:50.983982+0000","last_undegraded":"2024-04-25T00:00:50.983982+0000","last_fullsized":"2024-04-25T00:00:50.983982+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2557,"log_dups_size":0,"ondisk_log_size":2557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:28:56.048178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"27'2531","reported_seq":2566,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:48.430106+0000","last_change":"2024-04-24T23:34:54.375460+0000","last_active":"2024-04-25T00:00:48.430106+0000","last_peered":"2024-04-25T00:00:48.430106+0000","last_clean":"2024-04-25T00:00:48.430106+0000","last_became_active":"2024-04-24T23:34:54.375123+0000","last_became_peered":"2024-04-24T23:34:54.375123+0000","last_unstale":"2024-04-25T00:00:48.430106+0000","last_undegraded":"2024-04-25T00:00:48.430106+0000","last_fullsized":"2024-04-25T00:00:48.430106+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2531,"log_dups_size":0,"ondisk_log_size":2531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:18.207267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"27'2526","reported_seq":2561,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:51.025670+0000","last_change":"2024-04-24T23:34:54.369509+0000","last_active":"2024-04-25T00:00:51.025670+0000","last_peered":"2024-04-25T00:00:51.025670+0000","last_clean":"2024-04-25T00:00:51.025670+0000","last_became_active":"2024-04-24T23:34:54.369219+0000","last_became_peered":"2024-04-24T23:34:54.369219+0000","last_unstale":"2024-04-25T00:00:51.025670+0000","last_undegraded":"2024-04-25T00:00:51.025670+0000","last_fullsized":"2024-04-25T00:00:51.025670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2526,"log_dups_size":0,"ondisk_log_size":2526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:09:37.649503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"27'2551","reported_seq":2572,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:48.387420+0000","last_change":"2024-04-24T23:35:43.933911+0000","last_active":"2024-04-25T00:00:48.387420+0000","last_peered":"2024-04-25T00:00:48.387420+0000","last_clean":"2024-04-25T00:00:48.387420+0000","last_became_active":"2024-04-24T23:35:43.933686+0000","last_became_peered":"2024-04-24T23:35:43.933686+0000","last_unstale":"2024-04-25T00:00:48.387420+0000","last_undegraded":"2024-04-25T00:00:48.387420+0000","last_fullsized":"2024-04-25T00:00:48.387420+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2551,"log_dups_size":0,"ondisk_log_size":2551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:26.261786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2b","version":"27'2585","reported_seq":2620,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:48.808345+0000","last_change":"2024-04-24T23:34:54.371052+0000","last_active":"2024-04-25T00:00:48.808345+0000","last_peered":"2024-04-25T00:00:48.808345+0000","last_clean":"2024-04-25T00:00:48.808345+0000","last_became_active":"2024-04-24T23:34:54.370793+0000","last_became_peered":"2024-04-24T23:34:54.370793+0000","last_unstale":"2024-04-25T00:00:48.808345+0000","last_undegraded":"2024-04-25T00:00:48.808345+0000","last_fullsized":"2024-04-25T00:00:48.808345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2585,"log_dups_size":0,"ondisk_log_size":2585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:29:07.593498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"27'2595","reported_seq":2630,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.986378+0000","last_change":"2024-04-24T23:34:54.377190+0000","last_active":"2024-04-25T00:00:50.986378+0000","last_peered":"2024-04-25T00:00:50.986378+0000","last_clean":"2024-04-25T00:00:50.986378+0000","last_became_active":"2024-04-24T23:34:54.376940+0000","last_became_peered":"2024-04-24T23:34:54.376940+0000","last_unstale":"2024-04-25T00:00:50.986378+0000","last_undegraded":"2024-04-25T00:00:50.986378+0000","last_fullsized":"2024-04-25T00:00:50.986378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2595,"log_dups_size":0,"ondisk_log_size":2595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:03:24.745895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"27'2570","reported_seq":2604,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:49.071176+0000","last_change":"2024-04-24T23:34:54.368770+0000","last_active":"2024-04-25T00:00:49.071176+0000","last_peered":"2024-04-25T00:00:49.071176+0000","last_clean":"2024-04-25T00:00:49.071176+0000","last_became_active":"2024-04-24T23:34:54.368228+0000","last_became_peered":"2024-04-24T23:34:54.368228+0000","last_unstale":"2024-04-25T00:00:49.071176+0000","last_undegraded":"2024-04-25T00:00:49.071176+0000","last_fullsized":"2024-04-25T00:00:49.071176+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2570,"log_dups_size":0,"ondisk_log_size":2570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:51.634876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"27'2510","reported_seq":2545,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.391247+0000","last_change":"2024-04-24T23:34:54.373038+0000","last_active":"2024-04-25T00:00:47.391247+0000","last_peered":"2024-04-25T00:00:47.391247+0000","last_clean":"2024-04-25T00:00:47.391247+0000","last_became_active":"2024-04-24T23:34:54.372766+0000","last_became_peered":"2024-04-24T23:34:54.372766+0000","last_unstale":"2024-04-25T00:00:47.391247+0000","last_undegraded":"2024-04-25T00:00:47.391247+0000","last_fullsized":"2024-04-25T00:00:47.391247+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2510,"log_dups_size":0,"ondisk_log_size":2510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:13.263529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"27'2577","reported_seq":2612,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.872643+0000","last_change":"2024-04-24T23:34:54.383733+0000","last_active":"2024-04-25T00:00:47.872643+0000","last_peered":"2024-04-25T00:00:47.872643+0000","last_clean":"2024-04-25T00:00:47.872643+0000","last_became_active":"2024-04-24T23:34:54.383463+0000","last_became_peered":"2024-04-24T23:34:54.383463+0000","last_unstale":"2024-04-25T00:00:47.872643+0000","last_undegraded":"2024-04-25T00:00:47.872643+0000","last_fullsized":"2024-04-25T00:00:47.872643+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2577,"log_dups_size":0,"ondisk_log_size":2577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:51.634876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"27'2508","reported_seq":2543,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:48.402598+0000","last_change":"2024-04-24T23:34:54.380959+0000","last_active":"2024-04-25T00:00:48.402598+0000","last_peered":"2024-04-25T00:00:48.402598+0000","last_clean":"2024-04-25T00:00:48.402598+0000","last_became_active":"2024-04-24T23:34:54.380702+0000","last_became_peered":"2024-04-24T23:34:54.380702+0000","last_unstale":"2024-04-25T00:00:48.402598+0000","last_undegraded":"2024-04-25T00:00:48.402598+0000","last_fullsized":"2024-04-25T00:00:48.402598+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2508,"log_dups_size":0,"ondisk_log_size":2508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:45.424046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"27'2641","reported_seq":2676,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.055368+0000","last_change":"2024-04-24T23:34:54.358461+0000","last_active":"2024-04-25T00:00:50.055368+0000","last_peered":"2024-04-25T00:00:50.055368+0000","last_clean":"2024-04-25T00:00:50.055368+0000","last_became_active":"2024-04-24T23:34:54.358203+0000","last_became_peered":"2024-04-24T23:34:54.358203+0000","last_unstale":"2024-04-25T00:00:50.055368+0000","last_undegraded":"2024-04-25T00:00:50.055368+0000","last_fullsized":"2024-04-25T00:00:50.055368+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2641,"log_dups_size":0,"ondisk_log_size":2641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:18.207267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"27'2489","reported_seq":2524,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:49.786683+0000","last_change":"2024-04-24T23:34:54.367860+0000","last_active":"2024-04-25T00:00:49.786683+0000","last_peered":"2024-04-25T00:00:49.786683+0000","last_clean":"2024-04-25T00:00:49.786683+0000","last_became_active":"2024-04-24T23:34:54.367595+0000","last_became_peered":"2024-04-24T23:34:54.367595+0000","last_unstale":"2024-04-25T00:00:49.786683+0000","last_undegraded":"2024-04-25T00:00:49.786683+0000","last_fullsized":"2024-04-25T00:00:49.786683+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2489,"log_dups_size":0,"ondisk_log_size":2489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:40:40.928386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"27'2477","reported_seq":2497,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.404714+0000","last_change":"2024-04-24T23:35:43.529991+0000","last_active":"2024-04-25T00:00:50.404714+0000","last_peered":"2024-04-25T00:00:50.404714+0000","last_clean":"2024-04-25T00:00:50.404714+0000","last_became_active":"2024-04-24T23:35:43.529631+0000","last_became_peered":"2024-04-24T23:35:43.529631+0000","last_unstale":"2024-04-25T00:00:50.404714+0000","last_undegraded":"2024-04-25T00:00:50.404714+0000","last_fullsized":"2024-04-25T00:00:50.404714+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2477,"log_dups_size":0,"ondisk_log_size":2477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:40:39.166560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.34","version":"27'2450","reported_seq":2484,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.997251+0000","last_change":"2024-04-24T23:34:54.377573+0000","last_active":"2024-04-25T00:00:50.997251+0000","last_peered":"2024-04-25T00:00:50.997251+0000","last_clean":"2024-04-25T00:00:50.997251+0000","last_became_active":"2024-04-24T23:34:54.377277+0000","last_became_peered":"2024-04-24T23:34:54.377277+0000","last_unstale":"2024-04-25T00:00:50.997251+0000","last_undegraded":"2024-04-25T00:00:50.997251+0000","last_fullsized":"2024-04-25T00:00:50.997251+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2450,"log_dups_size":0,"ondisk_log_size":2450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:43.211314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"27'2496","reported_seq":2530,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.983969+0000","last_change":"2024-04-24T23:34:54.388134+0000","last_active":"2024-04-25T00:00:50.983969+0000","last_peered":"2024-04-25T00:00:50.983969+0000","last_clean":"2024-04-25T00:00:50.983969+0000","last_became_active":"2024-04-24T23:34:54.387781+0000","last_became_peered":"2024-04-24T23:34:54.387781+0000","last_unstale":"2024-04-25T00:00:50.983969+0000","last_undegraded":"2024-04-25T00:00:50.983969+0000","last_fullsized":"2024-04-25T00:00:50.983969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2496,"log_dups_size":0,"ondisk_log_size":2496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:56:17.763159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"27'2613","reported_seq":2647,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:51.008155+0000","last_change":"2024-04-24T23:34:54.388182+0000","last_active":"2024-04-25T00:00:51.008155+0000","last_peered":"2024-04-25T00:00:51.008155+0000","last_clean":"2024-04-25T00:00:51.008155+0000","last_became_active":"2024-04-24T23:34:54.387913+0000","last_became_peered":"2024-04-24T23:34:54.387913+0000","last_unstale":"2024-04-25T00:00:51.008155+0000","last_undegraded":"2024-04-25T00:00:51.008155+0000","last_fullsized":"2024-04-25T00:00:51.008155+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2613,"log_dups_size":0,"ondisk_log_size":2613,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:18.207267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"27'2556","reported_seq":2589,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.919316+0000","last_change":"2024-04-24T23:34:54.383303+0000","last_active":"2024-04-25T00:00:47.919316+0000","last_peered":"2024-04-25T00:00:47.919316+0000","last_clean":"2024-04-25T00:00:47.919316+0000","last_became_active":"2024-04-24T23:34:54.383024+0000","last_became_peered":"2024-04-24T23:34:54.383024+0000","last_unstale":"2024-04-25T00:00:47.919316+0000","last_undegraded":"2024-04-25T00:00:47.919316+0000","last_fullsized":"2024-04-25T00:00:47.919316+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2556,"log_dups_size":0,"ondisk_log_size":2556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:45.424046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"27'2526","reported_seq":2560,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:51.021931+0000","last_change":"2024-04-24T23:34:54.382803+0000","last_active":"2024-04-25T00:00:51.021931+0000","last_peered":"2024-04-25T00:00:51.021931+0000","last_clean":"2024-04-25T00:00:51.021931+0000","last_became_active":"2024-04-24T23:34:54.382482+0000","last_became_peered":"2024-04-24T23:34:54.382482+0000","last_unstale":"2024-04-25T00:00:51.021931+0000","last_undegraded":"2024-04-25T00:00:51.021931+0000","last_fullsized":"2024-04-25T00:00:51.021931+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2526,"log_dups_size":0,"ondisk_log_size":2526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:27:33.721584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"27'2525","reported_seq":2558,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:49.069349+0000","last_change":"2024-04-24T23:34:54.369167+0000","last_active":"2024-04-25T00:00:49.069349+0000","last_peered":"2024-04-25T00:00:49.069349+0000","last_clean":"2024-04-25T00:00:49.069349+0000","last_became_active":"2024-04-24T23:34:54.368663+0000","last_became_peered":"2024-04-24T23:34:54.368663+0000","last_unstale":"2024-04-25T00:00:49.069349+0000","last_undegraded":"2024-04-25T00:00:49.069349+0000","last_fullsized":"2024-04-25T00:00:49.069349+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2525,"log_dups_size":0,"ondisk_log_size":2525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:18.207267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"27'2452","reported_seq":2485,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:48.608029+0000","last_change":"2024-04-24T23:34:54.380345+0000","last_active":"2024-04-25T00:00:48.608029+0000","last_peered":"2024-04-25T00:00:48.608029+0000","last_clean":"2024-04-25T00:00:48.608029+0000","last_became_active":"2024-04-24T23:34:54.380052+0000","last_became_peered":"2024-04-24T23:34:54.380052+0000","last_unstale":"2024-04-25T00:00:48.608029+0000","last_undegraded":"2024-04-25T00:00:48.608029+0000","last_fullsized":"2024-04-25T00:00:48.608029+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2452,"log_dups_size":0,"ondisk_log_size":2452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:42.199856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"27'2497","reported_seq":2530,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:49.181531+0000","last_change":"2024-04-24T23:34:54.365114+0000","last_active":"2024-04-25T00:00:49.181531+0000","last_peered":"2024-04-25T00:00:49.181531+0000","last_clean":"2024-04-25T00:00:49.181531+0000","last_became_active":"2024-04-24T23:34:54.364605+0000","last_became_peered":"2024-04-24T23:34:54.364605+0000","last_unstale":"2024-04-25T00:00:49.181531+0000","last_undegraded":"2024-04-25T00:00:49.181531+0000","last_fullsized":"2024-04-25T00:00:49.181531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2497,"log_dups_size":0,"ondisk_log_size":2497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:07:28.785202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"27'2544","reported_seq":2566,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.971798+0000","last_change":"2024-04-24T23:35:43.525630+0000","last_active":"2024-04-25T00:00:47.971798+0000","last_peered":"2024-04-25T00:00:47.971798+0000","last_clean":"2024-04-25T00:00:47.971798+0000","last_became_active":"2024-04-24T23:35:43.525191+0000","last_became_peered":"2024-04-24T23:35:43.525191+0000","last_unstale":"2024-04-25T00:00:47.971798+0000","last_undegraded":"2024-04-25T00:00:47.971798+0000","last_fullsized":"2024-04-25T00:00:47.971798+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2544,"log_dups_size":0,"ondisk_log_size":2544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:07:28.785202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"27'2608","reported_seq":2644,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.023985+0000","last_change":"2024-04-24T23:34:54.366079+0000","last_active":"2024-04-25T00:00:50.023985+0000","last_peered":"2024-04-25T00:00:50.023985+0000","last_clean":"2024-04-25T00:00:50.023985+0000","last_became_active":"2024-04-24T23:34:54.365814+0000","last_became_peered":"2024-04-24T23:34:54.365814+0000","last_unstale":"2024-04-25T00:00:50.023985+0000","last_undegraded":"2024-04-25T00:00:50.023985+0000","last_fullsized":"2024-04-25T00:00:50.023985+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2608,"log_dups_size":0,"ondisk_log_size":2608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:18.207267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"27'2471","reported_seq":2507,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:49.860449+0000","last_change":"2024-04-24T23:34:54.368976+0000","last_active":"2024-04-25T00:00:49.860449+0000","last_peered":"2024-04-25T00:00:49.860449+0000","last_clean":"2024-04-25T00:00:49.860449+0000","last_became_active":"2024-04-24T23:34:54.368663+0000","last_became_peered":"2024-04-24T23:34:54.368663+0000","last_unstale":"2024-04-25T00:00:49.860449+0000","last_undegraded":"2024-04-25T00:00:49.860449+0000","last_fullsized":"2024-04-25T00:00:49.860449+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2471,"log_dups_size":0,"ondisk_log_size":2471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:56:17.763159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"27'2453","reported_seq":2488,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:49.117795+0000","last_change":"2024-04-24T23:34:54.373810+0000","last_active":"2024-04-25T00:00:49.117795+0000","last_peered":"2024-04-25T00:00:49.117795+0000","last_clean":"2024-04-25T00:00:49.117795+0000","last_became_active":"2024-04-24T23:34:54.373531+0000","last_became_peered":"2024-04-24T23:34:54.373531+0000","last_unstale":"2024-04-25T00:00:49.117795+0000","last_undegraded":"2024-04-25T00:00:49.117795+0000","last_fullsized":"2024-04-25T00:00:49.117795+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2453,"log_dups_size":0,"ondisk_log_size":2453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:50:17.621315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":5687423,"num_objects":61,"num_object_clones":0,"num_object_copies":244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":12310,"num_read_kb":3126740,"num_write":212463,"num_write_kb":30664179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6545408,"data_stored":47833088,"data_compressed":2192083,"data_compressed_allocated":4247552,"data_compressed_original":45535232,"omap_allocated":0,"internal_metadata":0},"log_size":230470,"ondisk_log_size":230470,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1093,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":162301,"ondisk_log_size":162301,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":236407396,"num_objects":79,"num_object_clones":0,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":0,"num_read":3152,"num_read_kb":19253,"num_write":210108,"num_write_kb":1682526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":287875072,"data_stored":472814792,"data_compressed":55740636,"data_compressed_allocated":81428480,"data_compressed_original":266428416,"omap_allocated":0,"internal_metadata":0},"log_size":195657,"ondisk_log_size":195657,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65248,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542458,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":963152,"kb_used_data":29704,"kb_used_omap":1,"kb_used_meta":933438,"kb_avail":93408688,"statfs":{"total":96636764160,"available":95650496512,"internally_reserved":0,"allocated":30416896,"data_stored":56977813,"data_compressed":5989303,"data_compressed_allocated":9019392,"data_compressed_original":35758080,"omap_allocated":1590,"internal_metadata":955840970},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,9,8,11,10,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542457,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1126144,"kb_used_data":40056,"kb_used_omap":1,"kb_used_meta":1086078,"kb_avail":93245696,"statfs":{"total":96636764160,"available":95483592704,"internally_reserved":0,"allocated":41017344,"data_stored":68055450,"data_compressed":8055094,"data_compressed_allocated":11661312,"data_compressed_original":38879232,"omap_allocated":1594,"internal_metadata":1112144326},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542457,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":978456,"kb_used_data":27024,"kb_used_omap":1,"kb_used_meta":951422,"kb_avail":93393384,"statfs":{"total":96636764160,"available":95634825216,"internally_reserved":0,"allocated":27672576,"data_stored":58276474,"data_compressed":5800819,"data_compressed_allocated":8785920,"data_compressed_original":39563264,"omap_allocated":1595,"internal_metadata":974256581},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542457,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":920748,"kb_used_data":41380,"kb_used_omap":1,"kb_used_meta":879358,"kb_avail":93451092,"statfs":{"total":96636764160,"available":95693918208,"internally_reserved":0,"allocated":42373120,"data_stored":70089374,"data_compressed":7916354,"data_compressed_allocated":11673600,"data_compressed_original":39571456,"omap_allocated":1595,"internal_metadata":900463045},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542458,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":663352,"kb_used_data":36592,"kb_used_omap":1,"kb_used_meta":626750,"kb_avail":93708488,"statfs":{"total":96636764160,"available":95957491712,"internally_reserved":0,"allocated":37470208,"data_stored":74025504,"data_compressed":7080739,"data_compressed_allocated":10862592,"data_compressed_original":47587328,"omap_allocated":1595,"internal_metadata":641792453},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542458,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1253912,"kb_used_data":41424,"kb_used_omap":1,"kb_used_meta":1212478,"kb_avail":93117928,"statfs":{"total":96636764160,"available":95352758272,"internally_reserved":0,"allocated":42418176,"data_stored":76753548,"data_compressed":7956335,"data_compressed_allocated":12107776,"data_compressed_original":46620672,"omap_allocated":1595,"internal_metadata":1241577925},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,4,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542458,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":823368,"kb_used_data":34304,"kb_used_omap":1,"kb_used_meta":789054,"kb_avail":93548472,"statfs":{"total":96636764160,"available":95793635328,"internally_reserved":0,"allocated":35127296,"data_stored":63717002,"data_compressed":7508106,"data_compressed_allocated":10883072,"data_compressed_original":39657472,"omap_allocated":1595,"internal_metadata":807991749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4],"upper_bound":16},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575163,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1147316,"kb_used_data":43116,"kb_used_omap":1,"kb_used_meta":1104190,"kb_avail":93224524,"statfs":{"total":96636764160,"available":95461912576,"internally_reserved":0,"allocated":44150784,"data_stored":83659968,"data_compressed":9013661,"data_compressed_allocated":13275136,"data_compressed_original":52961280,"omap_allocated":1593,"internal_metadata":1130691015},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,5,4,8,6,10,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32624,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32624,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":34275328,"data_stored":59417162,"data_compressed":7311941,"data_compressed_allocated":10522624,"data_compressed_original":35676160,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40407040,"data_stored":63610988,"data_compressed":7358219,"data_compressed_allocated":10981376,"data_compressed_original":34193408,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":34680832,"data_stored":54525952,"data_compressed":6190216,"data_compressed_allocated":9224192,"data_compressed_original":29069312,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":42323968,"data_stored":71303840,"data_compressed":8449066,"data_compressed_allocated":12222464,"data_compressed_original":41209856,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":41668608,"data_stored":67108990,"data_compressed":7785480,"data_compressed_allocated":11411456,"data_compressed_original":36864000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":25022464,"data_stored":41943130,"data_compressed":5058379,"data_compressed_allocated":7299072,"data_compressed_original":24223744,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40730624,"data_stored":67938170,"data_compressed":8055094,"data_compressed_allocated":11661312,"data_compressed_original":38879232,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28766208,"data_stored":46966560,"data_compressed":5532241,"data_compressed_allocated":8105984,"data_compressed_original":26312704,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":385024,"data_stored":2519040,"data_compressed":113842,"data_compressed_allocated":229376,"data_compressed_original":2363392,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1032192,"data_stored":8355840,"data_compressed":386380,"data_compressed_allocated":716800,"data_compressed_original":8040448,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2228224,"data_stored":16973824,"data_compressed":779748,"data_compressed_allocated":1454080,"data_compressed_original":16199680,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1859584,"data_stored":12443648,"data_compressed":567456,"data_compressed_allocated":1142784,"data_compressed_original":11726848,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1040384,"data_stored":7540736,"data_compressed":344657,"data_compressed_allocated":704512,"data_compressed_original":7204864,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:00:52.420 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:00:52.556 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.556+0000 7fdab8ca5640 1 Processor -- start 2024-04-25T00:00:52.556 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.556+0000 7fdab8ca5640 1 -- start start 2024-04-25T00:00:52.557 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.557+0000 7fdab8ca5640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdab40c4b20 0x7fdab40c2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:52.557 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.557+0000 7fdab8ca5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdab414b720 0x7fdab414bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:52.557 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.557+0000 7fdab8ca5640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdab406d3c0 con 0x7fdab4073130 2024-04-25T00:00:52.557 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.557+0000 7fdab8ca5640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fdab406c400 con 0x7fdab414b720 2024-04-25T00:00:52.557 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.557+0000 7fdab8ca5640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fdab4073510 con 0x7fdab40c4b20 2024-04-25T00:00:52.557 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.557+0000 7fdab2d9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdab414b720 0x7fdab414bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:52.557 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.557+0000 7fdab2d9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdab414b720 0x7fdab414bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:52008/0 (socket says 172.21.15.157:52008) 2024-04-25T00:00:52.558 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.557+0000 7fdab2d9d640 1 -- 172.21.15.157:0/2226217167 learned_addr learned my addr 172.21.15.157:0/2226217167 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:00:52.558 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.557+0000 7fdab1d9b640 1 --2- 172.21.15.157:0/2226217167 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdab40c4b20 0x7fdab40c2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:52.558 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.557+0000 7fdab2d9d640 1 -- 172.21.15.157:0/2226217167 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdab40c4b20 msgr2=0x7fdab40c2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:52.558 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.557+0000 7fdab2d9d640 1 --2- 172.21.15.157:0/2226217167 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdab40c4b20 0x7fdab40c2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:52.558 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.557+0000 7fdab2d9d640 1 -- 172.21.15.157:0/2226217167 >> v1:172.21.15.169:6789/0 conn(0x7fdab4073130 legacy=0x7fdab406ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:52.558 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.557+0000 7fdab2d9d640 1 -- 172.21.15.157:0/2226217167 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdab406c6a0 con 0x7fdab414b720 2024-04-25T00:00:52.558 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.557+0000 7fdab159a640 1 -- 172.21.15.157:0/2226217167 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1453526703 0 0) 0x7fdab406d3c0 con 0x7fdab4073130 2024-04-25T00:00:52.559 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.558+0000 7fdab2d9d640 1 --2- 172.21.15.157:0/2226217167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdab414b720 0x7fdab414bbb0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fdaac009e50 tx=0x7fdaac02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b1e7d1830d47286 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:52.560 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.558+0000 7fdab159a640 1 -- 172.21.15.157:0/2226217167 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaac036070 con 0x7fdab414b720 2024-04-25T00:00:52.560 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.558+0000 7fdab159a640 1 -- 172.21.15.157:0/2226217167 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaac031600 con 0x7fdab414b720 2024-04-25T00:00:52.560 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.558+0000 7fdab159a640 1 -- 172.21.15.157:0/2226217167 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaac0319d0 con 0x7fdab414b720 2024-04-25T00:00:52.560 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.559+0000 7fdab8ca5640 1 -- 172.21.15.157:0/2226217167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdab414b720 msgr2=0x7fdab414bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:52.560 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.559+0000 7fdab8ca5640 1 --2- 172.21.15.157:0/2226217167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdab414b720 0x7fdab414bbb0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fdaac009e50 tx=0x7fdaac02a680 comp rx=0 tx=0).stop 2024-04-25T00:00:52.560 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.559+0000 7fdab8ca5640 1 -- 172.21.15.157:0/2226217167 shutdown_connections 2024-04-25T00:00:52.560 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.559+0000 7fdab8ca5640 1 --2- 172.21.15.157:0/2226217167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdab414b720 0x7fdab414bbb0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:52.560 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.559+0000 7fdab8ca5640 1 --2- 172.21.15.157:0/2226217167 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdab40c4b20 0x7fdab40c2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:52.560 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.559+0000 7fdab8ca5640 1 -- 172.21.15.157:0/2226217167 >> 172.21.15.157:0/2226217167 conn(0x7fdab40a8720 msgr2=0x7fdab40a8b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:52.560 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.559+0000 7fdab8ca5640 1 -- 172.21.15.157:0/2226217167 shutdown_connections 2024-04-25T00:00:52.560 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.559+0000 7fdab8ca5640 1 -- 172.21.15.157:0/2226217167 wait complete. 2024-04-25T00:00:52.560 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.560+0000 7fdab8ca5640 1 Processor -- start 2024-04-25T00:00:52.560 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.560+0000 7fdab8ca5640 1 -- start start 2024-04-25T00:00:52.563 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.561+0000 7fdab8ca5640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdab4073130 0x7fdab41ecbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:52.563 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.561+0000 7fdab8ca5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdab40c4b20 0x7fdab40a70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:52.563 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.561+0000 7fdab8ca5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdab414b720 0x7fdab40a9b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:52.563 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.561+0000 7fdab8ca5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdab40c2470 con 0x7fdab40c4b20 2024-04-25T00:00:52.564 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.561+0000 7fdab8ca5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdab40c2190 con 0x7fdab414b720 2024-04-25T00:00:52.564 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.561+0000 7fdab8ca5640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fdab414c0f0 con 0x7fdab4073130 2024-04-25T00:00:52.564 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.561+0000 7fdab1d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdab40c4b20 0x7fdab40a70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:52.564 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.561+0000 7fdab1d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdab40c4b20 0x7fdab40a70b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:52020/0 (socket says 172.21.15.157:52020) 2024-04-25T00:00:52.564 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.561+0000 7fdab1d9b640 1 -- 172.21.15.157:0/362225920 learned_addr learned my addr 172.21.15.157:0/362225920 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:00:52.564 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.561+0000 7fdab259c640 1 --2- 172.21.15.157:0/362225920 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdab4073130 0x7fdab41ecbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:52.564 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.561+0000 7fdab2d9d640 1 --2- 172.21.15.157:0/362225920 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdab414b720 0x7fdab40a9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:52.564 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.561+0000 7fdab1d9b640 1 -- 172.21.15.157:0/362225920 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdab4073130 msgr2=0x7fdab41ecbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:52.564 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.561+0000 7fdab1d9b640 1 --2- 172.21.15.157:0/362225920 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdab4073130 0x7fdab41ecbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:52.564 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.561+0000 7fdab1d9b640 1 -- 172.21.15.157:0/362225920 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdab414b720 msgr2=0x7fdab40a9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:52.565 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.561+0000 7fdab1d9b640 1 --2- 172.21.15.157:0/362225920 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdab414b720 0x7fdab40a9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:52.565 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.561+0000 7fdab1d9b640 1 -- 172.21.15.157:0/362225920 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdab40aa390 con 0x7fdab40c4b20 2024-04-25T00:00:52.565 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.561+0000 7fdab1d9b640 1 --2- 172.21.15.157:0/362225920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdab40c4b20 0x7fdab40a70b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdaa800e980 tx=0x7fdaa800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:52.565 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.561+0000 7fdaa2ffd640 1 -- 172.21.15.157:0/362225920 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa8017070 con 0x7fdab40c4b20 2024-04-25T00:00:52.565 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.562+0000 7fdaa2ffd640 1 -- 172.21.15.157:0/362225920 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaa800ca00 con 0x7fdab40c4b20 2024-04-25T00:00:52.565 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.562+0000 7fdaa2ffd640 1 -- 172.21.15.157:0/362225920 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa800cdd0 con 0x7fdab40c4b20 2024-04-25T00:00:52.565 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.562+0000 7fdab8ca5640 1 -- 172.21.15.157:0/362225920 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdab40aa670 con 0x7fdab40c4b20 2024-04-25T00:00:52.565 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.562+0000 7fdab8ca5640 1 -- 172.21.15.157:0/362225920 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdab40a75f0 con 0x7fdab40c4b20 2024-04-25T00:00:52.565 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.563+0000 7fdab8ca5640 1 -- 172.21.15.157:0/362225920 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdab40a5f20 con 0x7fdab40c4b20 2024-04-25T00:00:52.565 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.563+0000 7fdaa2ffd640 1 -- 172.21.15.157:0/362225920 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdaa801a070 con 0x7fdab40c4b20 2024-04-25T00:00:52.565 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.563+0000 7fdaa2ffd640 1 --2- 172.21.15.157:0/362225920 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fda9406be30 0x7fda9406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:52.565 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.564+0000 7fdaa2ffd640 1 -- 172.21.15.157:0/362225920 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fdaa808b4a0 con 0x7fdab40c4b20 2024-04-25T00:00:52.565 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.564+0000 7fdab259c640 1 --2- 172.21.15.157:0/362225920 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fda9406be30 0x7fda9406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:52.566 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.564+0000 7fdab259c640 1 --2- 172.21.15.157:0/362225920 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fda9406be30 0x7fda9406e2f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4000f60 tx=0x7fdaa4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:52.567 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.568+0000 7fdaa2ffd640 1 -- 172.21.15.157:0/362225920 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdab40a5f20 con 0x7fdab40c4b20 2024-04-25T00:00:52.742 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.741+0000 7fdab8ca5640 1 -- 172.21.15.157:0/362225920 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdab40681c0 con 0x7fdab40c4b20 2024-04-25T00:00:52.742 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.742+0000 7fdaa2ffd640 1 -- 172.21.15.157:0/362225920 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18928 (secure 0 0 0) 0x7fdaa8059490 con 0x7fdab40c4b20 2024-04-25T00:00:52.742 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-25T00:00:52.743 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":27,"fsid":"223685fb-5b64-46a5-9857-3791c9c07df2","created":"2024-04-24T23:34:37.504202+0000","modified":"2024-04-24T23:35:43.517316+0000","last_up_change":"2024-04-24T23:34:47.584963+0000","last_in_change":"2024-04-24T23:34:39.632954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:34:47.796655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:34:52.177208+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:34:52.667022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:34:56.395517+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"95d14ac8-fccf-4261-be8e-edafa746f629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6809","nonce":4003822160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6811","nonce":4003822160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6815","nonce":4003822160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6813","nonce":4003822160}]},"public_addr":"172.21.15.157:6809/4003822160","cluster_addr":"172.21.15.157:6811/4003822160","heartbeat_back_addr":"172.21.15.157:6815/4003822160","heartbeat_front_addr":"172.21.15.157:6813/4003822160","state":["exists","up"]},{"osd":1,"uuid":"7be79da9-8eec-4990-9178-71d602e39d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6817","nonce":1557575576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6819","nonce":1557575576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6823","nonce":1557575576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6821","nonce":1557575576}]},"public_addr":"172.21.15.157:6817/1557575576","cluster_addr":"172.21.15.157:6819/1557575576","heartbeat_back_addr":"172.21.15.157:6823/1557575576","heartbeat_front_addr":"172.21.15.157:6821/1557575576","state":["exists","up"]},{"osd":2,"uuid":"0127ed00-b7c3-466d-bfb0-01b0fd315f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6825","nonce":2793948092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6827","nonce":2793948092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6831","nonce":2793948092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6829","nonce":2793948092}]},"public_addr":"172.21.15.157:6825/2793948092","cluster_addr":"172.21.15.157:6827/2793948092","heartbeat_back_addr":"172.21.15.157:6831/2793948092","heartbeat_front_addr":"172.21.15.157:6829/2793948092","state":["exists","up"]},{"osd":3,"uuid":"f2331acc-aa37-433e-868e-3561b6039562","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6801","nonce":422758549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6803","nonce":422758549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6807","nonce":422758549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6805","nonce":422758549}]},"public_addr":"172.21.15.157:6801/422758549","cluster_addr":"172.21.15.157:6803/422758549","heartbeat_back_addr":"172.21.15.157:6807/422758549","heartbeat_front_addr":"172.21.15.157:6805/422758549","state":["exists","up"]},{"osd":4,"uuid":"1998854b-7398-43f5-ab7d-79fa0debcf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6801","nonce":321910099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6803","nonce":321910099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6807","nonce":321910099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6805","nonce":321910099}]},"public_addr":"172.21.15.169:6801/321910099","cluster_addr":"172.21.15.169:6803/321910099","heartbeat_back_addr":"172.21.15.169:6807/321910099","heartbeat_front_addr":"172.21.15.169:6805/321910099","state":["exists","up"]},{"osd":5,"uuid":"c37153ff-9b2b-48b9-a684-2c82b6f39f15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6825","nonce":243321711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6827","nonce":243321711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6831","nonce":243321711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6829","nonce":243321711}]},"public_addr":"172.21.15.169:6825/243321711","cluster_addr":"172.21.15.169:6827/243321711","heartbeat_back_addr":"172.21.15.169:6831/243321711","heartbeat_front_addr":"172.21.15.169:6829/243321711","state":["exists","up"]},{"osd":6,"uuid":"8db79e1c-f36b-49cb-8d8a-a20c2aec57c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6809","nonce":3441420785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6811","nonce":3441420785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6815","nonce":3441420785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6813","nonce":3441420785}]},"public_addr":"172.21.15.169:6809/3441420785","cluster_addr":"172.21.15.169:6811/3441420785","heartbeat_back_addr":"172.21.15.169:6815/3441420785","heartbeat_front_addr":"172.21.15.169:6813/3441420785","state":["exists","up"]},{"osd":7,"uuid":"928586e9-296c-4b8b-9b27-42f9f155dbdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6817","nonce":1663179020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6819","nonce":1663179020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6823","nonce":1663179020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6821","nonce":1663179020}]},"public_addr":"172.21.15.169:6817/1663179020","cluster_addr":"172.21.15.169:6819/1663179020","heartbeat_back_addr":"172.21.15.169:6823/1663179020","heartbeat_front_addr":"172.21.15.169:6821/1663179020","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.848039+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.826489+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.801120+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.727706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.824343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.791915+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.878073+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.810831+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.33","mappings":[{"from":3,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:00:52.744 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.744+0000 7fdab8ca5640 1 -- 172.21.15.157:0/362225920 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fda9406be30 msgr2=0x7fda9406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:52.744 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.744+0000 7fdab8ca5640 1 --2- 172.21.15.157:0/362225920 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fda9406be30 0x7fda9406e2f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4000f60 tx=0x7fdaa4009040 comp rx=0 tx=0).stop 2024-04-25T00:00:52.744 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.744+0000 7fdab8ca5640 1 -- 172.21.15.157:0/362225920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdab40c4b20 msgr2=0x7fdab40a70b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:52.744 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.744+0000 7fdab8ca5640 1 --2- 172.21.15.157:0/362225920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdab40c4b20 0x7fdab40a70b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdaa800e980 tx=0x7fdaa800ee40 comp rx=0 tx=0).stop 2024-04-25T00:00:52.744 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.744+0000 7fdab8ca5640 1 -- 172.21.15.157:0/362225920 shutdown_connections 2024-04-25T00:00:52.744 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.744+0000 7fdab8ca5640 1 --2- 172.21.15.157:0/362225920 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fda9406be30 0x7fda9406e2f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:52.745 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.744+0000 7fdab8ca5640 1 --2- 172.21.15.157:0/362225920 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdab414b720 0x7fdab40a9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:52.745 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.744+0000 7fdab8ca5640 1 --2- 172.21.15.157:0/362225920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdab40c4b20 0x7fdab40a70b0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:52.745 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.744+0000 7fdab8ca5640 1 --2- 172.21.15.157:0/362225920 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdab4073130 0x7fdab41ecbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:52.745 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.744+0000 7fdab8ca5640 1 -- 172.21.15.157:0/362225920 >> 172.21.15.157:0/362225920 conn(0x7fdab40a8720 msgr2=0x7fdab406d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:52.745 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.744+0000 7fdab8ca5640 1 -- 172.21.15.157:0/362225920 shutdown_connections 2024-04-25T00:00:52.745 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:52.744+0000 7fdab8ca5640 1 -- 172.21.15.157:0/362225920 wait complete. 2024-04-25T00:00:53.757 INFO:tasks.ceph:Scrubbing osd.0 2024-04-25T00:00:53.757 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-25T00:00:53.861 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.861+0000 7ffa3af63640 1 Processor -- start 2024-04-25T00:00:53.861 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.861+0000 7ffa3af63640 1 -- start start 2024-04-25T00:00:53.863 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.862+0000 7ffa3af63640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa3406d530 0x7ffa340b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:53.863 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.862+0000 7ffa3af63640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ffa340c7860 0x7ffa340c7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:53.863 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.862+0000 7ffa3af63640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa340c75f0 con 0x7ffa3414f3a0 2024-04-25T00:00:53.864 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.862+0000 7ffa3af63640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7ffa34155ef0 con 0x7ffa340c7860 2024-04-25T00:00:53.864 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.862+0000 7ffa3af63640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ffa341561d0 con 0x7ffa3406d530 2024-04-25T00:00:53.864 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.862+0000 7ffa39500640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ffa340c7860 0x7ffa340c7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:53.864 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.862+0000 7ffa38cff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7ffa3414f3a0 0x7ffa340afd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:32936/0 (socket says 172.21.15.157:32936) 2024-04-25T00:00:53.864 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.862+0000 7ffa39500640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ffa340c7860 0x7ffa340c7cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37616/0 (socket says 172.21.15.157:37616) 2024-04-25T00:00:53.864 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.862+0000 7ffa38cff640 1 -- 172.21.15.157:0/3102308428 learned_addr learned my addr 172.21.15.157:0/3102308428 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-25T00:00:53.864 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.862+0000 7ffa33fff640 1 --2- 172.21.15.157:0/3102308428 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa3406d530 0x7ffa340b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:53.864 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.862+0000 7ffa39500640 1 -- 172.21.15.157:0/3102308428 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa3406d530 msgr2=0x7ffa340b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:53.864 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.862+0000 7ffa39500640 1 --2- 172.21.15.157:0/3102308428 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa3406d530 0x7ffa340b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:53.864 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.862+0000 7ffa39500640 1 -- 172.21.15.157:0/3102308428 >> v1:172.21.15.157:6789/0 conn(0x7ffa3414f3a0 legacy=0x7ffa340afd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:53.864 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.862+0000 7ffa39500640 1 -- 172.21.15.157:0/3102308428 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa340c57d0 con 0x7ffa340c7860 2024-04-25T00:00:53.864 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.863+0000 7ffa39500640 1 --2- 172.21.15.157:0/3102308428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa340c7860 0x7ffa340c7cf0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c009d90 tx=0x7ffa2c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af8f2de903e4419f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:53.864 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.863+0000 7ffa337fe640 1 -- 172.21.15.157:0/3102308428 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c007740 con 0x7ffa340c7860 2024-04-25T00:00:53.865 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.863+0000 7ffa337fe640 1 -- 172.21.15.157:0/3102308428 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa2c0078e0 con 0x7ffa340c7860 2024-04-25T00:00:53.865 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.863+0000 7ffa337fe640 1 -- 172.21.15.157:0/3102308428 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c007cb0 con 0x7ffa340c7860 2024-04-25T00:00:53.865 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.864+0000 7ffa3af63640 1 -- 172.21.15.157:0/3102308428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa340c7860 msgr2=0x7ffa340c7cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:53.865 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.864+0000 7ffa3af63640 1 --2- 172.21.15.157:0/3102308428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa340c7860 0x7ffa340c7cf0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c009d90 tx=0x7ffa2c02a660 comp rx=0 tx=0).stop 2024-04-25T00:00:53.865 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.864+0000 7ffa3af63640 1 -- 172.21.15.157:0/3102308428 shutdown_connections 2024-04-25T00:00:53.865 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.864+0000 7ffa3af63640 1 --2- 172.21.15.157:0/3102308428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa340c7860 0x7ffa340c7cf0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:53.865 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.864+0000 7ffa3af63640 1 --2- 172.21.15.157:0/3102308428 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa3406d530 0x7ffa340b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:53.865 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.864+0000 7ffa3af63640 1 -- 172.21.15.157:0/3102308428 >> 172.21.15.157:0/3102308428 conn(0x7ffa34066ea0 msgr2=0x7ffa340672b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:53.865 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.864+0000 7ffa3af63640 1 -- 172.21.15.157:0/3102308428 shutdown_connections 2024-04-25T00:00:53.865 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.864+0000 7ffa3af63640 1 -- 172.21.15.157:0/3102308428 wait complete. 2024-04-25T00:00:53.869 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.865+0000 7ffa3af63640 1 Processor -- start 2024-04-25T00:00:53.869 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.865+0000 7ffa3af63640 1 -- start start 2024-04-25T00:00:53.869 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.865+0000 7ffa3af63640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa3406d530 0x7ffa341ee580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:53.869 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.865+0000 7ffa3af63640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa340c7860 0x7ffa340bed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:53.869 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.865+0000 7ffa3af63640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa3414f3a0 0x7ffa341eb640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:53.869 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.865+0000 7ffa3af63640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3414f780 con 0x7ffa3414f3a0 2024-04-25T00:00:53.869 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.865+0000 7ffa3af63640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ffa340c5d90 con 0x7ffa3406d530 2024-04-25T00:00:53.869 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.865+0000 7ffa3af63640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ffa340b0950 con 0x7ffa340c7860 2024-04-25T00:00:53.869 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.865+0000 7ffa39500640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa3414f3a0 0x7ffa341eb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:53.869 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.865+0000 7ffa39500640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa3414f3a0 0x7ffa341eb640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37622/0 (socket says 172.21.15.157:37622) 2024-04-25T00:00:53.869 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.866+0000 7ffa39500640 1 -- 172.21.15.157:0/3272575220 learned_addr learned my addr 172.21.15.157:0/3272575220 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:00:53.870 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.866+0000 7ffa38cff640 1 --2- 172.21.15.157:0/3272575220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa3406d530 0x7ffa341ee580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:53.870 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.866+0000 7ffa33fff640 1 --2- 172.21.15.157:0/3272575220 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa340c7860 0x7ffa340bed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:53.870 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.866+0000 7ffa39500640 1 -- 172.21.15.157:0/3272575220 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa340c7860 msgr2=0x7ffa340bed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:53.870 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.866+0000 7ffa39500640 1 --2- 172.21.15.157:0/3272575220 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa340c7860 0x7ffa340bed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:53.870 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.866+0000 7ffa39500640 1 -- 172.21.15.157:0/3272575220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa3406d530 msgr2=0x7ffa341ee580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:53.870 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.866+0000 7ffa39500640 1 --2- 172.21.15.157:0/3272575220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa3406d530 0x7ffa341ee580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:53.870 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.866+0000 7ffa39500640 1 -- 172.21.15.157:0/3272575220 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa341eeb60 con 0x7ffa3414f3a0 2024-04-25T00:00:53.870 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.866+0000 7ffa39500640 1 --2- 172.21.15.157:0/3272575220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa3414f3a0 0x7ffa341eb640 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c009eb0 tx=0x7ffa2c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:53.870 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.866+0000 7ffa317fa640 1 -- 172.21.15.157:0/3272575220 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c040070 con 0x7ffa3414f3a0 2024-04-25T00:00:53.870 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.866+0000 7ffa317fa640 1 -- 172.21.15.157:0/3272575220 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa2c031bc0 con 0x7ffa3414f3a0 2024-04-25T00:00:53.870 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.866+0000 7ffa3af63640 1 -- 172.21.15.157:0/3272575220 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa341eee40 con 0x7ffa3414f3a0 2024-04-25T00:00:53.871 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.866+0000 7ffa317fa640 1 -- 172.21.15.157:0/3272575220 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c009450 con 0x7ffa3414f3a0 2024-04-25T00:00:53.871 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.866+0000 7ffa3af63640 1 -- 172.21.15.157:0/3272575220 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa341ef400 con 0x7ffa3414f3a0 2024-04-25T00:00:53.871 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.867+0000 7ffa3af63640 1 -- 172.21.15.157:0/3272575220 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffa340681c0 con 0x7ffa3414f3a0 2024-04-25T00:00:53.871 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.868+0000 7ffa317fa640 1 -- 172.21.15.157:0/3272575220 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffa2c039040 con 0x7ffa3414f3a0 2024-04-25T00:00:53.871 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.868+0000 7ffa317fa640 1 --2- 172.21.15.157:0/3272575220 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7ffa1806be30 0x7ffa1806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:53.871 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.868+0000 7ffa317fa640 1 -- 172.21.15.157:0/3272575220 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7ffa2c0acd40 con 0x7ffa3414f3a0 2024-04-25T00:00:53.871 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.868+0000 7ffa38cff640 1 --2- 172.21.15.157:0/3272575220 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7ffa1806be30 0x7ffa1806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:53.871 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.868+0000 7ffa317fa640 1 --2- 172.21.15.157:0/3272575220 >> [v2:172.21.15.157:6808/4003822160,v1:172.21.15.157:6809/4003822160] conn(0x7ffa18075840 0x7ffa18077ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:53.871 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.868+0000 7ffa317fa640 1 -- 172.21.15.157:0/3272575220 --> [v2:172.21.15.157:6808/4003822160,v1:172.21.15.157:6809/4003822160] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffa2c00ae60 con 0x7ffa18075840 2024-04-25T00:00:53.871 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.868+0000 7ffa317fa640 1 -- 172.21.15.157:0/3272575220 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ffa2c0ad1d0 con 0x7ffa3414f3a0 2024-04-25T00:00:53.871 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.869+0000 7ffa33fff640 1 --2- 172.21.15.157:0/3272575220 >> [v2:172.21.15.157:6808/4003822160,v1:172.21.15.157:6809/4003822160] conn(0x7ffa18075840 0x7ffa18077ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:53.871 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.869+0000 7ffa33fff640 1 --2- 172.21.15.157:0/3272575220 >> [v2:172.21.15.157:6808/4003822160,v1:172.21.15.157:6809/4003822160] conn(0x7ffa18075840 0x7ffa18077ce0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:53.871 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.869+0000 7ffa38cff640 1 --2- 172.21.15.157:0/3272575220 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7ffa1806be30 0x7ffa1806e2f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ffa28007fd0 tx=0x7ffa28007e80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:53.871 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.870+0000 7ffa317fa640 1 -- 172.21.15.157:0/3272575220 <== osd.0 v2:172.21.15.157:6808/4003822160 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffa2c00ae60 con 0x7ffa18075840 2024-04-25T00:00:53.882 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.883+0000 7ffa3af63640 1 -- 172.21.15.157:0/3272575220 --> [v2:172.21.15.157:6808/4003822160,v1:172.21.15.157:6809/4003822160] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7ffa3406d310 con 0x7ffa18075840 2024-04-25T00:00:53.883 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.883+0000 7ffa317fa640 1 -- 172.21.15.157:0/3272575220 <== osd.0 v2:172.21.15.157:6808/4003822160 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7ffa3406d310 con 0x7ffa18075840 2024-04-25T00:00:53.883 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-04-25T00:00:53.883 INFO:teuthology.orchestra.run.smithi157.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T00:00:53.883 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-04-25T00:00:53.884 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.883+0000 7ffa3af63640 1 -- 172.21.15.157:0/3272575220 >> [v2:172.21.15.157:6808/4003822160,v1:172.21.15.157:6809/4003822160] conn(0x7ffa18075840 msgr2=0x7ffa18077ce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:53.884 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.883+0000 7ffa3af63640 1 --2- 172.21.15.157:0/3272575220 >> [v2:172.21.15.157:6808/4003822160,v1:172.21.15.157:6809/4003822160] conn(0x7ffa18075840 0x7ffa18077ce0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:53.884 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.883+0000 7ffa3af63640 1 -- 172.21.15.157:0/3272575220 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7ffa1806be30 msgr2=0x7ffa1806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:53.884 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.883+0000 7ffa3af63640 1 --2- 172.21.15.157:0/3272575220 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7ffa1806be30 0x7ffa1806e2f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ffa28007fd0 tx=0x7ffa28007e80 comp rx=0 tx=0).stop 2024-04-25T00:00:53.884 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.883+0000 7ffa3af63640 1 -- 172.21.15.157:0/3272575220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa3414f3a0 msgr2=0x7ffa341eb640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:53.884 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.883+0000 7ffa3af63640 1 --2- 172.21.15.157:0/3272575220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa3414f3a0 0x7ffa341eb640 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c009eb0 tx=0x7ffa2c004a40 comp rx=0 tx=0).stop 2024-04-25T00:00:53.884 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.883+0000 7ffa39500640 1 -- 172.21.15.157:0/3272575220 reap_dead start 2024-04-25T00:00:53.884 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.883+0000 7ffa3af63640 1 -- 172.21.15.157:0/3272575220 shutdown_connections 2024-04-25T00:00:53.884 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.883+0000 7ffa3af63640 1 -- 172.21.15.157:0/3272575220 >> 172.21.15.157:0/3272575220 conn(0x7ffa34066ea0 msgr2=0x7ffa34067280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:53.885 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.883+0000 7ffa3af63640 1 -- 172.21.15.157:0/3272575220 shutdown_connections 2024-04-25T00:00:53.885 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.883+0000 7ffa3af63640 1 -- 172.21.15.157:0/3272575220 wait complete. 2024-04-25T00:00:53.894 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-25T00:00:53.993 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.994+0000 7f1921bdc640 1 Processor -- start 2024-04-25T00:00:53.994 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.994+0000 7f1921bdc640 1 -- start start 2024-04-25T00:00:53.996 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.994+0000 7f1921bdc640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f191c0c68d0 0x7f191c0c6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:53.997 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.995+0000 7f1921bdc640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f191c06bd80 con 0x7f191c0c58a0 2024-04-25T00:00:53.997 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.995+0000 7f1921bdc640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f191c1561a0 con 0x7f191c0c71f0 2024-04-25T00:00:53.997 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.995+0000 7f1921bdc640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f191c0c6730 con 0x7f191c0c68d0 2024-04-25T00:00:53.997 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.995+0000 7f191bfff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f191c0c58a0 0x7f191c15fe40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:32944/0 (socket says 172.21.15.157:32944) 2024-04-25T00:00:53.997 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.995+0000 7f191bfff640 1 -- 172.21.15.157:0/3442853413 learned_addr learned my addr 172.21.15.157:0/3442853413 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-25T00:00:53.997 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.995+0000 7f191b7fe640 1 --2- 172.21.15.157:0/3442853413 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f191c0c68d0 0x7f191c0c6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:53.997 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.995+0000 7f191a7fc640 1 -- 172.21.15.157:0/3442853413 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1869951930 0 0) 0x7f191c06bd80 con 0x7f191c0c58a0 2024-04-25T00:00:53.997 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.995+0000 7f191a7fc640 1 -- 172.21.15.157:0/3442853413 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1904003520 con 0x7f191c0c58a0 2024-04-25T00:00:53.997 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.995+0000 7f191a7fc640 1 -- 172.21.15.157:0/3442853413 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2292490337 0 0) 0x7f191c1561a0 con 0x7f191c0c71f0 2024-04-25T00:00:53.997 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.995+0000 7f191a7fc640 1 -- 172.21.15.157:0/3442853413 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f191c06bd80 con 0x7f191c0c71f0 2024-04-25T00:00:53.997 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.995+0000 7f191b7fe640 1 -- 172.21.15.157:0/3442853413 >> v1:172.21.15.169:6789/0 conn(0x7f191c0c71f0 legacy=0x7f191c0c8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:53.997 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.995+0000 7f191b7fe640 1 -- 172.21.15.157:0/3442853413 >> v1:172.21.15.157:6789/0 conn(0x7f191c0c58a0 legacy=0x7f191c15fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:53.998 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.995+0000 7f191b7fe640 1 -- 172.21.15.157:0/3442853413 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f191c0c6370 con 0x7f191c0c68d0 2024-04-25T00:00:53.998 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.996+0000 7f191b7fe640 1 --2- 172.21.15.157:0/3442853413 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f191c0c68d0 0x7f191c0c6cb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f190c0097d0 tx=0x7f190c02a290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=817727fe65ead9c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:53.998 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.996+0000 7f191a7fc640 1 -- 172.21.15.157:0/3442853413 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190c037070 con 0x7f191c0c68d0 2024-04-25T00:00:53.998 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.996+0000 7f191a7fc640 1 -- 172.21.15.157:0/3442853413 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f190c032440 con 0x7f191c0c68d0 2024-04-25T00:00:53.998 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.996+0000 7f191a7fc640 1 -- 172.21.15.157:0/3442853413 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190c0327f0 con 0x7f191c0c68d0 2024-04-25T00:00:53.998 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.996+0000 7f1921bdc640 1 -- 172.21.15.157:0/3442853413 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f191c0c68d0 msgr2=0x7f191c0c6cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:53.998 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.996+0000 7f1921bdc640 1 --2- 172.21.15.157:0/3442853413 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f191c0c68d0 0x7f191c0c6cb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f190c0097d0 tx=0x7f190c02a290 comp rx=0 tx=0).stop 2024-04-25T00:00:53.998 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.997+0000 7f1921bdc640 1 -- 172.21.15.157:0/3442853413 shutdown_connections 2024-04-25T00:00:53.998 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.997+0000 7f1921bdc640 1 --2- 172.21.15.157:0/3442853413 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f191c0c68d0 0x7f191c0c6cb0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:53.998 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.997+0000 7f1921bdc640 1 -- 172.21.15.157:0/3442853413 >> 172.21.15.157:0/3442853413 conn(0x7f191c0caca0 msgr2=0x7f191c0cb0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:53.998 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.997+0000 7f1921bdc640 1 -- 172.21.15.157:0/3442853413 shutdown_connections 2024-04-25T00:00:53.999 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.997+0000 7f1921bdc640 1 -- 172.21.15.157:0/3442853413 wait complete. 2024-04-25T00:00:54.001 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.998+0000 7f1921bdc640 1 Processor -- start 2024-04-25T00:00:54.001 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.998+0000 7f1921bdc640 1 -- start start 2024-04-25T00:00:54.002 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.998+0000 7f1921bdc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f191c0c58a0 0x7f191c1ec730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.002 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.998+0000 7f1921bdc640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f191c0c68d0 0x7f191c154800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.002 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.998+0000 7f1921bdc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f191c0c71f0 0x7f191c1ebaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.002 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.998+0000 7f1921bdc640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f191c152ed0 con 0x7f191c0c71f0 2024-04-25T00:00:54.002 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.998+0000 7f1921bdc640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f191c06b830 con 0x7f191c0c58a0 2024-04-25T00:00:54.002 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.998+0000 7f1921bdc640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f191c153050 con 0x7f191c0c68d0 2024-04-25T00:00:54.002 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.998+0000 7f191bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f191c0c71f0 0x7f191c1ebaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.002 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.998+0000 7f191bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f191c0c71f0 0x7f191c1ebaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37630/0 (socket says 172.21.15.157:37630) 2024-04-25T00:00:54.002 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.998+0000 7f191bfff640 1 -- 172.21.15.157:0/3911985685 learned_addr learned my addr 172.21.15.157:0/3911985685 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:00:54.002 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.999+0000 7f191b7fe640 1 --2- 172.21.15.157:0/3911985685 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f191c0c58a0 0x7f191c1ec730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.002 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.999+0000 7f191affd640 1 --2- 172.21.15.157:0/3911985685 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f191c0c68d0 0x7f191c154800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.002 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.999+0000 7f191bfff640 1 -- 172.21.15.157:0/3911985685 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f191c0c68d0 msgr2=0x7f191c154800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:54.003 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.999+0000 7f191bfff640 1 --2- 172.21.15.157:0/3911985685 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f191c0c68d0 0x7f191c154800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.003 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.999+0000 7f191bfff640 1 -- 172.21.15.157:0/3911985685 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f191c0c58a0 msgr2=0x7f191c1ec730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:54.003 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.999+0000 7f191bfff640 1 --2- 172.21.15.157:0/3911985685 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f191c0c58a0 0x7f191c1ec730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.003 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.999+0000 7f191bfff640 1 -- 172.21.15.157:0/3911985685 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f191c148c60 con 0x7f191c0c71f0 2024-04-25T00:00:54.003 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.999+0000 7f191b7fe640 1 --2- 172.21.15.157:0/3911985685 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f191c0c58a0 0x7f191c1ec730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:00:54.003 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.999+0000 7f191bfff640 1 --2- 172.21.15.157:0/3911985685 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f191c0c71f0 0x7f191c1ebaa0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f191400cad0 tx=0x7f191400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:54.003 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.999+0000 7f1903fff640 1 -- 172.21.15.157:0/3911985685 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1914017070 con 0x7f191c0c71f0 2024-04-25T00:00:54.003 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.999+0000 7f1903fff640 1 -- 172.21.15.157:0/3911985685 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f191400abf0 con 0x7f191c0c71f0 2024-04-25T00:00:54.003 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.999+0000 7f1921bdc640 1 -- 172.21.15.157:0/3911985685 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f191c1eb2c0 con 0x7f191c0c71f0 2024-04-25T00:00:54.003 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.999+0000 7f1903fff640 1 -- 172.21.15.157:0/3911985685 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1914012400 con 0x7f191c0c71f0 2024-04-25T00:00:54.003 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:53.999+0000 7f1921bdc640 1 -- 172.21.15.157:0/3911985685 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f191c1ef5c0 con 0x7f191c0c71f0 2024-04-25T00:00:54.003 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.000+0000 7f1921bdc640 1 -- 172.21.15.157:0/3911985685 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f191c0681c0 con 0x7f191c0c71f0 2024-04-25T00:00:54.003 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.001+0000 7f1903fff640 1 -- 172.21.15.157:0/3911985685 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f191401a070 con 0x7f191c0c71f0 2024-04-25T00:00:54.003 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.001+0000 7f1903fff640 1 --2- 172.21.15.157:0/3911985685 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f190806be60 0x7f190806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.004 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.001+0000 7f1903fff640 1 -- 172.21.15.157:0/3911985685 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f191408ae70 con 0x7f191c0c71f0 2024-04-25T00:00:54.004 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.002+0000 7f191b7fe640 1 --2- 172.21.15.157:0/3911985685 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f190806be60 0x7f190806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.004 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.002+0000 7f191b7fe640 1 --2- 172.21.15.157:0/3911985685 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f190806be60 0x7f190806e320 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f190c034000 tx=0x7f190c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:54.005 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.006+0000 7f1903fff640 1 -- 172.21.15.157:0/3911985685 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1914058e60 con 0x7f191c0c71f0 2024-04-25T00:00:54.186 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.186+0000 7f1921bdc640 1 -- 172.21.15.157:0/3911985685 --> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f191c0c5680 con 0x7f190806be60 2024-04-25T00:00:54.187 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.187+0000 7f1903fff640 1 -- 172.21.15.157:0/3911985685 <== mgr.4105 v2:172.21.15.169:6832/662378289 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f191c0c5680 con 0x7f190806be60 2024-04-25T00:00:54.187 INFO:teuthology.orchestra.run.smithi157.stderr:instructed osd(s) 0 to deep-scrub 2024-04-25T00:00:54.189 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.189+0000 7f1921bdc640 1 -- 172.21.15.157:0/3911985685 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f190806be60 msgr2=0x7f190806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:54.189 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.189+0000 7f1921bdc640 1 --2- 172.21.15.157:0/3911985685 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f190806be60 0x7f190806e320 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f190c034000 tx=0x7f190c02e000 comp rx=0 tx=0).stop 2024-04-25T00:00:54.189 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.189+0000 7f1921bdc640 1 -- 172.21.15.157:0/3911985685 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f191c0c71f0 msgr2=0x7f191c1ebaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:54.189 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.189+0000 7f1921bdc640 1 --2- 172.21.15.157:0/3911985685 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f191c0c71f0 0x7f191c1ebaa0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f191400cad0 tx=0x7f191400cf90 comp rx=0 tx=0).stop 2024-04-25T00:00:54.189 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.189+0000 7f1921bdc640 1 -- 172.21.15.157:0/3911985685 shutdown_connections 2024-04-25T00:00:54.189 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.189+0000 7f1921bdc640 1 --2- 172.21.15.157:0/3911985685 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f190806be60 0x7f190806e320 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.189 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.189+0000 7f1921bdc640 1 --2- 172.21.15.157:0/3911985685 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f191c0c71f0 0x7f191c1ebaa0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.189 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.189+0000 7f1921bdc640 1 --2- 172.21.15.157:0/3911985685 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f191c0c68d0 0x7f191c154800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.190 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.189+0000 7f1921bdc640 1 --2- 172.21.15.157:0/3911985685 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f191c0c58a0 0x7f191c1ec730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.190 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.189+0000 7f1921bdc640 1 -- 172.21.15.157:0/3911985685 >> 172.21.15.157:0/3911985685 conn(0x7f191c0caca0 msgr2=0x7f191c14bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:54.190 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.189+0000 7f1921bdc640 1 -- 172.21.15.157:0/3911985685 shutdown_connections 2024-04-25T00:00:54.190 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.189+0000 7f1921bdc640 1 -- 172.21.15.157:0/3911985685 wait complete. 2024-04-25T00:00:54.200 INFO:tasks.ceph:Scrubbing osd.1 2024-04-25T00:00:54.200 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-25T00:00:54.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.304+0000 7f19dbb84640 1 Processor -- start 2024-04-25T00:00:54.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.305+0000 7f19dbb84640 1 -- start start 2024-04-25T00:00:54.306 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.305+0000 7f19dbb84640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19d414f3a0 0x7f19d406d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.306 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.305+0000 7f19dbb84640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f19d40afd00 0x7f19d40b0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.306 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.305+0000 7f19dbb84640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f19d40b06a0 0x7f19d41589c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.306 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.305+0000 7f19dbb84640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f19d40c75f0 con 0x7f19d40afd00 2024-04-25T00:00:54.306 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.305+0000 7f19dbb84640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f19d4155ef0 con 0x7f19d40b06a0 2024-04-25T00:00:54.306 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.305+0000 7f19dbb84640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f19d41561d0 con 0x7f19d414f3a0 2024-04-25T00:00:54.306 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.305+0000 7f19d911f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f19d40afd00 0x7f19d40b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.306 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.305+0000 7f19d911f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f19d40afd00 0x7f19d40b0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37640/0 (socket says 172.21.15.157:37640) 2024-04-25T00:00:54.306 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.305+0000 7f19d911f640 1 -- 172.21.15.157:0/2740200071 learned_addr learned my addr 172.21.15.157:0/2740200071 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:00:54.306 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.305+0000 7f19d9920640 1 --2- 172.21.15.157:0/2740200071 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19d414f3a0 0x7f19d406d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.306 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.306+0000 7f19da121640 1 --2- 172.21.15.157:0/2740200071 >> v2:172.21.15.169:3300/0 conn(0x7f19d40b06a0 0x7f19d41589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.307 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.306+0000 7f19d911f640 1 -- 172.21.15.157:0/2740200071 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19d414f3a0 msgr2=0x7f19d406d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:54.307 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.306+0000 7f19d911f640 1 --2- 172.21.15.157:0/2740200071 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19d414f3a0 0x7f19d406d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.307 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.306+0000 7f19d911f640 1 -- 172.21.15.157:0/2740200071 >> v2:172.21.15.169:3300/0 conn(0x7f19d40b06a0 msgr2=0x7f19d41589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:54.307 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.306+0000 7f19d911f640 1 --2- 172.21.15.157:0/2740200071 >> v2:172.21.15.169:3300/0 conn(0x7f19d40b06a0 0x7f19d41589c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.307 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.306+0000 7f19d911f640 1 -- 172.21.15.157:0/2740200071 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19d40c57d0 con 0x7f19d40afd00 2024-04-25T00:00:54.311 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.307+0000 7f19d911f640 1 --2- 172.21.15.157:0/2740200071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19d40afd00 0x7f19d40b0160 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f19d40cba70 tx=0x7f19d002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1b4b6f2d7de821 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:54.311 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.307+0000 7f19d891e640 1 -- 172.21.15.157:0/2740200071 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d0037070 con 0x7f19d40afd00 2024-04-25T00:00:54.311 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.307+0000 7f19d891e640 1 -- 172.21.15.157:0/2740200071 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19d0032440 con 0x7f19d40afd00 2024-04-25T00:00:54.311 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.307+0000 7f19d891e640 1 -- 172.21.15.157:0/2740200071 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d0032810 con 0x7f19d40afd00 2024-04-25T00:00:54.312 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.307+0000 7f19dbb84640 1 -- 172.21.15.157:0/2740200071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19d40afd00 msgr2=0x7f19d40b0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:54.312 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.307+0000 7f19dbb84640 1 --2- 172.21.15.157:0/2740200071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19d40afd00 0x7f19d40b0160 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f19d40cba70 tx=0x7f19d002a1e0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.312 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.308+0000 7f19dbb84640 1 -- 172.21.15.157:0/2740200071 shutdown_connections 2024-04-25T00:00:54.312 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.308+0000 7f19dbb84640 1 --2- 172.21.15.157:0/2740200071 >> v2:172.21.15.169:3300/0 conn(0x7f19d40b06a0 0x7f19d41589c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.312 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.308+0000 7f19dbb84640 1 --2- 172.21.15.157:0/2740200071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19d40afd00 0x7f19d40b0160 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.312 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.308+0000 7f19dbb84640 1 --2- 172.21.15.157:0/2740200071 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19d414f3a0 0x7f19d406d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.312 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.308+0000 7f19dbb84640 1 -- 172.21.15.157:0/2740200071 >> 172.21.15.157:0/2740200071 conn(0x7f19d4066ea0 msgr2=0x7f19d40672b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:54.312 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.308+0000 7f19dbb84640 1 -- 172.21.15.157:0/2740200071 shutdown_connections 2024-04-25T00:00:54.312 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.308+0000 7f19dbb84640 1 -- 172.21.15.157:0/2740200071 wait complete. 2024-04-25T00:00:54.313 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.309+0000 7f19dbb84640 1 Processor -- start 2024-04-25T00:00:54.313 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.309+0000 7f19dbb84640 1 -- start start 2024-04-25T00:00:54.313 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.309+0000 7f19dbb84640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19d40afd00 0x7f19d414dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.313 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.309+0000 7f19dbb84640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19d40b06a0 0x7f19d414e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.313 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.309+0000 7f19dbb84640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19d414f3a0 0x7f19d41ee7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.313 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.309+0000 7f19dbb84640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f19d414f780 con 0x7f19d40afd00 2024-04-25T00:00:54.313 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.309+0000 7f19dbb84640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f19d40c5d90 con 0x7f19d40b06a0 2024-04-25T00:00:54.313 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.309+0000 7f19dbb84640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f19d4158f00 con 0x7f19d414f3a0 2024-04-25T00:00:54.313 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.309+0000 7f19d9920640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19d40afd00 0x7f19d414dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.313 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.310+0000 7f19d9920640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19d40afd00 0x7f19d414dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37644/0 (socket says 172.21.15.157:37644) 2024-04-25T00:00:54.314 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.310+0000 7f19d9920640 1 -- 172.21.15.157:0/2657662778 learned_addr learned my addr 172.21.15.157:0/2657662778 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:00:54.314 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.310+0000 7f19da121640 1 --2- 172.21.15.157:0/2657662778 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19d414f3a0 0x7f19d41ee7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.314 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.310+0000 7f19d911f640 1 --2- 172.21.15.157:0/2657662778 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19d40b06a0 0x7f19d414e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.314 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.310+0000 7f19d9920640 1 -- 172.21.15.157:0/2657662778 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19d414f3a0 msgr2=0x7f19d41ee7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:54.314 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.310+0000 7f19d9920640 1 --2- 172.21.15.157:0/2657662778 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19d414f3a0 0x7f19d41ee7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.314 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.310+0000 7f19d9920640 1 -- 172.21.15.157:0/2657662778 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19d40b06a0 msgr2=0x7f19d414e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:54.314 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.310+0000 7f19d9920640 1 --2- 172.21.15.157:0/2657662778 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19d40b06a0 0x7f19d414e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.314 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.310+0000 7f19d9920640 1 -- 172.21.15.157:0/2657662778 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19d40c3d20 con 0x7f19d40afd00 2024-04-25T00:00:54.314 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.310+0000 7f19d9920640 1 --2- 172.21.15.157:0/2657662778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19d40afd00 0x7f19d414dc40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f19c400e8a0 tx=0x7f19c400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:54.314 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.310+0000 7f19ca7fc640 1 -- 172.21.15.157:0/2657662778 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c4017070 con 0x7f19d40afd00 2024-04-25T00:00:54.314 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.310+0000 7f19ca7fc640 1 -- 172.21.15.157:0/2657662778 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19c400c940 con 0x7f19d40afd00 2024-04-25T00:00:54.314 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.310+0000 7f19ca7fc640 1 -- 172.21.15.157:0/2657662778 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c400cd30 con 0x7f19d40afd00 2024-04-25T00:00:54.314 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.310+0000 7f19dbb84640 1 -- 172.21.15.157:0/2657662778 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19d40c4000 con 0x7f19d40afd00 2024-04-25T00:00:54.315 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.310+0000 7f19dbb84640 1 -- 172.21.15.157:0/2657662778 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19d40c4f80 con 0x7f19d40afd00 2024-04-25T00:00:54.315 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.311+0000 7f19dbb84640 1 -- 172.21.15.157:0/2657662778 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f19d40681c0 con 0x7f19d40afd00 2024-04-25T00:00:54.315 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.312+0000 7f19ca7fc640 1 -- 172.21.15.157:0/2657662778 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f19c401a070 con 0x7f19d40afd00 2024-04-25T00:00:54.316 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.312+0000 7f19ca7fc640 1 --2- 172.21.15.157:0/2657662778 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f19b806be30 0x7f19b806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.316 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.312+0000 7f19ca7fc640 1 -- 172.21.15.157:0/2657662778 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f19c408b370 con 0x7f19d40afd00 2024-04-25T00:00:54.316 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.312+0000 7f19d911f640 1 --2- 172.21.15.157:0/2657662778 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f19b806be30 0x7f19b806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.316 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.312+0000 7f19ca7fc640 1 --2- 172.21.15.157:0/2657662778 >> [v2:172.21.15.157:6816/1557575576,v1:172.21.15.157:6817/1557575576] conn(0x7f19b8075850 0x7f19b8077cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.316 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.312+0000 7f19ca7fc640 1 -- 172.21.15.157:0/2657662778 --> [v2:172.21.15.157:6816/1557575576,v1:172.21.15.157:6817/1557575576] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f19c4016ec0 con 0x7f19b8075850 2024-04-25T00:00:54.316 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.312+0000 7f19ca7fc640 1 -- 172.21.15.157:0/2657662778 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f19c4092050 con 0x7f19d40afd00 2024-04-25T00:00:54.316 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.312+0000 7f19da121640 1 --2- 172.21.15.157:0/2657662778 >> [v2:172.21.15.157:6816/1557575576,v1:172.21.15.157:6817/1557575576] conn(0x7f19b8075850 0x7f19b8077cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.317 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.313+0000 7f19d911f640 1 --2- 172.21.15.157:0/2657662778 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f19b806be30 0x7f19b806e2f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f19d0034000 tx=0x7f19d002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:54.317 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.313+0000 7f19da121640 1 --2- 172.21.15.157:0/2657662778 >> [v2:172.21.15.157:6816/1557575576,v1:172.21.15.157:6817/1557575576] conn(0x7f19b8075850 0x7f19b8077cf0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:54.317 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.313+0000 7f19ca7fc640 1 -- 172.21.15.157:0/2657662778 <== osd.1 v2:172.21.15.157:6816/1557575576 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f19c4016ec0 con 0x7f19b8075850 2024-04-25T00:00:54.326 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.327+0000 7f19dbb84640 1 -- 172.21.15.157:0/2657662778 --> [v2:172.21.15.157:6816/1557575576,v1:172.21.15.157:6817/1557575576] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f19d414e930 con 0x7f19b8075850 2024-04-25T00:00:54.326 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.327+0000 7f19ca7fc640 1 -- 172.21.15.157:0/2657662778 <== osd.1 v2:172.21.15.157:6816/1557575576 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f19d414e930 con 0x7f19b8075850 2024-04-25T00:00:54.327 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-04-25T00:00:54.327 INFO:teuthology.orchestra.run.smithi157.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T00:00:54.327 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-04-25T00:00:54.328 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.327+0000 7f19dbb84640 1 -- 172.21.15.157:0/2657662778 >> [v2:172.21.15.157:6816/1557575576,v1:172.21.15.157:6817/1557575576] conn(0x7f19b8075850 msgr2=0x7f19b8077cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:54.328 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.327+0000 7f19dbb84640 1 --2- 172.21.15.157:0/2657662778 >> [v2:172.21.15.157:6816/1557575576,v1:172.21.15.157:6817/1557575576] conn(0x7f19b8075850 0x7f19b8077cf0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.328 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.327+0000 7f19dbb84640 1 -- 172.21.15.157:0/2657662778 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f19b806be30 msgr2=0x7f19b806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:54.328 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.327+0000 7f19dbb84640 1 --2- 172.21.15.157:0/2657662778 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f19b806be30 0x7f19b806e2f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f19d0034000 tx=0x7f19d002e000 comp rx=0 tx=0).stop 2024-04-25T00:00:54.328 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.327+0000 7f19dbb84640 1 -- 172.21.15.157:0/2657662778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19d40afd00 msgr2=0x7f19d414dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:54.328 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.327+0000 7f19dbb84640 1 --2- 172.21.15.157:0/2657662778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19d40afd00 0x7f19d414dc40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f19c400e8a0 tx=0x7f19c400ed60 comp rx=0 tx=0).stop 2024-04-25T00:00:54.328 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.327+0000 7f19da121640 1 -- 172.21.15.157:0/2657662778 reap_dead start 2024-04-25T00:00:54.328 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.327+0000 7f19dbb84640 1 -- 172.21.15.157:0/2657662778 shutdown_connections 2024-04-25T00:00:54.328 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.327+0000 7f19dbb84640 1 -- 172.21.15.157:0/2657662778 >> 172.21.15.157:0/2657662778 conn(0x7f19d4066ea0 msgr2=0x7f19d4155440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:54.328 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.327+0000 7f19dbb84640 1 -- 172.21.15.157:0/2657662778 shutdown_connections 2024-04-25T00:00:54.328 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.327+0000 7f19dbb84640 1 -- 172.21.15.157:0/2657662778 wait complete. 2024-04-25T00:00:54.337 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-25T00:00:54.440 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.438+0000 7f701d5b0640 1 Processor -- start 2024-04-25T00:00:54.440 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.438+0000 7f701d5b0640 1 -- start start 2024-04-25T00:00:54.440 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.439+0000 7f701d5b0640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f70180c68d0 0x7f70180c6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.440 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.439+0000 7f701d5b0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f70180c71f0 0x7f70180c8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.441 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.439+0000 7f701d5b0640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70180c58a0 0x7f70180c90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.441 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.439+0000 7f701d5b0640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f70180c6730 con 0x7f70180c71f0 2024-04-25T00:00:54.441 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.439+0000 7f701d5b0640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f70180c9620 con 0x7f70180c68d0 2024-04-25T00:00:54.441 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.439+0000 7f701d5b0640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f701806bd80 con 0x7f70180c58a0 2024-04-25T00:00:54.441 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.439+0000 7f700e7fc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f70180c71f0 0x7f70180c8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.441 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.439+0000 7f700e7fc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f70180c71f0 0x7f70180c8ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37650/0 (socket says 172.21.15.157:37650) 2024-04-25T00:00:54.441 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.439+0000 7f700e7fc640 1 -- 172.21.15.157:0/82555124 learned_addr learned my addr 172.21.15.157:0/82555124 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:00:54.441 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.439+0000 7f7016ffd640 1 --2- 172.21.15.157:0/82555124 >> v2:172.21.15.169:3300/0 conn(0x7f70180c68d0 0x7f70180c6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.441 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.439+0000 7f70177fe640 1 --2- 172.21.15.157:0/82555124 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70180c58a0 0x7f70180c90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.441 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.439+0000 7f700e7fc640 1 -- 172.21.15.157:0/82555124 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70180c58a0 msgr2=0x7f70180c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:54.441 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.439+0000 7f700e7fc640 1 --2- 172.21.15.157:0/82555124 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70180c58a0 0x7f70180c90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.442 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.439+0000 7f700e7fc640 1 -- 172.21.15.157:0/82555124 >> v2:172.21.15.169:3300/0 conn(0x7f70180c68d0 msgr2=0x7f70180c6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:54.442 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.439+0000 7f700e7fc640 1 --2- 172.21.15.157:0/82555124 >> v2:172.21.15.169:3300/0 conn(0x7f70180c68d0 0x7f70180c6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.442 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.439+0000 7f700e7fc640 1 -- 172.21.15.157:0/82555124 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70181561a0 con 0x7f70180c71f0 2024-04-25T00:00:54.442 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.440+0000 7f700e7fc640 1 --2- 172.21.15.157:0/82555124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70180c71f0 0x7f70180c8ba0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f70180cbd40 tx=0x7f700002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc238c41d30fc360 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:54.442 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.440+0000 7f70167fc640 1 -- 172.21.15.157:0/82555124 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7000037070 con 0x7f70180c71f0 2024-04-25T00:00:54.442 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.440+0000 7f70167fc640 1 -- 172.21.15.157:0/82555124 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7000032440 con 0x7f70180c71f0 2024-04-25T00:00:54.442 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.440+0000 7f70167fc640 1 -- 172.21.15.157:0/82555124 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7000032810 con 0x7f70180c71f0 2024-04-25T00:00:54.442 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.440+0000 7f701d5b0640 1 -- 172.21.15.157:0/82555124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70180c71f0 msgr2=0x7f70180c8ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:54.442 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.440+0000 7f701d5b0640 1 --2- 172.21.15.157:0/82555124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70180c71f0 0x7f70180c8ba0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f70180cbd40 tx=0x7f700002a270 comp rx=0 tx=0).stop 2024-04-25T00:00:54.442 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.441+0000 7f701d5b0640 1 -- 172.21.15.157:0/82555124 shutdown_connections 2024-04-25T00:00:54.442 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.441+0000 7f701d5b0640 1 --2- 172.21.15.157:0/82555124 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70180c58a0 0x7f70180c90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.443 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.441+0000 7f701d5b0640 1 --2- 172.21.15.157:0/82555124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70180c71f0 0x7f70180c8ba0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.443 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.441+0000 7f701d5b0640 1 --2- 172.21.15.157:0/82555124 >> v2:172.21.15.169:3300/0 conn(0x7f70180c68d0 0x7f70180c6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.443 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.441+0000 7f701d5b0640 1 -- 172.21.15.157:0/82555124 >> 172.21.15.157:0/82555124 conn(0x7f70180caca0 msgr2=0x7f70180cb0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:54.443 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.441+0000 7f701d5b0640 1 -- 172.21.15.157:0/82555124 shutdown_connections 2024-04-25T00:00:54.443 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.441+0000 7f701d5b0640 1 -- 172.21.15.157:0/82555124 wait complete. 2024-04-25T00:00:54.446 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.442+0000 7f701d5b0640 1 Processor -- start 2024-04-25T00:00:54.446 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.442+0000 7f701d5b0640 1 -- start start 2024-04-25T00:00:54.446 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.442+0000 7f701d5b0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70180c58a0 0x7f70180bf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.446 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.442+0000 7f701d5b0640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70180c68d0 0x7f70181eea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.446 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.442+0000 7f701d5b0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70180c71f0 0x7f70181eef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.446 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.442+0000 7f701d5b0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f701806b830 con 0x7f70180c71f0 2024-04-25T00:00:54.446 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.442+0000 7f701d5b0640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7018156480 con 0x7f70180c58a0 2024-04-25T00:00:54.446 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.442+0000 7f701d5b0640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7018152ed0 con 0x7f70180c68d0 2024-04-25T00:00:54.446 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.442+0000 7f70177fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70180c71f0 0x7f70181eef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.446 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.443+0000 7f70177fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70180c71f0 0x7f70181eef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37660/0 (socket says 172.21.15.157:37660) 2024-04-25T00:00:54.447 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.443+0000 7f70177fe640 1 -- 172.21.15.157:0/1187115935 learned_addr learned my addr 172.21.15.157:0/1187115935 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:00:54.447 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.443+0000 7f7016ffd640 1 --2- 172.21.15.157:0/1187115935 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70180c58a0 0x7f70180bf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.447 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.443+0000 7f700e7fc640 1 --2- 172.21.15.157:0/1187115935 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70180c68d0 0x7f70181eea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.447 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.443+0000 7f70177fe640 1 -- 172.21.15.157:0/1187115935 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70180c68d0 msgr2=0x7f70181eea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:54.447 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.443+0000 7f70177fe640 1 --2- 172.21.15.157:0/1187115935 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70180c68d0 0x7f70181eea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.447 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.443+0000 7f70177fe640 1 -- 172.21.15.157:0/1187115935 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70180c58a0 msgr2=0x7f70180bf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:54.447 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.443+0000 7f70177fe640 1 --2- 172.21.15.157:0/1187115935 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70180c58a0 0x7f70180bf9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.448 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.443+0000 7f70177fe640 1 -- 172.21.15.157:0/1187115935 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70181ef480 con 0x7f70180c71f0 2024-04-25T00:00:54.448 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.443+0000 7f70177fe640 1 --2- 172.21.15.157:0/1187115935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70180c71f0 0x7f70181eef40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f701000eed0 tx=0x7f701000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:54.448 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.443+0000 7f700ffff640 1 -- 172.21.15.157:0/1187115935 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7010018020 con 0x7f70180c71f0 2024-04-25T00:00:54.448 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.443+0000 7f700ffff640 1 -- 172.21.15.157:0/1187115935 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f701000bd20 con 0x7f70180c71f0 2024-04-25T00:00:54.448 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.443+0000 7f701d5b0640 1 -- 172.21.15.157:0/1187115935 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70180c0da0 con 0x7f70180c71f0 2024-04-25T00:00:54.448 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.443+0000 7f700ffff640 1 -- 172.21.15.157:0/1187115935 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7010014640 con 0x7f70180c71f0 2024-04-25T00:00:54.448 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.443+0000 7f701d5b0640 1 -- 172.21.15.157:0/1187115935 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70180c1360 con 0x7f70180c71f0 2024-04-25T00:00:54.448 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.444+0000 7f701d5b0640 1 -- 172.21.15.157:0/1187115935 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70180681c0 con 0x7f70180c71f0 2024-04-25T00:00:54.448 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.445+0000 7f700ffff640 1 -- 172.21.15.157:0/1187115935 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7010005d10 con 0x7f70180c71f0 2024-04-25T00:00:54.448 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.445+0000 7f700ffff640 1 --2- 172.21.15.157:0/1187115935 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f6ff806be30 0x7f6ff806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.448 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.445+0000 7f700ffff640 1 -- 172.21.15.157:0/1187115935 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f701008b700 con 0x7f70180c71f0 2024-04-25T00:00:54.448 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.445+0000 7f7016ffd640 1 --2- 172.21.15.157:0/1187115935 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f6ff806be30 0x7f6ff806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.448 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.446+0000 7f7016ffd640 1 --2- 172.21.15.157:0/1187115935 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f6ff806be30 0x7f6ff806e2f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7008009aa0 tx=0x7f7008009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:54.449 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.450+0000 7f700ffff640 1 -- 172.21.15.157:0/1187115935 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7010022070 con 0x7f70180c71f0 2024-04-25T00:00:54.631 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.632+0000 7f701d5b0640 1 -- 172.21.15.157:0/1187115935 --> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f70180c5680 con 0x7f6ff806be30 2024-04-25T00:00:54.632 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.632+0000 7f700ffff640 1 -- 172.21.15.157:0/1187115935 <== mgr.4105 v2:172.21.15.169:6832/662378289 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f70180c5680 con 0x7f6ff806be30 2024-04-25T00:00:54.632 INFO:teuthology.orchestra.run.smithi157.stderr:instructed osd(s) 1 to deep-scrub 2024-04-25T00:00:54.635 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.635+0000 7f701d5b0640 1 -- 172.21.15.157:0/1187115935 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f6ff806be30 msgr2=0x7f6ff806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:54.635 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.635+0000 7f701d5b0640 1 --2- 172.21.15.157:0/1187115935 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f6ff806be30 0x7f6ff806e2f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7008009aa0 tx=0x7f7008009190 comp rx=0 tx=0).stop 2024-04-25T00:00:54.635 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.635+0000 7f701d5b0640 1 -- 172.21.15.157:0/1187115935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70180c71f0 msgr2=0x7f70181eef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:54.635 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.635+0000 7f701d5b0640 1 --2- 172.21.15.157:0/1187115935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70180c71f0 0x7f70181eef40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f701000eed0 tx=0x7f701000b480 comp rx=0 tx=0).stop 2024-04-25T00:00:54.635 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.635+0000 7f701d5b0640 1 -- 172.21.15.157:0/1187115935 shutdown_connections 2024-04-25T00:00:54.635 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.635+0000 7f701d5b0640 1 --2- 172.21.15.157:0/1187115935 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f6ff806be30 0x7f6ff806e2f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.635 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.635+0000 7f701d5b0640 1 --2- 172.21.15.157:0/1187115935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70180c71f0 0x7f70181eef40 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.635 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.635+0000 7f701d5b0640 1 --2- 172.21.15.157:0/1187115935 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70180c68d0 0x7f70181eea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.636 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.635+0000 7f701d5b0640 1 --2- 172.21.15.157:0/1187115935 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70180c58a0 0x7f70180bf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.636 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.635+0000 7f701d5b0640 1 -- 172.21.15.157:0/1187115935 >> 172.21.15.157:0/1187115935 conn(0x7f70180caca0 msgr2=0x7f70180ac0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:54.636 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.635+0000 7f701d5b0640 1 -- 172.21.15.157:0/1187115935 shutdown_connections 2024-04-25T00:00:54.636 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.635+0000 7f701d5b0640 1 -- 172.21.15.157:0/1187115935 wait complete. 2024-04-25T00:00:54.645 INFO:tasks.ceph:Scrubbing osd.2 2024-04-25T00:00:54.645 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-25T00:00:54.748 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.749+0000 7fa24c48c640 1 Processor -- start 2024-04-25T00:00:54.749 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.749+0000 7fa24c48c640 1 -- start start 2024-04-25T00:00:54.750 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.750+0000 7fa24c48c640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa24414f3a0 0x7fa24406d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.750 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.750+0000 7fa24c48c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2440c75f0 con 0x7fa2440b0870 2024-04-25T00:00:54.750 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.750+0000 7fa24c48c640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa244155ef0 con 0x7fa2440afd00 2024-04-25T00:00:54.750 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.750+0000 7fa24c48c640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa2441561d0 con 0x7fa24414f3a0 2024-04-25T00:00:54.750 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.750+0000 7fa24aa29640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fa2440b0870 0x7fa2440c7860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:32958/0 (socket says 172.21.15.157:32958) 2024-04-25T00:00:54.750 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.750+0000 7fa24aa29640 1 -- 172.21.15.157:0/2837400538 learned_addr learned my addr 172.21.15.157:0/2837400538 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-25T00:00:54.751 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.750+0000 7fa24a228640 1 --2- 172.21.15.157:0/2837400538 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa24414f3a0 0x7fa24406d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.751 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.750+0000 7fa249226640 1 -- 172.21.15.157:0/2837400538 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2547436394 0 0) 0x7fa2440c75f0 con 0x7fa2440b0870 2024-04-25T00:00:54.751 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.750+0000 7fa249226640 1 -- 172.21.15.157:0/2837400538 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa230003520 con 0x7fa2440b0870 2024-04-25T00:00:54.751 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.750+0000 7fa249226640 1 -- 172.21.15.157:0/2837400538 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1286786365 0 0) 0x7fa244155ef0 con 0x7fa2440afd00 2024-04-25T00:00:54.751 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.750+0000 7fa249226640 1 -- 172.21.15.157:0/2837400538 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa2440c75f0 con 0x7fa2440afd00 2024-04-25T00:00:54.751 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.750+0000 7fa24a228640 1 -- 172.21.15.157:0/2837400538 >> v1:172.21.15.169:6789/0 conn(0x7fa2440afd00 legacy=0x7fa2440b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:54.751 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.750+0000 7fa24a228640 1 -- 172.21.15.157:0/2837400538 >> v1:172.21.15.157:6789/0 conn(0x7fa2440b0870 legacy=0x7fa2440c7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:54.751 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.750+0000 7fa249226640 1 -- 172.21.15.157:0/2837400538 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1621863661 0 0) 0x7fa230003520 con 0x7fa2440b0870 2024-04-25T00:00:54.751 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.750+0000 7fa24a228640 1 -- 172.21.15.157:0/2837400538 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2440c57d0 con 0x7fa24414f3a0 2024-04-25T00:00:54.752 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.751+0000 7fa24a228640 1 --2- 172.21.15.157:0/2837400538 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa24414f3a0 0x7fa24406d530 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa2340097d0 tx=0x7fa23402a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8ec5cd867d9718d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:54.753 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.751+0000 7fa249226640 1 -- 172.21.15.157:0/2837400538 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa234037070 con 0x7fa24414f3a0 2024-04-25T00:00:54.753 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.751+0000 7fa249226640 1 -- 172.21.15.157:0/2837400538 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa23402ac30 con 0x7fa24414f3a0 2024-04-25T00:00:54.753 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.751+0000 7fa249226640 1 -- 172.21.15.157:0/2837400538 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23c0019f0 con 0x7fa24414f3a0 2024-04-25T00:00:54.753 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.752+0000 7fa24c48c640 1 -- 172.21.15.157:0/2837400538 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa24414f3a0 msgr2=0x7fa24406d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:54.753 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.752+0000 7fa24c48c640 1 --2- 172.21.15.157:0/2837400538 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa24414f3a0 0x7fa24406d530 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa2340097d0 tx=0x7fa23402a1a0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.753 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.752+0000 7fa24c48c640 1 -- 172.21.15.157:0/2837400538 shutdown_connections 2024-04-25T00:00:54.753 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.752+0000 7fa24c48c640 1 --2- 172.21.15.157:0/2837400538 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa24414f3a0 0x7fa24406d530 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.753 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.752+0000 7fa24c48c640 1 -- 172.21.15.157:0/2837400538 >> 172.21.15.157:0/2837400538 conn(0x7fa244066ea0 msgr2=0x7fa2440672b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:54.753 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.752+0000 7fa24c48c640 1 -- 172.21.15.157:0/2837400538 shutdown_connections 2024-04-25T00:00:54.753 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.752+0000 7fa24c48c640 1 -- 172.21.15.157:0/2837400538 wait complete. 2024-04-25T00:00:54.753 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.753+0000 7fa24c48c640 1 Processor -- start 2024-04-25T00:00:54.753 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.753+0000 7fa24c48c640 1 -- start start 2024-04-25T00:00:54.756 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.754+0000 7fa24c48c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2440afd00 0x7fa2440c1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.756 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.754+0000 7fa24c48c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa24414f3a0 0x7fa2440c4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.756 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.754+0000 7fa24c48c640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa24414e550 0x7fa24414d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.756 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.754+0000 7fa24c48c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa24414f780 con 0x7fa2440afd00 2024-04-25T00:00:54.756 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.754+0000 7fa24c48c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa2440c5d90 con 0x7fa24414f3a0 2024-04-25T00:00:54.756 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.754+0000 7fa24c48c640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa2440c7f70 con 0x7fa24414e550 2024-04-25T00:00:54.756 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.754+0000 7fa24a228640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2440afd00 0x7fa2440c1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.756 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.754+0000 7fa24a228640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2440afd00 0x7fa2440c1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37672/0 (socket says 172.21.15.157:37672) 2024-04-25T00:00:54.756 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.754+0000 7fa24aa29640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa24414e550 0x7fa24414d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.756 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.754+0000 7fa24a228640 1 -- 172.21.15.157:0/811256600 learned_addr learned my addr 172.21.15.157:0/811256600 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:00:54.756 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.754+0000 7fa249a27640 1 --2- 172.21.15.157:0/811256600 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa24414f3a0 0x7fa2440c4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.756 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.754+0000 7fa24a228640 1 -- 172.21.15.157:0/811256600 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa24414e550 msgr2=0x7fa24414d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:54.757 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.754+0000 7fa24a228640 1 --2- 172.21.15.157:0/811256600 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa24414e550 0x7fa24414d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.757 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.754+0000 7fa24a228640 1 -- 172.21.15.157:0/811256600 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa24414f3a0 msgr2=0x7fa2440c4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:54.757 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.754+0000 7fa24a228640 1 --2- 172.21.15.157:0/811256600 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa24414f3a0 0x7fa2440c4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.757 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.754+0000 7fa24a228640 1 -- 172.21.15.157:0/811256600 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa24414eb80 con 0x7fa2440afd00 2024-04-25T00:00:54.757 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.754+0000 7fa24a228640 1 --2- 172.21.15.157:0/811256600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2440afd00 0x7fa2440c1920 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa2340097d0 tx=0x7fa2340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:54.757 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.755+0000 7fa23affd640 1 -- 172.21.15.157:0/811256600 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa234037050 con 0x7fa2440afd00 2024-04-25T00:00:54.757 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.755+0000 7fa23affd640 1 -- 172.21.15.157:0/811256600 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa234043070 con 0x7fa2440afd00 2024-04-25T00:00:54.757 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.755+0000 7fa23affd640 1 -- 172.21.15.157:0/811256600 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23403ca50 con 0x7fa2440afd00 2024-04-25T00:00:54.757 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.755+0000 7fa24c48c640 1 -- 172.21.15.157:0/811256600 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2440bf990 con 0x7fa2440afd00 2024-04-25T00:00:54.757 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.755+0000 7fa24c48c640 1 -- 172.21.15.157:0/811256600 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2440ad450 con 0x7fa2440afd00 2024-04-25T00:00:54.757 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.756+0000 7fa24c48c640 1 -- 172.21.15.157:0/811256600 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa24406d310 con 0x7fa2440afd00 2024-04-25T00:00:54.757 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.756+0000 7fa23affd640 1 -- 172.21.15.157:0/811256600 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa23403cbf0 con 0x7fa2440afd00 2024-04-25T00:00:54.759 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.757+0000 7fa23affd640 1 --2- 172.21.15.157:0/811256600 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fa22806be60 0x7fa22806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.759 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.757+0000 7fa23affd640 1 -- 172.21.15.157:0/811256600 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fa2340abbc0 con 0x7fa2440afd00 2024-04-25T00:00:54.759 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.757+0000 7fa249a27640 1 --2- 172.21.15.157:0/811256600 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fa22806be60 0x7fa22806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.759 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.757+0000 7fa23affd640 1 --2- 172.21.15.157:0/811256600 >> [v2:172.21.15.157:6824/2793948092,v1:172.21.15.157:6825/2793948092] conn(0x7fa228075880 0x7fa228077d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.759 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.757+0000 7fa23affd640 1 -- 172.21.15.157:0/811256600 --> [v2:172.21.15.157:6824/2793948092,v1:172.21.15.157:6825/2793948092] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa23402be50 con 0x7fa228075880 2024-04-25T00:00:54.759 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.757+0000 7fa23affd640 1 -- 172.21.15.157:0/811256600 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa2340b2020 con 0x7fa2440afd00 2024-04-25T00:00:54.759 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.757+0000 7fa24aa29640 1 --2- 172.21.15.157:0/811256600 >> [v2:172.21.15.157:6824/2793948092,v1:172.21.15.157:6825/2793948092] conn(0x7fa228075880 0x7fa228077d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.759 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.758+0000 7fa249a27640 1 --2- 172.21.15.157:0/811256600 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fa22806be60 0x7fa22806e320 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa240007fd0 tx=0x7fa240009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:54.759 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.758+0000 7fa24aa29640 1 --2- 172.21.15.157:0/811256600 >> [v2:172.21.15.157:6824/2793948092,v1:172.21.15.157:6825/2793948092] conn(0x7fa228075880 0x7fa228077d20 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:54.759 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.758+0000 7fa23affd640 1 -- 172.21.15.157:0/811256600 <== osd.2 v2:172.21.15.157:6824/2793948092 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa23402be50 con 0x7fa228075880 2024-04-25T00:00:54.771 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.771+0000 7fa24c48c640 1 -- 172.21.15.157:0/811256600 --> [v2:172.21.15.157:6824/2793948092,v1:172.21.15.157:6825/2793948092] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fa2440c7990 con 0x7fa228075880 2024-04-25T00:00:54.771 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.772+0000 7fa23affd640 1 -- 172.21.15.157:0/811256600 <== osd.2 v2:172.21.15.157:6824/2793948092 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fa2440c7990 con 0x7fa228075880 2024-04-25T00:00:54.771 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-04-25T00:00:54.772 INFO:teuthology.orchestra.run.smithi157.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T00:00:54.772 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-04-25T00:00:54.773 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.772+0000 7fa24c48c640 1 -- 172.21.15.157:0/811256600 >> [v2:172.21.15.157:6824/2793948092,v1:172.21.15.157:6825/2793948092] conn(0x7fa228075880 msgr2=0x7fa228077d20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:54.773 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.772+0000 7fa24c48c640 1 --2- 172.21.15.157:0/811256600 >> [v2:172.21.15.157:6824/2793948092,v1:172.21.15.157:6825/2793948092] conn(0x7fa228075880 0x7fa228077d20 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.773 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.772+0000 7fa24c48c640 1 -- 172.21.15.157:0/811256600 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fa22806be60 msgr2=0x7fa22806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:54.773 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.772+0000 7fa24c48c640 1 --2- 172.21.15.157:0/811256600 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fa22806be60 0x7fa22806e320 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa240007fd0 tx=0x7fa240009040 comp rx=0 tx=0).stop 2024-04-25T00:00:54.774 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.772+0000 7fa24c48c640 1 -- 172.21.15.157:0/811256600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2440afd00 msgr2=0x7fa2440c1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:54.774 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.772+0000 7fa24c48c640 1 --2- 172.21.15.157:0/811256600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2440afd00 0x7fa2440c1920 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa2340097d0 tx=0x7fa2340046c0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.774 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.772+0000 7fa24aa29640 1 -- 172.21.15.157:0/811256600 reap_dead start 2024-04-25T00:00:54.774 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.772+0000 7fa24c48c640 1 -- 172.21.15.157:0/811256600 shutdown_connections 2024-04-25T00:00:54.774 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.772+0000 7fa24c48c640 1 -- 172.21.15.157:0/811256600 >> 172.21.15.157:0/811256600 conn(0x7fa244066ea0 msgr2=0x7fa244153ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:54.774 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.772+0000 7fa24c48c640 1 -- 172.21.15.157:0/811256600 shutdown_connections 2024-04-25T00:00:54.774 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.772+0000 7fa24c48c640 1 -- 172.21.15.157:0/811256600 wait complete. 2024-04-25T00:00:54.782 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-25T00:00:54.882 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.882+0000 7fcc034d6640 1 Processor -- start 2024-04-25T00:00:54.883 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.883+0000 7fcc034d6640 1 -- start start 2024-04-25T00:00:54.890 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.883+0000 7fcc034d6640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcbfc0c68d0 0x7fcbfc0c6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.890 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.883+0000 7fcc034d6640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcbfc0c58a0 0x7fcbfc0c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.890 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.883+0000 7fcc034d6640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbfc06bd80 con 0x7fcbfc0c71f0 2024-04-25T00:00:54.890 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.883+0000 7fcc034d6640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fcbfc0c6730 con 0x7fcbfc0c58a0 2024-04-25T00:00:54.890 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.883+0000 7fcc034d6640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fcbfc1561a0 con 0x7fcbfc0c68d0 2024-04-25T00:00:54.890 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.883+0000 7fcc01a73640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcbfc0c58a0 0x7fcbfc0c92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.890 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.883+0000 7fcc00a71640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fcbfc0c71f0 0x7fcbfc0c8ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:32966/0 (socket says 172.21.15.157:32966) 2024-04-25T00:00:54.890 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.883+0000 7fcc00a71640 1 -- 172.21.15.157:0/1957602673 learned_addr learned my addr 172.21.15.157:0/1957602673 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-25T00:00:54.890 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.883+0000 7fcc01a73640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcbfc0c58a0 0x7fcbfc0c92b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37676/0 (socket says 172.21.15.157:37676) 2024-04-25T00:00:54.891 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.883+0000 7fcc01272640 1 --2- 172.21.15.157:0/1957602673 >> v2:172.21.15.169:3300/0 conn(0x7fcbfc0c68d0 0x7fcbfc0c6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.891 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.884+0000 7fcc01a73640 1 -- 172.21.15.157:0/1957602673 >> v2:172.21.15.169:3300/0 conn(0x7fcbfc0c68d0 msgr2=0x7fcbfc0c6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:54.891 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.884+0000 7fcc01a73640 1 --2- 172.21.15.157:0/1957602673 >> v2:172.21.15.169:3300/0 conn(0x7fcbfc0c68d0 0x7fcbfc0c6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.891 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.884+0000 7fcc01a73640 1 -- 172.21.15.157:0/1957602673 >> v1:172.21.15.157:6789/0 conn(0x7fcbfc0c71f0 legacy=0x7fcbfc0c8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:54.891 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.884+0000 7fcbf3fff640 1 -- 172.21.15.157:0/1957602673 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 627828588 0 0) 0x7fcbfc06bd80 con 0x7fcbfc0c71f0 2024-04-25T00:00:54.891 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.884+0000 7fcc01a73640 1 -- 172.21.15.157:0/1957602673 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbfc0c6370 con 0x7fcbfc0c58a0 2024-04-25T00:00:54.891 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.884+0000 7fcc01a73640 1 --2- 172.21.15.157:0/1957602673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbfc0c58a0 0x7fcbfc0c92b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc0cbd40 tx=0x7fcbf402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33f32d9c74a2fd53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:54.891 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.885+0000 7fcbf3fff640 1 -- 172.21.15.157:0/1957602673 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf4007740 con 0x7fcbfc0c58a0 2024-04-25T00:00:54.891 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.885+0000 7fcbf3fff640 1 -- 172.21.15.157:0/1957602673 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbf40078e0 con 0x7fcbfc0c58a0 2024-04-25T00:00:54.891 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.885+0000 7fcbf3fff640 1 -- 172.21.15.157:0/1957602673 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf4007cb0 con 0x7fcbfc0c58a0 2024-04-25T00:00:54.891 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.885+0000 7fcc034d6640 1 -- 172.21.15.157:0/1957602673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbfc0c58a0 msgr2=0x7fcbfc0c92b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:54.891 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.885+0000 7fcc034d6640 1 --2- 172.21.15.157:0/1957602673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbfc0c58a0 0x7fcbfc0c92b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc0cbd40 tx=0x7fcbf402a710 comp rx=0 tx=0).stop 2024-04-25T00:00:54.891 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.885+0000 7fcc034d6640 1 -- 172.21.15.157:0/1957602673 shutdown_connections 2024-04-25T00:00:54.892 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.885+0000 7fcc034d6640 1 --2- 172.21.15.157:0/1957602673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbfc0c58a0 0x7fcbfc0c92b0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.892 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.885+0000 7fcc034d6640 1 --2- 172.21.15.157:0/1957602673 >> v2:172.21.15.169:3300/0 conn(0x7fcbfc0c68d0 0x7fcbfc0c6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.892 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.885+0000 7fcc034d6640 1 -- 172.21.15.157:0/1957602673 >> 172.21.15.157:0/1957602673 conn(0x7fcbfc0caca0 msgr2=0x7fcbfc0cb0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:54.892 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.885+0000 7fcc034d6640 1 -- 172.21.15.157:0/1957602673 shutdown_connections 2024-04-25T00:00:54.892 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.885+0000 7fcc034d6640 1 -- 172.21.15.157:0/1957602673 wait complete. 2024-04-25T00:00:54.892 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.887+0000 7fcc034d6640 1 Processor -- start 2024-04-25T00:00:54.892 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.887+0000 7fcc034d6640 1 -- start start 2024-04-25T00:00:54.892 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.888+0000 7fcc034d6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbfc0c58a0 0x7fcbfc14b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.892 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.888+0000 7fcc034d6640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbfc0c68d0 0x7fcbfc0afaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.893 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.888+0000 7fcc034d6640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbfc0c71f0 0x7fcbfc14acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.893 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.888+0000 7fcc034d6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcbfc152ed0 con 0x7fcbfc0c58a0 2024-04-25T00:00:54.893 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.888+0000 7fcc034d6640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcbfc06b830 con 0x7fcbfc0c68d0 2024-04-25T00:00:54.893 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.888+0000 7fcc034d6640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcbfc153050 con 0x7fcbfc0c71f0 2024-04-25T00:00:54.893 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.888+0000 7fcc01272640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbfc0c58a0 0x7fcbfc14b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.893 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.888+0000 7fcc01272640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbfc0c58a0 0x7fcbfc14b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37682/0 (socket says 172.21.15.157:37682) 2024-04-25T00:00:54.893 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.888+0000 7fcc01272640 1 -- 172.21.15.157:0/4141188875 learned_addr learned my addr 172.21.15.157:0/4141188875 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:00:54.893 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.888+0000 7fcc00a71640 1 --2- 172.21.15.157:0/4141188875 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbfc0c68d0 0x7fcbfc0afaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.893 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.888+0000 7fcc01a73640 1 --2- 172.21.15.157:0/4141188875 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbfc0c71f0 0x7fcbfc14acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.893 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.888+0000 7fcc01272640 1 -- 172.21.15.157:0/4141188875 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbfc0c71f0 msgr2=0x7fcbfc14acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:54.893 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.888+0000 7fcc01272640 1 --2- 172.21.15.157:0/4141188875 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbfc0c71f0 0x7fcbfc14acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.894 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.888+0000 7fcc01272640 1 -- 172.21.15.157:0/4141188875 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbfc0c68d0 msgr2=0x7fcbfc0afaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:54.894 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.888+0000 7fcc01272640 1 --2- 172.21.15.157:0/4141188875 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbfc0c68d0 0x7fcbfc0afaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.894 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.888+0000 7fcc01272640 1 -- 172.21.15.157:0/4141188875 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbfc14c240 con 0x7fcbfc0c58a0 2024-04-25T00:00:54.894 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.889+0000 7fcc01272640 1 --2- 172.21.15.157:0/4141188875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbfc0c58a0 0x7fcbfc14b9f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fcbf800e870 tx=0x7fcbf800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:54.894 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.889+0000 7fcbf1ffb640 1 -- 172.21.15.157:0/4141188875 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8017070 con 0x7fcbfc0c58a0 2024-04-25T00:00:54.894 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.889+0000 7fcbf1ffb640 1 -- 172.21.15.157:0/4141188875 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbf800c940 con 0x7fcbfc0c58a0 2024-04-25T00:00:54.894 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.889+0000 7fcc034d6640 1 -- 172.21.15.157:0/4141188875 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbfc1eafb0 con 0x7fcbfc0c58a0 2024-04-25T00:00:54.894 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.889+0000 7fcbf1ffb640 1 -- 172.21.15.157:0/4141188875 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf800cd30 con 0x7fcbfc0c58a0 2024-04-25T00:00:54.894 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.889+0000 7fcc034d6640 1 -- 172.21.15.157:0/4141188875 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbfc1eb570 con 0x7fcbfc0c58a0 2024-04-25T00:00:54.895 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.890+0000 7fcc034d6640 1 -- 172.21.15.157:0/4141188875 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbfc0681c0 con 0x7fcbfc0c58a0 2024-04-25T00:00:54.895 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.890+0000 7fcbf1ffb640 1 -- 172.21.15.157:0/4141188875 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcbf801a070 con 0x7fcbfc0c58a0 2024-04-25T00:00:54.896 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.891+0000 7fcbf1ffb640 1 --2- 172.21.15.157:0/4141188875 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fcbe006be30 0x7fcbe006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.896 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.891+0000 7fcbf1ffb640 1 -- 172.21.15.157:0/4141188875 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fcbf808afd0 con 0x7fcbfc0c58a0 2024-04-25T00:00:54.896 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.891+0000 7fcc00a71640 1 --2- 172.21.15.157:0/4141188875 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fcbe006be30 0x7fcbe006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.896 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.891+0000 7fcc00a71640 1 --2- 172.21.15.157:0/4141188875 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fcbe006be30 0x7fcbe006e2f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc14ac80 tx=0x7fcbec007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:54.896 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:54.895+0000 7fcbf1ffb640 1 -- 172.21.15.157:0/4141188875 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcbf8058fc0 con 0x7fcbfc0c58a0 2024-04-25T00:00:55.077 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.077+0000 7fcc034d6640 1 -- 172.21.15.157:0/4141188875 --> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7fcbfc0c5680 con 0x7fcbe006be30 2024-04-25T00:00:55.077 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.078+0000 7fcbf1ffb640 1 -- 172.21.15.157:0/4141188875 <== mgr.4105 v2:172.21.15.169:6832/662378289 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fcbfc0c5680 con 0x7fcbe006be30 2024-04-25T00:00:55.077 INFO:teuthology.orchestra.run.smithi157.stderr:instructed osd(s) 2 to deep-scrub 2024-04-25T00:00:55.080 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.080+0000 7fcc034d6640 1 -- 172.21.15.157:0/4141188875 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fcbe006be30 msgr2=0x7fcbe006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:55.080 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.080+0000 7fcc034d6640 1 --2- 172.21.15.157:0/4141188875 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fcbe006be30 0x7fcbe006e2f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc14ac80 tx=0x7fcbec007d50 comp rx=0 tx=0).stop 2024-04-25T00:00:55.080 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.080+0000 7fcc034d6640 1 -- 172.21.15.157:0/4141188875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbfc0c58a0 msgr2=0x7fcbfc14b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:55.080 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.080+0000 7fcc034d6640 1 --2- 172.21.15.157:0/4141188875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbfc0c58a0 0x7fcbfc14b9f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fcbf800e870 tx=0x7fcbf800ed30 comp rx=0 tx=0).stop 2024-04-25T00:00:55.080 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.080+0000 7fcc034d6640 1 -- 172.21.15.157:0/4141188875 shutdown_connections 2024-04-25T00:00:55.080 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.080+0000 7fcc034d6640 1 --2- 172.21.15.157:0/4141188875 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fcbe006be30 0x7fcbe006e2f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:55.080 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.080+0000 7fcc034d6640 1 --2- 172.21.15.157:0/4141188875 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbfc0c71f0 0x7fcbfc14acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:55.080 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.080+0000 7fcc034d6640 1 --2- 172.21.15.157:0/4141188875 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbfc0c68d0 0x7fcbfc0afaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:55.080 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.080+0000 7fcc034d6640 1 --2- 172.21.15.157:0/4141188875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbfc0c58a0 0x7fcbfc14b9f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:55.080 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.080+0000 7fcc034d6640 1 -- 172.21.15.157:0/4141188875 >> 172.21.15.157:0/4141188875 conn(0x7fcbfc0caca0 msgr2=0x7fcbfc155420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:55.080 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.080+0000 7fcc034d6640 1 -- 172.21.15.157:0/4141188875 shutdown_connections 2024-04-25T00:00:55.081 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.080+0000 7fcc034d6640 1 -- 172.21.15.157:0/4141188875 wait complete. 2024-04-25T00:00:55.090 INFO:tasks.ceph:Scrubbing osd.3 2024-04-25T00:00:55.091 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-25T00:00:55.194 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.195+0000 7f7345129640 1 Processor -- start 2024-04-25T00:00:55.194 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.195+0000 7f7345129640 1 -- start start 2024-04-25T00:00:55.196 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.195+0000 7f7345129640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f734006d530 0x7f73400b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:55.196 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.195+0000 7f7345129640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f73400c7860 0x7f73400c7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:55.196 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.195+0000 7f7345129640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73400c75f0 con 0x7f734014f3a0 2024-04-25T00:00:55.196 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.195+0000 7f7345129640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f7340155ef0 con 0x7f734006d530 2024-04-25T00:00:55.196 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.195+0000 7f7345129640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f73401561d0 con 0x7f73400c7860 2024-04-25T00:00:55.196 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.196+0000 7f733ed9d640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f734014f3a0 0x7f73400afd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:32970/0 (socket says 172.21.15.157:32970) 2024-04-25T00:00:55.196 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.196+0000 7f733ed9d640 1 -- 172.21.15.157:0/2149803001 learned_addr learned my addr 172.21.15.157:0/2149803001 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-25T00:00:55.196 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.196+0000 7f733f59e640 1 --2- 172.21.15.157:0/2149803001 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f73400c7860 0x7f73400c7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:55.196 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.196+0000 7f733e59c640 1 --2- 172.21.15.157:0/2149803001 >> v2:172.21.15.169:3300/0 conn(0x7f734006d530 0x7f73400b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:55.196 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.196+0000 7f733dd9b640 1 -- 172.21.15.157:0/2149803001 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1548656979 0 0) 0x7f73400c75f0 con 0x7f734014f3a0 2024-04-25T00:00:55.196 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.196+0000 7f733dd9b640 1 -- 172.21.15.157:0/2149803001 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7328003520 con 0x7f734014f3a0 2024-04-25T00:00:55.197 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.196+0000 7f733f59e640 1 -- 172.21.15.157:0/2149803001 >> v2:172.21.15.169:3300/0 conn(0x7f734006d530 msgr2=0x7f73400b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:55.197 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.196+0000 7f733f59e640 1 --2- 172.21.15.157:0/2149803001 >> v2:172.21.15.169:3300/0 conn(0x7f734006d530 0x7f73400b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:55.197 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.196+0000 7f733f59e640 1 -- 172.21.15.157:0/2149803001 >> v1:172.21.15.157:6789/0 conn(0x7f734014f3a0 legacy=0x7f73400afd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:55.197 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.196+0000 7f733f59e640 1 -- 172.21.15.157:0/2149803001 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73400c57d0 con 0x7f73400c7860 2024-04-25T00:00:55.197 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.196+0000 7f733e59c640 1 --2- 172.21.15.157:0/2149803001 >> v2:172.21.15.169:3300/0 conn(0x7f734006d530 0x7f73400b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:00:55.199 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.197+0000 7f733f59e640 1 --2- 172.21.15.157:0/2149803001 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f73400c7860 0x7f73400c7cd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7338009d90 tx=0x7f733802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21715e5dae0fc521 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:55.199 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.197+0000 7f733dd9b640 1 -- 172.21.15.157:0/2149803001 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7338007740 con 0x7f73400c7860 2024-04-25T00:00:55.199 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.197+0000 7f733dd9b640 1 -- 172.21.15.157:0/2149803001 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73380078e0 con 0x7f73400c7860 2024-04-25T00:00:55.199 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.197+0000 7f733dd9b640 1 -- 172.21.15.157:0/2149803001 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7338007cb0 con 0x7f73400c7860 2024-04-25T00:00:55.199 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.197+0000 7f7345129640 1 -- 172.21.15.157:0/2149803001 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f73400c7860 msgr2=0x7f73400c7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:55.199 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.197+0000 7f7345129640 1 --2- 172.21.15.157:0/2149803001 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f73400c7860 0x7f73400c7cd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7338009d90 tx=0x7f733802a660 comp rx=0 tx=0).stop 2024-04-25T00:00:55.199 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.197+0000 7f7345129640 1 -- 172.21.15.157:0/2149803001 shutdown_connections 2024-04-25T00:00:55.199 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.197+0000 7f7345129640 1 --2- 172.21.15.157:0/2149803001 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f73400c7860 0x7f73400c7cd0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:55.199 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.197+0000 7f7345129640 1 --2- 172.21.15.157:0/2149803001 >> v2:172.21.15.169:3300/0 conn(0x7f734006d530 0x7f73400b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:55.199 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.197+0000 7f7345129640 1 -- 172.21.15.157:0/2149803001 >> 172.21.15.157:0/2149803001 conn(0x7f7340066ea0 msgr2=0x7f73400672b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:55.199 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.198+0000 7f7345129640 1 -- 172.21.15.157:0/2149803001 shutdown_connections 2024-04-25T00:00:55.199 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.198+0000 7f7345129640 1 -- 172.21.15.157:0/2149803001 wait complete. 2024-04-25T00:00:55.199 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.199+0000 7f7345129640 1 Processor -- start 2024-04-25T00:00:55.200 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.199+0000 7f7345129640 1 -- start start 2024-04-25T00:00:55.203 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.200+0000 7f7345129640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f734006d530 0x7f73400aa9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:55.203 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.200+0000 7f7345129640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73400c7860 0x7f73400c4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:55.203 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.200+0000 7f7345129640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f734014f3a0 0x7f734014e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:55.203 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.200+0000 7f7345129640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f734014f780 con 0x7f734006d530 2024-04-25T00:00:55.203 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.200+0000 7f7345129640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f73400c5d90 con 0x7f73400c7860 2024-04-25T00:00:55.203 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.200+0000 7f7345129640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f73400b0950 con 0x7f734014f3a0 2024-04-25T00:00:55.203 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.200+0000 7f733ed9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f734006d530 0x7f73400aa9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:55.203 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.200+0000 7f733ed9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f734006d530 0x7f73400aa9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37690/0 (socket says 172.21.15.157:37690) 2024-04-25T00:00:55.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.200+0000 7f733ed9d640 1 -- 172.21.15.157:0/3034665610 learned_addr learned my addr 172.21.15.157:0/3034665610 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:00:55.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.200+0000 7f733e59c640 1 --2- 172.21.15.157:0/3034665610 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73400c7860 0x7f73400c4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:55.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.200+0000 7f733f59e640 1 --2- 172.21.15.157:0/3034665610 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f734014f3a0 0x7f734014e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:55.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.200+0000 7f733ed9d640 1 -- 172.21.15.157:0/3034665610 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f734014f3a0 msgr2=0x7f734014e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:55.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.200+0000 7f733ed9d640 1 --2- 172.21.15.157:0/3034665610 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f734014f3a0 0x7f734014e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:55.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.200+0000 7f733ed9d640 1 -- 172.21.15.157:0/3034665610 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73400c7860 msgr2=0x7f73400c4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:55.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.200+0000 7f733ed9d640 1 --2- 172.21.15.157:0/3034665610 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73400c7860 0x7f73400c4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:55.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.200+0000 7f733ed9d640 1 -- 172.21.15.157:0/3034665610 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73401eac00 con 0x7f734006d530 2024-04-25T00:00:55.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.200+0000 7f733ed9d640 1 --2- 172.21.15.157:0/3034665610 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f734006d530 0x7f73400aa9d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f733000c430 tx=0x7f733000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:55.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.200+0000 7f73277fe640 1 -- 172.21.15.157:0/3034665610 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7330017070 con 0x7f734006d530 2024-04-25T00:00:55.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.200+0000 7f73277fe640 1 -- 172.21.15.157:0/3034665610 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7330005ce0 con 0x7f734006d530 2024-04-25T00:00:55.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.200+0000 7f7345129640 1 -- 172.21.15.157:0/3034665610 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73401eaee0 con 0x7f734006d530 2024-04-25T00:00:55.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.200+0000 7f73277fe640 1 -- 172.21.15.157:0/3034665610 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73300072f0 con 0x7f734006d530 2024-04-25T00:00:55.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.201+0000 7f7345129640 1 -- 172.21.15.157:0/3034665610 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73401eb4a0 con 0x7f734006d530 2024-04-25T00:00:55.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.201+0000 7f7345129640 1 -- 172.21.15.157:0/3034665610 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f73400681c0 con 0x7f734006d530 2024-04-25T00:00:55.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.202+0000 7f73277fe640 1 -- 172.21.15.157:0/3034665610 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f733001a070 con 0x7f734006d530 2024-04-25T00:00:55.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.202+0000 7f73277fe640 1 --2- 172.21.15.157:0/3034665610 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f732c06be30 0x7f732c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:55.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.203+0000 7f73277fe640 1 -- 172.21.15.157:0/3034665610 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f733008acd0 con 0x7f734006d530 2024-04-25T00:00:55.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.203+0000 7f73277fe640 1 --2- 172.21.15.157:0/3034665610 >> [v2:172.21.15.157:6800/422758549,v1:172.21.15.157:6801/422758549] conn(0x7f732c075840 0x7f732c077ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:55.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.203+0000 7f73277fe640 1 -- 172.21.15.157:0/3034665610 --> [v2:172.21.15.157:6800/422758549,v1:172.21.15.157:6801/422758549] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7330006e50 con 0x7f732c075840 2024-04-25T00:00:55.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.203+0000 7f733e59c640 1 --2- 172.21.15.157:0/3034665610 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f732c06be30 0x7f732c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:55.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.203+0000 7f73277fe640 1 -- 172.21.15.157:0/3034665610 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f733008b160 con 0x7f734006d530 2024-04-25T00:00:55.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.203+0000 7f733f59e640 1 --2- 172.21.15.157:0/3034665610 >> [v2:172.21.15.157:6800/422758549,v1:172.21.15.157:6801/422758549] conn(0x7f732c075840 0x7f732c077ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:55.206 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.203+0000 7f733f59e640 1 --2- 172.21.15.157:0/3034665610 >> [v2:172.21.15.157:6800/422758549,v1:172.21.15.157:6801/422758549] conn(0x7f732c075840 0x7f732c077ce0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:55.206 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.203+0000 7f733e59c640 1 --2- 172.21.15.157:0/3034665610 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f732c06be30 0x7f732c06e2f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7334009560 tx=0x7f7334009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:55.206 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.204+0000 7f73277fe640 1 -- 172.21.15.157:0/3034665610 <== osd.3 v2:172.21.15.157:6800/422758549 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7330006e50 con 0x7f732c075840 2024-04-25T00:00:55.217 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.217+0000 7f7345129640 1 -- 172.21.15.157:0/3034665610 --> [v2:172.21.15.157:6800/422758549,v1:172.21.15.157:6801/422758549] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f73400c4f50 con 0x7f732c075840 2024-04-25T00:00:55.217 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.217+0000 7f73277fe640 1 -- 172.21.15.157:0/3034665610 <== osd.3 v2:172.21.15.157:6800/422758549 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f73400c4f50 con 0x7f732c075840 2024-04-25T00:00:55.217 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-04-25T00:00:55.217 INFO:teuthology.orchestra.run.smithi157.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T00:00:55.217 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-04-25T00:00:55.218 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.218+0000 7f7345129640 1 -- 172.21.15.157:0/3034665610 >> [v2:172.21.15.157:6800/422758549,v1:172.21.15.157:6801/422758549] conn(0x7f732c075840 msgr2=0x7f732c077ce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:55.219 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.218+0000 7f7345129640 1 --2- 172.21.15.157:0/3034665610 >> [v2:172.21.15.157:6800/422758549,v1:172.21.15.157:6801/422758549] conn(0x7f732c075840 0x7f732c077ce0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:55.219 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.218+0000 7f7345129640 1 -- 172.21.15.157:0/3034665610 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f732c06be30 msgr2=0x7f732c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:55.219 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.218+0000 7f7345129640 1 --2- 172.21.15.157:0/3034665610 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f732c06be30 0x7f732c06e2f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7334009560 tx=0x7f7334009190 comp rx=0 tx=0).stop 2024-04-25T00:00:55.219 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.218+0000 7f7345129640 1 -- 172.21.15.157:0/3034665610 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f734006d530 msgr2=0x7f73400aa9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:55.219 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.218+0000 7f7345129640 1 --2- 172.21.15.157:0/3034665610 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f734006d530 0x7f73400aa9d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f733000c430 tx=0x7f733000c8f0 comp rx=0 tx=0).stop 2024-04-25T00:00:55.219 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.218+0000 7f733f59e640 1 -- 172.21.15.157:0/3034665610 reap_dead start 2024-04-25T00:00:55.219 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.218+0000 7f7345129640 1 -- 172.21.15.157:0/3034665610 shutdown_connections 2024-04-25T00:00:55.219 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.218+0000 7f7345129640 1 -- 172.21.15.157:0/3034665610 >> 172.21.15.157:0/3034665610 conn(0x7f7340066ea0 msgr2=0x7f7340153240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:55.219 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.218+0000 7f7345129640 1 -- 172.21.15.157:0/3034665610 shutdown_connections 2024-04-25T00:00:55.219 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.218+0000 7f7345129640 1 -- 172.21.15.157:0/3034665610 wait complete. 2024-04-25T00:00:55.229 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-25T00:00:55.330 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.330+0000 7f4e584ae640 1 Processor -- start 2024-04-25T00:00:55.330 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.330+0000 7f4e584ae640 1 -- start start 2024-04-25T00:00:55.332 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.331+0000 7f4e584ae640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4e500c68d0 0x7f4e500c6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:55.332 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.331+0000 7f4e584ae640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4e500c58a0 0x7f4e500c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:55.332 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.331+0000 7f4e584ae640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e5006bd80 con 0x7f4e500c71f0 2024-04-25T00:00:55.333 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.331+0000 7f4e584ae640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f4e500c6730 con 0x7f4e500c68d0 2024-04-25T00:00:55.333 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.331+0000 7f4e584ae640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4e501561a0 con 0x7f4e500c58a0 2024-04-25T00:00:55.333 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.331+0000 7f4e55a49640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f4e500c71f0 0x7f4e500c8ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:32982/0 (socket says 172.21.15.157:32982) 2024-04-25T00:00:55.333 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.331+0000 7f4e55a49640 1 -- 172.21.15.157:0/2174814923 learned_addr learned my addr 172.21.15.157:0/2174814923 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-25T00:00:55.333 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.332+0000 7f4e56a4b640 1 --2- 172.21.15.157:0/2174814923 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4e500c58a0 0x7f4e500c92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:55.333 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.332+0000 7f4e5624a640 1 --2- 172.21.15.157:0/2174814923 >> v2:172.21.15.169:3300/0 conn(0x7f4e500c68d0 0x7f4e500c6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:55.333 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.332+0000 7f4e55248640 1 -- 172.21.15.157:0/2174814923 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1836733638 0 0) 0x7f4e5006bd80 con 0x7f4e500c71f0 2024-04-25T00:00:55.333 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.332+0000 7f4e55248640 1 -- 172.21.15.157:0/2174814923 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e3c003520 con 0x7f4e500c71f0 2024-04-25T00:00:55.333 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.332+0000 7f4e5624a640 1 -- 172.21.15.157:0/2174814923 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4e500c58a0 msgr2=0x7f4e500c92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:55.333 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.332+0000 7f4e5624a640 1 --2- 172.21.15.157:0/2174814923 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4e500c58a0 0x7f4e500c92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:55.333 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.332+0000 7f4e5624a640 1 -- 172.21.15.157:0/2174814923 >> v1:172.21.15.157:6789/0 conn(0x7f4e500c71f0 legacy=0x7f4e500c8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:55.333 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.332+0000 7f4e5624a640 1 -- 172.21.15.157:0/2174814923 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e500c6370 con 0x7f4e500c68d0 2024-04-25T00:00:55.333 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.332+0000 7f4e56a4b640 1 --2- 172.21.15.157:0/2174814923 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4e500c58a0 0x7f4e500c92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:00:55.333 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.333+0000 7f4e5624a640 1 --2- 172.21.15.157:0/2174814923 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e500c68d0 0x7f4e500c6cb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4e400097a0 tx=0x7f4e4002a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cbdded877ec99cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:55.334 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.333+0000 7f4e55248640 1 -- 172.21.15.157:0/2174814923 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e40037070 con 0x7f4e500c68d0 2024-04-25T00:00:55.334 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.333+0000 7f4e55248640 1 -- 172.21.15.157:0/2174814923 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e40032440 con 0x7f4e500c68d0 2024-04-25T00:00:55.334 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.333+0000 7f4e55248640 1 -- 172.21.15.157:0/2174814923 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e40032810 con 0x7f4e500c68d0 2024-04-25T00:00:55.337 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.333+0000 7f4e584ae640 1 -- 172.21.15.157:0/2174814923 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e500c68d0 msgr2=0x7f4e500c6cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:55.337 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.333+0000 7f4e584ae640 1 --2- 172.21.15.157:0/2174814923 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e500c68d0 0x7f4e500c6cb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4e400097a0 tx=0x7f4e4002a210 comp rx=0 tx=0).stop 2024-04-25T00:00:55.337 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.333+0000 7f4e584ae640 1 -- 172.21.15.157:0/2174814923 shutdown_connections 2024-04-25T00:00:55.337 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.333+0000 7f4e584ae640 1 --2- 172.21.15.157:0/2174814923 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4e500c58a0 0x7f4e500c92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:55.337 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.333+0000 7f4e584ae640 1 --2- 172.21.15.157:0/2174814923 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e500c68d0 0x7f4e500c6cb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:55.337 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.333+0000 7f4e584ae640 1 -- 172.21.15.157:0/2174814923 >> 172.21.15.157:0/2174814923 conn(0x7f4e500caca0 msgr2=0x7f4e500cb0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:55.337 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.333+0000 7f4e584ae640 1 -- 172.21.15.157:0/2174814923 shutdown_connections 2024-04-25T00:00:55.337 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.334+0000 7f4e584ae640 1 -- 172.21.15.157:0/2174814923 wait complete. 2024-04-25T00:00:55.337 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.335+0000 7f4e584ae640 1 Processor -- start 2024-04-25T00:00:55.337 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.335+0000 7f4e584ae640 1 -- start start 2024-04-25T00:00:55.337 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.335+0000 7f4e584ae640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e500c58a0 0x7f4e50151bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:55.337 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.335+0000 7f4e584ae640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e500c68d0 0x7f4e501eae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:55.337 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.335+0000 7f4e584ae640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4e500c71f0 0x7f4e50148410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:55.337 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.335+0000 7f4e584ae640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4e50152ed0 con 0x7f4e500c58a0 2024-04-25T00:00:55.338 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.335+0000 7f4e584ae640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5006b830 con 0x7f4e500c68d0 2024-04-25T00:00:55.338 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.335+0000 7f4e584ae640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4e50153050 con 0x7f4e500c71f0 2024-04-25T00:00:55.338 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.335+0000 7f4e5624a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e500c58a0 0x7f4e50151bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:55.338 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.335+0000 7f4e5624a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e500c58a0 0x7f4e50151bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37696/0 (socket says 172.21.15.157:37696) 2024-04-25T00:00:55.338 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.335+0000 7f4e5624a640 1 -- 172.21.15.157:0/684497806 learned_addr learned my addr 172.21.15.157:0/684497806 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:00:55.338 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.335+0000 7f4e55a49640 1 --2- 172.21.15.157:0/684497806 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e500c68d0 0x7f4e501eae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:55.338 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.335+0000 7f4e56a4b640 1 --2- 172.21.15.157:0/684497806 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4e500c71f0 0x7f4e50148410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:55.338 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.336+0000 7f4e5624a640 1 -- 172.21.15.157:0/684497806 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4e500c71f0 msgr2=0x7f4e50148410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:55.338 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.336+0000 7f4e5624a640 1 --2- 172.21.15.157:0/684497806 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4e500c71f0 0x7f4e50148410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:55.338 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.336+0000 7f4e5624a640 1 -- 172.21.15.157:0/684497806 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e500c68d0 msgr2=0x7f4e501eae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:55.338 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.336+0000 7f4e5624a640 1 --2- 172.21.15.157:0/684497806 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e500c68d0 0x7f4e501eae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:55.338 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.336+0000 7f4e5624a640 1 -- 172.21.15.157:0/684497806 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e501eb790 con 0x7f4e500c58a0 2024-04-25T00:00:55.338 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.336+0000 7f4e5624a640 1 --2- 172.21.15.157:0/684497806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e500c58a0 0x7f4e50151bf0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4e400096f0 tx=0x7f4e400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:55.338 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.336+0000 7f4e46ffd640 1 -- 172.21.15.157:0/684497806 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e40041020 con 0x7f4e500c58a0 2024-04-25T00:00:55.338 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.336+0000 7f4e46ffd640 1 -- 172.21.15.157:0/684497806 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e4003d700 con 0x7f4e500c58a0 2024-04-25T00:00:55.339 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.336+0000 7f4e584ae640 1 -- 172.21.15.157:0/684497806 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e501eba70 con 0x7f4e500c58a0 2024-04-25T00:00:55.339 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.336+0000 7f4e46ffd640 1 -- 172.21.15.157:0/684497806 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e40037040 con 0x7f4e500c58a0 2024-04-25T00:00:55.339 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.336+0000 7f4e584ae640 1 -- 172.21.15.157:0/684497806 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e50147c10 con 0x7f4e500c58a0 2024-04-25T00:00:55.339 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.337+0000 7f4e584ae640 1 -- 172.21.15.157:0/684497806 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e500681c0 con 0x7f4e500c58a0 2024-04-25T00:00:55.339 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.337+0000 7f4e46ffd640 1 -- 172.21.15.157:0/684497806 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4e40044020 con 0x7f4e500c58a0 2024-04-25T00:00:55.339 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.338+0000 7f4e46ffd640 1 --2- 172.21.15.157:0/684497806 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f4e3406be30 0x7f4e3406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:55.340 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.338+0000 7f4e46ffd640 1 -- 172.21.15.157:0/684497806 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f4e400ab7f0 con 0x7f4e500c58a0 2024-04-25T00:00:55.340 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.338+0000 7f4e55a49640 1 --2- 172.21.15.157:0/684497806 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f4e3406be30 0x7f4e3406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:55.340 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.339+0000 7f4e55a49640 1 --2- 172.21.15.157:0/684497806 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f4e3406be30 0x7f4e3406e2f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4e500c7160 tx=0x7f4e4c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:55.342 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.342+0000 7f4e46ffd640 1 -- 172.21.15.157:0/684497806 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4e4004b030 con 0x7f4e500c58a0 2024-04-25T00:00:55.540 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.540+0000 7f4e584ae640 1 -- 172.21.15.157:0/684497806 --> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f4e500aae00 con 0x7f4e3406be30 2024-04-25T00:00:55.541 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.541+0000 7f4e46ffd640 1 -- 172.21.15.157:0/684497806 <== mgr.4105 v2:172.21.15.169:6832/662378289 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f4e500aae00 con 0x7f4e3406be30 2024-04-25T00:00:55.541 INFO:teuthology.orchestra.run.smithi157.stderr:instructed osd(s) 3 to deep-scrub 2024-04-25T00:00:55.543 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.543+0000 7f4e584ae640 1 -- 172.21.15.157:0/684497806 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f4e3406be30 msgr2=0x7f4e3406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:55.543 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.543+0000 7f4e584ae640 1 --2- 172.21.15.157:0/684497806 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f4e3406be30 0x7f4e3406e2f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4e500c7160 tx=0x7f4e4c009040 comp rx=0 tx=0).stop 2024-04-25T00:00:55.543 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.543+0000 7f4e584ae640 1 -- 172.21.15.157:0/684497806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e500c58a0 msgr2=0x7f4e50151bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:55.543 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.543+0000 7f4e584ae640 1 --2- 172.21.15.157:0/684497806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e500c58a0 0x7f4e50151bf0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4e400096f0 tx=0x7f4e400046c0 comp rx=0 tx=0).stop 2024-04-25T00:00:55.543 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.543+0000 7f4e584ae640 1 -- 172.21.15.157:0/684497806 shutdown_connections 2024-04-25T00:00:55.544 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.543+0000 7f4e584ae640 1 --2- 172.21.15.157:0/684497806 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f4e3406be30 0x7f4e3406e2f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:55.544 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.543+0000 7f4e584ae640 1 --2- 172.21.15.157:0/684497806 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4e500c71f0 0x7f4e50148410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:55.544 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.543+0000 7f4e584ae640 1 --2- 172.21.15.157:0/684497806 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e500c68d0 0x7f4e501eae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:55.544 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.543+0000 7f4e584ae640 1 --2- 172.21.15.157:0/684497806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e500c58a0 0x7f4e50151bf0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:55.544 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.543+0000 7f4e584ae640 1 -- 172.21.15.157:0/684497806 >> 172.21.15.157:0/684497806 conn(0x7f4e500caca0 msgr2=0x7f4e500ab2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:55.544 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.543+0000 7f4e584ae640 1 -- 172.21.15.157:0/684497806 shutdown_connections 2024-04-25T00:00:55.544 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.543+0000 7f4e584ae640 1 -- 172.21.15.157:0/684497806 wait complete. 2024-04-25T00:00:55.555 INFO:tasks.ceph:Scrubbing osd.4 2024-04-25T00:00:55.555 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-25T00:00:55.658 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.658+0000 7fab375a2640 1 Processor -- start 2024-04-25T00:00:55.658 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.659+0000 7fab375a2640 1 -- start start 2024-04-25T00:00:55.659 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.659+0000 7fab375a2640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab30156ac0 0x7fab30156f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:55.660 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.659+0000 7fab375a2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab300c75f0 con 0x7fab3014f3a0 2024-04-25T00:00:55.660 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.659+0000 7fab375a2640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab30155ef0 con 0x7fab3006d530 2024-04-25T00:00:55.660 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.659+0000 7fab375a2640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fab301561d0 con 0x7fab30156ac0 2024-04-25T00:00:55.660 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.660+0000 7fab3533e640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fab3014f3a0 0x7fab300afd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:32992/0 (socket says 172.21.15.157:32992) 2024-04-25T00:00:55.660 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.660+0000 7fab3533e640 1 -- 172.21.15.157:0/2997002198 learned_addr learned my addr 172.21.15.157:0/2997002198 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-25T00:00:55.660 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.660+0000 7fab35b3f640 1 --2- 172.21.15.157:0/2997002198 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab30156ac0 0x7fab30156f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:55.660 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.660+0000 7fab23fff640 1 -- 172.21.15.157:0/2997002198 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1151843794 0 0) 0x7fab300c75f0 con 0x7fab3014f3a0 2024-04-25T00:00:55.660 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.660+0000 7fab23fff640 1 -- 172.21.15.157:0/2997002198 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fab1c003520 con 0x7fab3014f3a0 2024-04-25T00:00:55.660 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.660+0000 7fab23fff640 1 -- 172.21.15.157:0/2997002198 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1463944260 0 0) 0x7fab30155ef0 con 0x7fab3006d530 2024-04-25T00:00:55.660 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.660+0000 7fab23fff640 1 -- 172.21.15.157:0/2997002198 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fab300c75f0 con 0x7fab3006d530 2024-04-25T00:00:55.660 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.660+0000 7fab35b3f640 1 -- 172.21.15.157:0/2997002198 >> v1:172.21.15.169:6789/0 conn(0x7fab3006d530 legacy=0x7fab300b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:55.660 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.660+0000 7fab35b3f640 1 -- 172.21.15.157:0/2997002198 >> v1:172.21.15.157:6789/0 conn(0x7fab3014f3a0 legacy=0x7fab300afd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:55.660 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.660+0000 7fab35b3f640 1 -- 172.21.15.157:0/2997002198 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab300c57d0 con 0x7fab30156ac0 2024-04-25T00:00:55.662 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.661+0000 7fab35b3f640 1 --2- 172.21.15.157:0/2997002198 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab30156ac0 0x7fab30156f50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fab28009df0 tx=0x7fab2802a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12332d4572f5fae9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:55.662 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.661+0000 7fab23fff640 1 -- 172.21.15.157:0/2997002198 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab28007740 con 0x7fab30156ac0 2024-04-25T00:00:55.662 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.661+0000 7fab23fff640 1 -- 172.21.15.157:0/2997002198 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab280078e0 con 0x7fab30156ac0 2024-04-25T00:00:55.662 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.661+0000 7fab23fff640 1 -- 172.21.15.157:0/2997002198 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab28007cb0 con 0x7fab30156ac0 2024-04-25T00:00:55.662 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.661+0000 7fab375a2640 1 -- 172.21.15.157:0/2997002198 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab30156ac0 msgr2=0x7fab30156f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:55.662 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.661+0000 7fab375a2640 1 --2- 172.21.15.157:0/2997002198 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab30156ac0 0x7fab30156f50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fab28009df0 tx=0x7fab2802a710 comp rx=0 tx=0).stop 2024-04-25T00:00:55.662 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.661+0000 7fab375a2640 1 -- 172.21.15.157:0/2997002198 shutdown_connections 2024-04-25T00:00:55.662 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.661+0000 7fab375a2640 1 --2- 172.21.15.157:0/2997002198 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab30156ac0 0x7fab30156f50 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:55.662 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.661+0000 7fab375a2640 1 -- 172.21.15.157:0/2997002198 >> 172.21.15.157:0/2997002198 conn(0x7fab30066ea0 msgr2=0x7fab300672b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:55.663 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.662+0000 7fab375a2640 1 -- 172.21.15.157:0/2997002198 shutdown_connections 2024-04-25T00:00:55.663 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.662+0000 7fab375a2640 1 -- 172.21.15.157:0/2997002198 wait complete. 2024-04-25T00:00:55.663 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.663+0000 7fab375a2640 1 Processor -- start 2024-04-25T00:00:55.663 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.663+0000 7fab375a2640 1 -- start start 2024-04-25T00:00:55.665 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.663+0000 7fab375a2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab3006d530 0x7fab301ec090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:55.665 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.663+0000 7fab375a2640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab3014f3a0 0x7fab300c0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:55.665 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.663+0000 7fab375a2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab30156ac0 0x7fab30152760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:55.665 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.663+0000 7fab375a2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fab3014f780 con 0x7fab30156ac0 2024-04-25T00:00:55.665 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.663+0000 7fab375a2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fab300c5d90 con 0x7fab3006d530 2024-04-25T00:00:55.665 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.663+0000 7fab375a2640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fab30157490 con 0x7fab3014f3a0 2024-04-25T00:00:55.665 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.663+0000 7fab35b3f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab30156ac0 0x7fab30152760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:55.665 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.663+0000 7fab35b3f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab30156ac0 0x7fab30152760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37704/0 (socket says 172.21.15.157:37704) 2024-04-25T00:00:55.665 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.663+0000 7fab35b3f640 1 -- 172.21.15.157:0/4091774197 learned_addr learned my addr 172.21.15.157:0/4091774197 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:00:55.665 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.663+0000 7fab3533e640 1 --2- 172.21.15.157:0/4091774197 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab3006d530 0x7fab301ec090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:55.665 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.663+0000 7fab34b3d640 1 --2- 172.21.15.157:0/4091774197 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab3014f3a0 0x7fab300c0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:55.665 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.664+0000 7fab35b3f640 1 -- 172.21.15.157:0/4091774197 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab3014f3a0 msgr2=0x7fab300c0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:55.665 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.664+0000 7fab35b3f640 1 --2- 172.21.15.157:0/4091774197 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab3014f3a0 0x7fab300c0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:55.665 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.664+0000 7fab35b3f640 1 -- 172.21.15.157:0/4091774197 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab3006d530 msgr2=0x7fab301ec090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:55.666 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.664+0000 7fab35b3f640 1 --2- 172.21.15.157:0/4091774197 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab3006d530 0x7fab301ec090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:55.666 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.664+0000 7fab35b3f640 1 -- 172.21.15.157:0/4091774197 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab301518a0 con 0x7fab30156ac0 2024-04-25T00:00:55.666 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.664+0000 7fab35b3f640 1 --2- 172.21.15.157:0/4091774197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab30156ac0 0x7fab30152760 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fab2802ad00 tx=0x7fab28004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:55.666 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.664+0000 7fab21ffb640 1 -- 172.21.15.157:0/4091774197 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2803f070 con 0x7fab30156ac0 2024-04-25T00:00:55.666 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.664+0000 7fab21ffb640 1 -- 172.21.15.157:0/4091774197 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab28031bc0 con 0x7fab30156ac0 2024-04-25T00:00:55.666 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.664+0000 7fab375a2640 1 -- 172.21.15.157:0/4091774197 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab3014b4d0 con 0x7fab30156ac0 2024-04-25T00:00:55.666 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.664+0000 7fab21ffb640 1 -- 172.21.15.157:0/4091774197 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab28009450 con 0x7fab30156ac0 2024-04-25T00:00:55.666 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.664+0000 7fab375a2640 1 -- 172.21.15.157:0/4091774197 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab30150360 con 0x7fab30156ac0 2024-04-25T00:00:55.666 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.665+0000 7fab375a2640 1 -- 172.21.15.157:0/4091774197 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fab300681c0 con 0x7fab30156ac0 2024-04-25T00:00:55.667 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.666+0000 7fab21ffb640 1 -- 172.21.15.157:0/4091774197 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fab28007790 con 0x7fab30156ac0 2024-04-25T00:00:55.668 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.666+0000 7fab21ffb640 1 --2- 172.21.15.157:0/4091774197 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fab1406be60 0x7fab1406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:55.668 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.666+0000 7fab21ffb640 1 -- 172.21.15.157:0/4091774197 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fab280acd70 con 0x7fab30156ac0 2024-04-25T00:00:55.668 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.666+0000 7fab3533e640 1 --2- 172.21.15.157:0/4091774197 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fab1406be60 0x7fab1406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:55.668 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.666+0000 7fab21ffb640 1 --2- 172.21.15.157:0/4091774197 >> [v2:172.21.15.169:6800/321910099,v1:172.21.15.169:6801/321910099] conn(0x7fab14075880 0x7fab14077d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:55.668 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.666+0000 7fab21ffb640 1 -- 172.21.15.157:0/4091774197 --> [v2:172.21.15.169:6800/321910099,v1:172.21.15.169:6801/321910099] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fab140783f0 con 0x7fab14075880 2024-04-25T00:00:55.668 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.666+0000 7fab21ffb640 1 -- 172.21.15.157:0/4091774197 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fab280ad120 con 0x7fab30156ac0 2024-04-25T00:00:55.668 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.666+0000 7fab34b3d640 1 --2- 172.21.15.157:0/4091774197 >> [v2:172.21.15.169:6800/321910099,v1:172.21.15.169:6801/321910099] conn(0x7fab14075880 0x7fab14077d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:55.668 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.667+0000 7fab3533e640 1 --2- 172.21.15.157:0/4091774197 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fab1406be60 0x7fab1406e320 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fab24000f60 tx=0x7fab24007d70 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:55.668 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.667+0000 7fab34b3d640 1 --2- 172.21.15.157:0/4091774197 >> [v2:172.21.15.169:6800/321910099,v1:172.21.15.169:6801/321910099] conn(0x7fab14075880 0x7fab14077d20 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:55.668 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.667+0000 7fab21ffb640 1 -- 172.21.15.157:0/4091774197 <== osd.4 v2:172.21.15.169:6800/321910099 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fab140783f0 con 0x7fab14075880 2024-04-25T00:00:55.680 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.680+0000 7fab375a2640 1 -- 172.21.15.157:0/4091774197 --> [v2:172.21.15.169:6800/321910099,v1:172.21.15.169:6801/321910099] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fab3006d310 con 0x7fab14075880 2024-04-25T00:00:55.680 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.681+0000 7fab21ffb640 1 -- 172.21.15.157:0/4091774197 <== osd.4 v2:172.21.15.169:6800/321910099 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fab3006d310 con 0x7fab14075880 2024-04-25T00:00:55.680 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-04-25T00:00:55.681 INFO:teuthology.orchestra.run.smithi157.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T00:00:55.681 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-04-25T00:00:55.682 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.681+0000 7fab375a2640 1 -- 172.21.15.157:0/4091774197 >> [v2:172.21.15.169:6800/321910099,v1:172.21.15.169:6801/321910099] conn(0x7fab14075880 msgr2=0x7fab14077d20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:55.682 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.681+0000 7fab375a2640 1 --2- 172.21.15.157:0/4091774197 >> [v2:172.21.15.169:6800/321910099,v1:172.21.15.169:6801/321910099] conn(0x7fab14075880 0x7fab14077d20 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:55.682 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.681+0000 7fab375a2640 1 -- 172.21.15.157:0/4091774197 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fab1406be60 msgr2=0x7fab1406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:55.682 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.681+0000 7fab375a2640 1 --2- 172.21.15.157:0/4091774197 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fab1406be60 0x7fab1406e320 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fab24000f60 tx=0x7fab24007d70 comp rx=0 tx=0).stop 2024-04-25T00:00:55.682 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.681+0000 7fab375a2640 1 -- 172.21.15.157:0/4091774197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab30156ac0 msgr2=0x7fab30152760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:55.682 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.681+0000 7fab375a2640 1 --2- 172.21.15.157:0/4091774197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab30156ac0 0x7fab30152760 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fab2802ad00 tx=0x7fab28004a40 comp rx=0 tx=0).stop 2024-04-25T00:00:55.682 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.681+0000 7fab35b3f640 1 -- 172.21.15.157:0/4091774197 reap_dead start 2024-04-25T00:00:55.682 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.681+0000 7fab375a2640 1 -- 172.21.15.157:0/4091774197 shutdown_connections 2024-04-25T00:00:55.682 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.681+0000 7fab375a2640 1 -- 172.21.15.157:0/4091774197 >> 172.21.15.157:0/4091774197 conn(0x7fab30066ea0 msgr2=0x7fab301591b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:55.682 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.681+0000 7fab375a2640 1 -- 172.21.15.157:0/4091774197 shutdown_connections 2024-04-25T00:00:55.682 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.681+0000 7fab375a2640 1 -- 172.21.15.157:0/4091774197 wait complete. 2024-04-25T00:00:55.691 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-25T00:00:55.792 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.790+0000 7f5b4db89640 1 Processor -- start 2024-04-25T00:00:55.792 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.790+0000 7f5b4db89640 1 -- start start 2024-04-25T00:00:55.792 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.791+0000 7f5b4db89640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b480c68d0 0x7f5b480c6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:55.792 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.791+0000 7f5b4db89640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5b480c58a0 0x7f5b480c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:55.792 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.791+0000 7f5b4db89640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b4806bd80 con 0x7f5b480c71f0 2024-04-25T00:00:55.792 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.791+0000 7f5b4db89640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f5b480c6730 con 0x7f5b480c58a0 2024-04-25T00:00:55.792 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.791+0000 7f5b4db89640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5b481561a0 con 0x7f5b480c68d0 2024-04-25T00:00:55.792 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.791+0000 7f5b47fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5b480c58a0 0x7f5b480c92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:55.792 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.791+0000 7f5b47fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5b480c58a0 0x7f5b480c92b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37708/0 (socket says 172.21.15.157:37708) 2024-04-25T00:00:55.793 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.791+0000 7f5b47fff640 1 -- 172.21.15.157:0/3762165326 learned_addr learned my addr 172.21.15.157:0/3762165326 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:00:55.793 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.791+0000 7f5b477fe640 1 --2- 172.21.15.157:0/3762165326 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b480c68d0 0x7f5b480c6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:55.793 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.792+0000 7f5b47fff640 1 -- 172.21.15.157:0/3762165326 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b480c68d0 msgr2=0x7f5b480c6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:55.793 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.792+0000 7f5b47fff640 1 --2- 172.21.15.157:0/3762165326 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b480c68d0 0x7f5b480c6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:55.793 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.792+0000 7f5b47fff640 1 -- 172.21.15.157:0/3762165326 >> v1:172.21.15.169:6789/0 conn(0x7f5b480c71f0 legacy=0x7f5b480c8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:55.793 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.792+0000 7f5b47fff640 1 -- 172.21.15.157:0/3762165326 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b480c6370 con 0x7f5b480c58a0 2024-04-25T00:00:55.793 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.792+0000 7f5b47fff640 1 --2- 172.21.15.157:0/3762165326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b480c58a0 0x7f5b480c92b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5b40009e50 tx=0x7f5b4002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49849abc9b8481ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:55.793 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.792+0000 7f5b467fc640 1 -- 172.21.15.157:0/3762165326 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b40036070 con 0x7f5b480c58a0 2024-04-25T00:00:55.793 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.792+0000 7f5b467fc640 1 -- 172.21.15.157:0/3762165326 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b40031600 con 0x7f5b480c58a0 2024-04-25T00:00:55.793 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.792+0000 7f5b467fc640 1 -- 172.21.15.157:0/3762165326 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b400319d0 con 0x7f5b480c58a0 2024-04-25T00:00:55.794 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.793+0000 7f5b4db89640 1 -- 172.21.15.157:0/3762165326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b480c58a0 msgr2=0x7f5b480c92b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:55.794 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.793+0000 7f5b4db89640 1 --2- 172.21.15.157:0/3762165326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b480c58a0 0x7f5b480c92b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5b40009e50 tx=0x7f5b4002a7a0 comp rx=0 tx=0).stop 2024-04-25T00:00:55.794 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.793+0000 7f5b4db89640 1 -- 172.21.15.157:0/3762165326 shutdown_connections 2024-04-25T00:00:55.795 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.793+0000 7f5b4db89640 1 --2- 172.21.15.157:0/3762165326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b480c58a0 0x7f5b480c92b0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:55.795 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.793+0000 7f5b4db89640 1 --2- 172.21.15.157:0/3762165326 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b480c68d0 0x7f5b480c6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:55.795 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.793+0000 7f5b4db89640 1 -- 172.21.15.157:0/3762165326 >> 172.21.15.157:0/3762165326 conn(0x7f5b480caca0 msgr2=0x7f5b480cb0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:55.795 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.793+0000 7f5b4db89640 1 -- 172.21.15.157:0/3762165326 shutdown_connections 2024-04-25T00:00:55.795 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.793+0000 7f5b4db89640 1 -- 172.21.15.157:0/3762165326 wait complete. 2024-04-25T00:00:55.795 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.794+0000 7f5b4db89640 1 Processor -- start 2024-04-25T00:00:55.795 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.794+0000 7f5b4db89640 1 -- start start 2024-04-25T00:00:55.797 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.795+0000 7f5b4db89640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b480c58a0 0x7f5b48154c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:55.797 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.795+0000 7f5b4db89640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b480c68d0 0x7f5b481460e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:55.797 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.795+0000 7f5b4db89640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b480c71f0 0x7f5b48146620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:55.797 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.795+0000 7f5b4db89640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5b48152ed0 con 0x7f5b480c68d0 2024-04-25T00:00:55.797 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.795+0000 7f5b4db89640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4806b830 con 0x7f5b480c58a0 2024-04-25T00:00:55.798 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.795+0000 7f5b4db89640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5b48153050 con 0x7f5b480c71f0 2024-04-25T00:00:55.798 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.795+0000 7f5b46ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b480c68d0 0x7f5b481460e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:55.798 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.795+0000 7f5b46ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b480c68d0 0x7f5b481460e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37712/0 (socket says 172.21.15.157:37712) 2024-04-25T00:00:55.798 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.795+0000 7f5b46ffd640 1 -- 172.21.15.157:0/481615278 learned_addr learned my addr 172.21.15.157:0/481615278 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:00:55.798 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.795+0000 7f5b477fe640 1 --2- 172.21.15.157:0/481615278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b480c58a0 0x7f5b48154c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:55.798 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.795+0000 7f5b47fff640 1 --2- 172.21.15.157:0/481615278 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b480c71f0 0x7f5b48146620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:55.798 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.795+0000 7f5b46ffd640 1 -- 172.21.15.157:0/481615278 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b480c71f0 msgr2=0x7f5b48146620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:55.798 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.795+0000 7f5b46ffd640 1 --2- 172.21.15.157:0/481615278 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b480c71f0 0x7f5b48146620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:55.798 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.795+0000 7f5b46ffd640 1 -- 172.21.15.157:0/481615278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b480c58a0 msgr2=0x7f5b48154c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:55.798 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.795+0000 7f5b46ffd640 1 --2- 172.21.15.157:0/481615278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b480c58a0 0x7f5b48154c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:55.798 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.795+0000 7f5b46ffd640 1 -- 172.21.15.157:0/481615278 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b48146f30 con 0x7f5b480c68d0 2024-04-25T00:00:55.798 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.795+0000 7f5b477fe640 1 --2- 172.21.15.157:0/481615278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b480c58a0 0x7f5b48154c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:00:55.798 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.796+0000 7f5b46ffd640 1 --2- 172.21.15.157:0/481615278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b480c68d0 0x7f5b481460e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5b3800c400 tx=0x7f5b3800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:55.798 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.796+0000 7f5b2ffff640 1 -- 172.21.15.157:0/481615278 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b380052d0 con 0x7f5b480c68d0 2024-04-25T00:00:55.799 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.796+0000 7f5b2ffff640 1 -- 172.21.15.157:0/481615278 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b38005490 con 0x7f5b480c68d0 2024-04-25T00:00:55.799 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.796+0000 7f5b4db89640 1 -- 172.21.15.157:0/481615278 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b48147210 con 0x7f5b480c68d0 2024-04-25T00:00:55.799 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.796+0000 7f5b2ffff640 1 -- 172.21.15.157:0/481615278 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b38007260 con 0x7f5b480c68d0 2024-04-25T00:00:55.799 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.796+0000 7f5b4db89640 1 -- 172.21.15.157:0/481615278 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b481551b0 con 0x7f5b480c68d0 2024-04-25T00:00:55.799 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.797+0000 7f5b4db89640 1 -- 172.21.15.157:0/481615278 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b480681c0 con 0x7f5b480c68d0 2024-04-25T00:00:55.799 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.797+0000 7f5b2ffff640 1 -- 172.21.15.157:0/481615278 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5b38007400 con 0x7f5b480c68d0 2024-04-25T00:00:55.800 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.798+0000 7f5b2ffff640 1 --2- 172.21.15.157:0/481615278 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f5b3406be30 0x7f5b3406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:55.800 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.798+0000 7f5b2ffff640 1 -- 172.21.15.157:0/481615278 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f5b3808a6b0 con 0x7f5b480c68d0 2024-04-25T00:00:55.800 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.798+0000 7f5b477fe640 1 --2- 172.21.15.157:0/481615278 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f5b3406be30 0x7f5b3406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:55.800 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.798+0000 7f5b477fe640 1 --2- 172.21.15.157:0/481615278 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f5b3406be30 0x7f5b3406e2f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c009e90 tx=0x7f5b3c009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:55.801 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.802+0000 7f5b2ffff640 1 -- 172.21.15.157:0/481615278 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5b380586a0 con 0x7f5b480c68d0 2024-04-25T00:00:55.981 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.982+0000 7f5b4db89640 1 -- 172.21.15.157:0/481615278 --> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f5b480c5680 con 0x7f5b3406be30 2024-04-25T00:00:55.982 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.982+0000 7f5b2ffff640 1 -- 172.21.15.157:0/481615278 <== mgr.4105 v2:172.21.15.169:6832/662378289 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f5b480c5680 con 0x7f5b3406be30 2024-04-25T00:00:55.982 INFO:teuthology.orchestra.run.smithi157.stderr:instructed osd(s) 4 to deep-scrub 2024-04-25T00:00:55.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.985+0000 7f5b4db89640 1 -- 172.21.15.157:0/481615278 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f5b3406be30 msgr2=0x7f5b3406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:55.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.985+0000 7f5b4db89640 1 --2- 172.21.15.157:0/481615278 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f5b3406be30 0x7f5b3406e2f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c009e90 tx=0x7f5b3c009190 comp rx=0 tx=0).stop 2024-04-25T00:00:55.985 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.985+0000 7f5b4db89640 1 -- 172.21.15.157:0/481615278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b480c68d0 msgr2=0x7f5b481460e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:55.985 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.985+0000 7f5b4db89640 1 --2- 172.21.15.157:0/481615278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b480c68d0 0x7f5b481460e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5b3800c400 tx=0x7f5b3800c8c0 comp rx=0 tx=0).stop 2024-04-25T00:00:55.985 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.985+0000 7f5b4db89640 1 -- 172.21.15.157:0/481615278 shutdown_connections 2024-04-25T00:00:55.985 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.985+0000 7f5b4db89640 1 --2- 172.21.15.157:0/481615278 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f5b3406be30 0x7f5b3406e2f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:55.985 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.985+0000 7f5b4db89640 1 --2- 172.21.15.157:0/481615278 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b480c71f0 0x7f5b48146620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:55.985 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.985+0000 7f5b4db89640 1 --2- 172.21.15.157:0/481615278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b480c68d0 0x7f5b481460e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:55.985 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.985+0000 7f5b4db89640 1 --2- 172.21.15.157:0/481615278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b480c58a0 0x7f5b48154c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:55.985 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.985+0000 7f5b4db89640 1 -- 172.21.15.157:0/481615278 >> 172.21.15.157:0/481615278 conn(0x7f5b480caca0 msgr2=0x7f5b481506f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:55.985 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.985+0000 7f5b4db89640 1 -- 172.21.15.157:0/481615278 shutdown_connections 2024-04-25T00:00:55.985 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:55.985+0000 7f5b4db89640 1 -- 172.21.15.157:0/481615278 wait complete. 2024-04-25T00:00:55.997 INFO:tasks.ceph:Scrubbing osd.5 2024-04-25T00:00:55.997 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-25T00:00:56.110 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.111+0000 7ff3d5729640 1 Processor -- start 2024-04-25T00:00:56.110 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.111+0000 7ff3d5729640 1 -- start start 2024-04-25T00:00:56.112 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.111+0000 7ff3d5729640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff3d0152ad0 0x7ff3d00c29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:56.112 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.111+0000 7ff3d5729640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff3d00c5920 0x7ff3d00a7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:56.112 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.111+0000 7ff3d5729640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3d00c1f70 con 0x7ff3d00c5490 2024-04-25T00:00:56.112 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.111+0000 7ff3d5729640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7ff3d0156140 con 0x7ff3d00c5920 2024-04-25T00:00:56.112 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.111+0000 7ff3d5729640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7ff3d00c21e0 con 0x7ff3d0152ad0 2024-04-25T00:00:56.112 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.112+0000 7ff3cf7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff3d00c5920 0x7ff3d00a7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:56.112 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.112+0000 7ff3cf7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff3d00c5920 0x7ff3d00a7c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37720/0 (socket says 172.21.15.157:37720) 2024-04-25T00:00:56.113 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.112+0000 7ff3cf7fe640 1 -- 172.21.15.157:0/357961796 learned_addr learned my addr 172.21.15.157:0/357961796 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:00:56.113 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.112+0000 7ff3ceffd640 1 --2- 172.21.15.157:0/357961796 >> v2:172.21.15.169:3300/0 conn(0x7ff3d0152ad0 0x7ff3d00c29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:56.113 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.112+0000 7ff3cf7fe640 1 -- 172.21.15.157:0/357961796 >> v2:172.21.15.169:3300/0 conn(0x7ff3d0152ad0 msgr2=0x7ff3d00c29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:56.113 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.112+0000 7ff3cf7fe640 1 --2- 172.21.15.157:0/357961796 >> v2:172.21.15.169:3300/0 conn(0x7ff3d0152ad0 0x7ff3d00c29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:56.113 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.112+0000 7ff3cf7fe640 1 -- 172.21.15.157:0/357961796 >> v1:172.21.15.169:6789/0 conn(0x7ff3d00c5490 legacy=0x7ff3d01556c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:56.113 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.112+0000 7ff3cf7fe640 1 -- 172.21.15.157:0/357961796 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3d00c24c0 con 0x7ff3d00c5920 2024-04-25T00:00:56.113 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.112+0000 7ff3cf7fe640 1 --2- 172.21.15.157:0/357961796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3d00c5920 0x7ff3d00a7c70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff3d006bcd0 tx=0x7ff3c802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81841e5f8bbdf680 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:56.113 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.113+0000 7ff3ce7fc640 1 -- 172.21.15.157:0/357961796 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c8036070 con 0x7ff3d00c5920 2024-04-25T00:00:56.113 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.113+0000 7ff3ce7fc640 1 -- 172.21.15.157:0/357961796 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3c8031600 con 0x7ff3d00c5920 2024-04-25T00:00:56.113 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.113+0000 7ff3ce7fc640 1 -- 172.21.15.157:0/357961796 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c80319d0 con 0x7ff3d00c5920 2024-04-25T00:00:56.116 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.113+0000 7ff3d5729640 1 -- 172.21.15.157:0/357961796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3d00c5920 msgr2=0x7ff3d00a7c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:56.116 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.113+0000 7ff3d5729640 1 --2- 172.21.15.157:0/357961796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3d00c5920 0x7ff3d00a7c70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff3d006bcd0 tx=0x7ff3c802a730 comp rx=0 tx=0).stop 2024-04-25T00:00:56.116 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.113+0000 7ff3d5729640 1 -- 172.21.15.157:0/357961796 shutdown_connections 2024-04-25T00:00:56.117 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.113+0000 7ff3d5729640 1 --2- 172.21.15.157:0/357961796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3d00c5920 0x7ff3d00a7c70 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:56.117 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.113+0000 7ff3d5729640 1 --2- 172.21.15.157:0/357961796 >> v2:172.21.15.169:3300/0 conn(0x7ff3d0152ad0 0x7ff3d00c29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:56.117 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.113+0000 7ff3d5729640 1 -- 172.21.15.157:0/357961796 >> 172.21.15.157:0/357961796 conn(0x7ff3d00c8aa0 msgr2=0x7ff3d00c8e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:56.117 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.113+0000 7ff3d5729640 1 -- 172.21.15.157:0/357961796 shutdown_connections 2024-04-25T00:00:56.117 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.113+0000 7ff3d5729640 1 -- 172.21.15.157:0/357961796 wait complete. 2024-04-25T00:00:56.117 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.114+0000 7ff3d5729640 1 Processor -- start 2024-04-25T00:00:56.117 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.115+0000 7ff3d5729640 1 -- start start 2024-04-25T00:00:56.117 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.115+0000 7ff3d5729640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3d00c5490 0x7ff3d00acbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:56.117 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.115+0000 7ff3d5729640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3d00c5920 0x7ff3d01568e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:56.117 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.115+0000 7ff3d5729640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3d0152ad0 0x7ff3d0156e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:56.117 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.115+0000 7ff3d5729640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d0152eb0 con 0x7ff3d0152ad0 2024-04-25T00:00:56.117 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.115+0000 7ff3d5729640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d0155dd0 con 0x7ff3d00c5920 2024-04-25T00:00:56.117 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.115+0000 7ff3d5729640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff3d0153030 con 0x7ff3d00c5490 2024-04-25T00:00:56.117 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.115+0000 7ff3cf7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3d0152ad0 0x7ff3d0156e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:56.118 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.115+0000 7ff3cf7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3d0152ad0 0x7ff3d0156e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37728/0 (socket says 172.21.15.157:37728) 2024-04-25T00:00:56.118 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.115+0000 7ff3cf7fe640 1 -- 172.21.15.157:0/2957565105 learned_addr learned my addr 172.21.15.157:0/2957565105 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:00:56.118 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.115+0000 7ff3c7fff640 1 --2- 172.21.15.157:0/2957565105 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3d00c5920 0x7ff3d01568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:56.118 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.115+0000 7ff3ceffd640 1 --2- 172.21.15.157:0/2957565105 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3d00c5490 0x7ff3d00acbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:56.118 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.116+0000 7ff3cf7fe640 1 -- 172.21.15.157:0/2957565105 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3d00c5490 msgr2=0x7ff3d00acbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:56.118 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.116+0000 7ff3cf7fe640 1 --2- 172.21.15.157:0/2957565105 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3d00c5490 0x7ff3d00acbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:56.118 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.116+0000 7ff3cf7fe640 1 -- 172.21.15.157:0/2957565105 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3d00c5920 msgr2=0x7ff3d01568e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:56.118 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.116+0000 7ff3cf7fe640 1 --2- 172.21.15.157:0/2957565105 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3d00c5920 0x7ff3d01568e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:56.118 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.116+0000 7ff3cf7fe640 1 -- 172.21.15.157:0/2957565105 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3d01594f0 con 0x7ff3d0152ad0 2024-04-25T00:00:56.118 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.116+0000 7ff3cf7fe640 1 --2- 172.21.15.157:0/2957565105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3d0152ad0 0x7ff3d0156e20 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff3c8009f40 tx=0x7ff3c8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:56.118 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.116+0000 7ff3c77fe640 1 -- 172.21.15.157:0/2957565105 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c8036050 con 0x7ff3d0152ad0 2024-04-25T00:00:56.118 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.116+0000 7ff3c77fe640 1 -- 172.21.15.157:0/2957565105 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3c8042070 con 0x7ff3d0152ad0 2024-04-25T00:00:56.118 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.116+0000 7ff3d5729640 1 -- 172.21.15.157:0/2957565105 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3d01eb140 con 0x7ff3d0152ad0 2024-04-25T00:00:56.119 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.116+0000 7ff3c77fe640 1 -- 172.21.15.157:0/2957565105 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c8009510 con 0x7ff3d0152ad0 2024-04-25T00:00:56.119 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.116+0000 7ff3d5729640 1 -- 172.21.15.157:0/2957565105 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3d00ad100 con 0x7ff3d0152ad0 2024-04-25T00:00:56.119 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.117+0000 7ff3d5729640 1 -- 172.21.15.157:0/2957565105 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff3d01eaf30 con 0x7ff3d0152ad0 2024-04-25T00:00:56.120 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.118+0000 7ff3c77fe640 1 -- 172.21.15.157:0/2957565105 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff3c8007bf0 con 0x7ff3d0152ad0 2024-04-25T00:00:56.120 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.118+0000 7ff3c77fe640 1 --2- 172.21.15.157:0/2957565105 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7ff3b006be30 0x7ff3b006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:56.120 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.118+0000 7ff3c77fe640 1 -- 172.21.15.157:0/2957565105 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7ff3c80abe30 con 0x7ff3d0152ad0 2024-04-25T00:00:56.120 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.118+0000 7ff3ceffd640 1 --2- 172.21.15.157:0/2957565105 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7ff3b006be30 0x7ff3b006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:56.120 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.118+0000 7ff3c77fe640 1 --2- 172.21.15.157:0/2957565105 >> [v2:172.21.15.169:6824/243321711,v1:172.21.15.169:6825/243321711] conn(0x7ff3b00757d0 0x7ff3b0077c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:56.120 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.118+0000 7ff3c77fe640 1 -- 172.21.15.157:0/2957565105 --> [v2:172.21.15.169:6824/243321711,v1:172.21.15.169:6825/243321711] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff3b0078340 con 0x7ff3b00757d0 2024-04-25T00:00:56.120 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.118+0000 7ff3c77fe640 1 -- 172.21.15.157:0/2957565105 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff3c80ae530 con 0x7ff3d0152ad0 2024-04-25T00:00:56.121 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.118+0000 7ff3c7fff640 1 --2- 172.21.15.157:0/2957565105 >> [v2:172.21.15.169:6824/243321711,v1:172.21.15.169:6825/243321711] conn(0x7ff3b00757d0 0x7ff3b0077c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:56.121 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.119+0000 7ff3ceffd640 1 --2- 172.21.15.157:0/2957565105 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7ff3b006be30 0x7ff3b006e2f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0009b40 tx=0x7ff3c0009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:56.121 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.119+0000 7ff3c7fff640 1 --2- 172.21.15.157:0/2957565105 >> [v2:172.21.15.169:6824/243321711,v1:172.21.15.169:6825/243321711] conn(0x7ff3b00757d0 0x7ff3b0077c70 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:56.121 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.120+0000 7ff3c77fe640 1 -- 172.21.15.157:0/2957565105 <== osd.5 v2:172.21.15.169:6824/243321711 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff3b0078340 con 0x7ff3b00757d0 2024-04-25T00:00:56.134 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.134+0000 7ff3d5729640 1 -- 172.21.15.157:0/2957565105 --> [v2:172.21.15.169:6824/243321711,v1:172.21.15.169:6825/243321711] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7ff3d00681c0 con 0x7ff3b00757d0 2024-04-25T00:00:56.134 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.134+0000 7ff3c77fe640 1 -- 172.21.15.157:0/2957565105 <== osd.5 v2:172.21.15.169:6824/243321711 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7ff3d00681c0 con 0x7ff3b00757d0 2024-04-25T00:00:56.134 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-04-25T00:00:56.135 INFO:teuthology.orchestra.run.smithi157.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T00:00:56.135 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-04-25T00:00:56.136 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.135+0000 7ff3d5729640 1 -- 172.21.15.157:0/2957565105 >> [v2:172.21.15.169:6824/243321711,v1:172.21.15.169:6825/243321711] conn(0x7ff3b00757d0 msgr2=0x7ff3b0077c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:56.136 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.135+0000 7ff3d5729640 1 --2- 172.21.15.157:0/2957565105 >> [v2:172.21.15.169:6824/243321711,v1:172.21.15.169:6825/243321711] conn(0x7ff3b00757d0 0x7ff3b0077c70 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:56.136 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.135+0000 7ff3d5729640 1 -- 172.21.15.157:0/2957565105 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7ff3b006be30 msgr2=0x7ff3b006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:56.136 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.135+0000 7ff3d5729640 1 --2- 172.21.15.157:0/2957565105 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7ff3b006be30 0x7ff3b006e2f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0009b40 tx=0x7ff3c0009190 comp rx=0 tx=0).stop 2024-04-25T00:00:56.136 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.135+0000 7ff3d5729640 1 -- 172.21.15.157:0/2957565105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3d0152ad0 msgr2=0x7ff3d0156e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:56.136 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.135+0000 7ff3d5729640 1 --2- 172.21.15.157:0/2957565105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3d0152ad0 0x7ff3d0156e20 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff3c8009f40 tx=0x7ff3c8004a40 comp rx=0 tx=0).stop 2024-04-25T00:00:56.136 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.135+0000 7ff3cf7fe640 1 -- 172.21.15.157:0/2957565105 reap_dead start 2024-04-25T00:00:56.136 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.135+0000 7ff3d5729640 1 -- 172.21.15.157:0/2957565105 shutdown_connections 2024-04-25T00:00:56.136 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.135+0000 7ff3d5729640 1 -- 172.21.15.157:0/2957565105 >> 172.21.15.157:0/2957565105 conn(0x7ff3d00c8aa0 msgr2=0x7ff3d0154120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:56.136 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.135+0000 7ff3d5729640 1 -- 172.21.15.157:0/2957565105 shutdown_connections 2024-04-25T00:00:56.136 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.135+0000 7ff3d5729640 1 -- 172.21.15.157:0/2957565105 wait complete. 2024-04-25T00:00:56.145 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-25T00:00:56.246 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.246+0000 7fca2e5e9640 1 Processor -- start 2024-04-25T00:00:56.246 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.246+0000 7fca2e5e9640 1 -- start start 2024-04-25T00:00:56.248 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.247+0000 7fca2e5e9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fca280c68d0 0x7fca280c6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:56.248 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.247+0000 7fca2e5e9640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca280c71f0 0x7fca280c8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:56.248 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.247+0000 7fca2e5e9640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fca280c58a0 0x7fca280c90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:56.248 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.247+0000 7fca2e5e9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fca280c6730 con 0x7fca280c68d0 2024-04-25T00:00:56.248 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.247+0000 7fca2e5e9640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fca280c9620 con 0x7fca280c58a0 2024-04-25T00:00:56.248 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.247+0000 7fca2e5e9640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fca2806bd80 con 0x7fca280c71f0 2024-04-25T00:00:56.249 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.247+0000 7fca27fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fca280c68d0 0x7fca280c6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:56.249 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.247+0000 7fca27fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fca280c68d0 0x7fca280c6cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37732/0 (socket says 172.21.15.157:37732) 2024-04-25T00:00:56.249 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.247+0000 7fca27fff640 1 -- 172.21.15.157:0/4261118932 learned_addr learned my addr 172.21.15.157:0/4261118932 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:00:56.249 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.247+0000 7fca277fe640 1 --2- 172.21.15.157:0/4261118932 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca280c71f0 0x7fca280c8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:56.249 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.248+0000 7fca2cb86640 1 --2- 172.21.15.157:0/4261118932 >> v2:172.21.15.169:3300/0 conn(0x7fca280c58a0 0x7fca280c90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:56.249 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.248+0000 7fca27fff640 1 -- 172.21.15.157:0/4261118932 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca280c71f0 msgr2=0x7fca280c8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:56.249 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.248+0000 7fca27fff640 1 --2- 172.21.15.157:0/4261118932 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca280c71f0 0x7fca280c8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:56.249 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.248+0000 7fca27fff640 1 -- 172.21.15.157:0/4261118932 >> v2:172.21.15.169:3300/0 conn(0x7fca280c58a0 msgr2=0x7fca280c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:56.249 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.248+0000 7fca27fff640 1 --2- 172.21.15.157:0/4261118932 >> v2:172.21.15.169:3300/0 conn(0x7fca280c58a0 0x7fca280c90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:56.249 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.248+0000 7fca27fff640 1 -- 172.21.15.157:0/4261118932 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca281561a0 con 0x7fca280c68d0 2024-04-25T00:00:56.249 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.248+0000 7fca27fff640 1 --2- 172.21.15.157:0/4261118932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca280c68d0 0x7fca280c6cb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fca280be320 tx=0x7fca1c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f4ee6e96cf89dc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:56.249 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.249+0000 7fca26ffd640 1 -- 172.21.15.157:0/4261118932 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca1c037070 con 0x7fca280c68d0 2024-04-25T00:00:56.249 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.249+0000 7fca26ffd640 1 -- 172.21.15.157:0/4261118932 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca1c032440 con 0x7fca280c68d0 2024-04-25T00:00:56.250 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.249+0000 7fca26ffd640 1 -- 172.21.15.157:0/4261118932 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca1c032810 con 0x7fca280c68d0 2024-04-25T00:00:56.252 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.249+0000 7fca2e5e9640 1 -- 172.21.15.157:0/4261118932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca280c68d0 msgr2=0x7fca280c6cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:56.252 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.249+0000 7fca2e5e9640 1 --2- 172.21.15.157:0/4261118932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca280c68d0 0x7fca280c6cb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fca280be320 tx=0x7fca1c02a270 comp rx=0 tx=0).stop 2024-04-25T00:00:56.252 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.249+0000 7fca2e5e9640 1 -- 172.21.15.157:0/4261118932 shutdown_connections 2024-04-25T00:00:56.253 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.249+0000 7fca2e5e9640 1 --2- 172.21.15.157:0/4261118932 >> v2:172.21.15.169:3300/0 conn(0x7fca280c58a0 0x7fca280c90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:56.253 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.249+0000 7fca2e5e9640 1 --2- 172.21.15.157:0/4261118932 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca280c71f0 0x7fca280c8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:56.253 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.249+0000 7fca2e5e9640 1 --2- 172.21.15.157:0/4261118932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca280c68d0 0x7fca280c6cb0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:56.253 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.249+0000 7fca2e5e9640 1 -- 172.21.15.157:0/4261118932 >> 172.21.15.157:0/4261118932 conn(0x7fca280caca0 msgr2=0x7fca280cb0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:56.253 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.249+0000 7fca2e5e9640 1 -- 172.21.15.157:0/4261118932 shutdown_connections 2024-04-25T00:00:56.253 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.249+0000 7fca2e5e9640 1 -- 172.21.15.157:0/4261118932 wait complete. 2024-04-25T00:00:56.253 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.250+0000 7fca2e5e9640 1 Processor -- start 2024-04-25T00:00:56.253 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.251+0000 7fca2e5e9640 1 -- start start 2024-04-25T00:00:56.253 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.251+0000 7fca2e5e9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca280c58a0 0x7fca281592f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:56.253 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.251+0000 7fca2e5e9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca280c68d0 0x7fca2814ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:56.253 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.251+0000 7fca2e5e9640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca280c71f0 0x7fca28154e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:56.253 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.251+0000 7fca2e5e9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fca2806b830 con 0x7fca280c58a0 2024-04-25T00:00:56.253 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.251+0000 7fca2e5e9640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fca28156480 con 0x7fca280c68d0 2024-04-25T00:00:56.253 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.251+0000 7fca2e5e9640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fca28152ed0 con 0x7fca280c71f0 2024-04-25T00:00:56.254 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.251+0000 7fca27fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca280c58a0 0x7fca281592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:56.254 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.251+0000 7fca27fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca280c58a0 0x7fca281592f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37740/0 (socket says 172.21.15.157:37740) 2024-04-25T00:00:56.254 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.252+0000 7fca27fff640 1 -- 172.21.15.157:0/2606094542 learned_addr learned my addr 172.21.15.157:0/2606094542 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:00:56.254 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.252+0000 7fca277fe640 1 --2- 172.21.15.157:0/2606094542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca280c68d0 0x7fca2814ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:56.254 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.252+0000 7fca2cb86640 1 --2- 172.21.15.157:0/2606094542 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca280c71f0 0x7fca28154e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:56.254 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.252+0000 7fca27fff640 1 -- 172.21.15.157:0/2606094542 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca280c71f0 msgr2=0x7fca28154e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:56.254 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.252+0000 7fca27fff640 1 --2- 172.21.15.157:0/2606094542 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca280c71f0 0x7fca28154e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:56.254 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.252+0000 7fca27fff640 1 -- 172.21.15.157:0/2606094542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca280c68d0 msgr2=0x7fca2814ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:56.254 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.252+0000 7fca27fff640 1 --2- 172.21.15.157:0/2606094542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca280c68d0 0x7fca2814ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:56.254 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.252+0000 7fca27fff640 1 -- 172.21.15.157:0/2606094542 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca28158c80 con 0x7fca280c58a0 2024-04-25T00:00:56.254 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.252+0000 7fca27fff640 1 --2- 172.21.15.157:0/2606094542 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca280c58a0 0x7fca281592f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fca1c0097c0 tx=0x7fca1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:56.254 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.252+0000 7fca24ff9640 1 -- 172.21.15.157:0/2606094542 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca1c041020 con 0x7fca280c58a0 2024-04-25T00:00:56.254 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.252+0000 7fca24ff9640 1 -- 172.21.15.157:0/2606094542 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca1c03d620 con 0x7fca280c58a0 2024-04-25T00:00:56.254 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.252+0000 7fca2e5e9640 1 -- 172.21.15.157:0/2606094542 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca2814f3e0 con 0x7fca280c58a0 2024-04-25T00:00:56.255 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.252+0000 7fca24ff9640 1 -- 172.21.15.157:0/2606094542 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca1c037040 con 0x7fca280c58a0 2024-04-25T00:00:56.255 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.252+0000 7fca2e5e9640 1 -- 172.21.15.157:0/2606094542 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca281eb240 con 0x7fca280c58a0 2024-04-25T00:00:56.256 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.253+0000 7fca2e5e9640 1 -- 172.21.15.157:0/2606094542 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca280681c0 con 0x7fca280c58a0 2024-04-25T00:00:56.256 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.254+0000 7fca24ff9640 1 -- 172.21.15.157:0/2606094542 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fca1c044020 con 0x7fca280c58a0 2024-04-25T00:00:56.256 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.254+0000 7fca24ff9640 1 --2- 172.21.15.157:0/2606094542 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fca1406be30 0x7fca1406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:56.256 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.255+0000 7fca24ff9640 1 -- 172.21.15.157:0/2606094542 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fca1c0ab890 con 0x7fca280c58a0 2024-04-25T00:00:56.256 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.255+0000 7fca277fe640 1 --2- 172.21.15.157:0/2606094542 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fca1406be30 0x7fca1406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:56.256 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.255+0000 7fca277fe640 1 --2- 172.21.15.157:0/2606094542 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fca1406be30 0x7fca1406e2f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fca28155860 tx=0x7fca18009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:56.258 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.259+0000 7fca24ff9640 1 -- 172.21.15.157:0/2606094542 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fca1c04b030 con 0x7fca280c58a0 2024-04-25T00:00:56.438 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.439+0000 7fca2e5e9640 1 -- 172.21.15.157:0/2606094542 --> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7fca28155ee0 con 0x7fca1406be30 2024-04-25T00:00:56.439 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.439+0000 7fca24ff9640 1 -- 172.21.15.157:0/2606094542 <== mgr.4105 v2:172.21.15.169:6832/662378289 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fca28155ee0 con 0x7fca1406be30 2024-04-25T00:00:56.439 INFO:teuthology.orchestra.run.smithi157.stderr:instructed osd(s) 5 to deep-scrub 2024-04-25T00:00:56.441 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.441+0000 7fca2e5e9640 1 -- 172.21.15.157:0/2606094542 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fca1406be30 msgr2=0x7fca1406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:56.441 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.441+0000 7fca2e5e9640 1 --2- 172.21.15.157:0/2606094542 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fca1406be30 0x7fca1406e2f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fca28155860 tx=0x7fca18009190 comp rx=0 tx=0).stop 2024-04-25T00:00:56.442 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.441+0000 7fca2e5e9640 1 -- 172.21.15.157:0/2606094542 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca280c58a0 msgr2=0x7fca281592f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:56.442 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.441+0000 7fca2e5e9640 1 --2- 172.21.15.157:0/2606094542 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca280c58a0 0x7fca281592f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fca1c0097c0 tx=0x7fca1c0046c0 comp rx=0 tx=0).stop 2024-04-25T00:00:56.442 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.441+0000 7fca2e5e9640 1 -- 172.21.15.157:0/2606094542 shutdown_connections 2024-04-25T00:00:56.442 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.441+0000 7fca2e5e9640 1 --2- 172.21.15.157:0/2606094542 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fca1406be30 0x7fca1406e2f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:56.442 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.441+0000 7fca2e5e9640 1 --2- 172.21.15.157:0/2606094542 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca280c71f0 0x7fca28154e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:56.442 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.441+0000 7fca2e5e9640 1 --2- 172.21.15.157:0/2606094542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca280c68d0 0x7fca2814ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:56.442 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.441+0000 7fca2e5e9640 1 --2- 172.21.15.157:0/2606094542 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca280c58a0 0x7fca281592f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:56.442 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.441+0000 7fca2e5e9640 1 -- 172.21.15.157:0/2606094542 >> 172.21.15.157:0/2606094542 conn(0x7fca280caca0 msgr2=0x7fca28150640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:56.442 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.442+0000 7fca2e5e9640 1 -- 172.21.15.157:0/2606094542 shutdown_connections 2024-04-25T00:00:56.442 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.442+0000 7fca2e5e9640 1 -- 172.21.15.157:0/2606094542 wait complete. 2024-04-25T00:00:56.452 INFO:tasks.ceph:Scrubbing osd.6 2024-04-25T00:00:56.452 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-25T00:00:56.554 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.554+0000 7fbdeea1c640 1 Processor -- start 2024-04-25T00:00:56.554 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.554+0000 7fbdeea1c640 1 -- start start 2024-04-25T00:00:56.555 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.555+0000 7fbdeea1c640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbde814f3a0 0x7fbde806d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:56.555 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.555+0000 7fbdeea1c640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fbde80b0870 0x7fbde80c7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:56.556 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.555+0000 7fbdeea1c640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbde80c75f0 con 0x7fbde80afd00 2024-04-25T00:00:56.556 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.555+0000 7fbdeea1c640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fbde8155ef0 con 0x7fbde80b0870 2024-04-25T00:00:56.556 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.555+0000 7fbdeea1c640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbde81561d0 con 0x7fbde814f3a0 2024-04-25T00:00:56.556 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.555+0000 7fbde7fff640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbde814f3a0 0x7fbde806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:56.556 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.555+0000 7fbdecfb9640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fbde80b0870 0x7fbde80c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:56.556 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.555+0000 7fbdecfb9640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fbde80b0870 0x7fbde80c7860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.157:36268/0 (socket says 172.21.15.157:36268) 2024-04-25T00:00:56.556 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.555+0000 7fbde7fff640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbde814f3a0 0x7fbde806d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.157:39888/0 (socket says 172.21.15.157:39888) 2024-04-25T00:00:56.556 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.555+0000 7fbde77fe640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fbde80afd00 0x7fbde80b0160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.157:39574/0 (socket says 172.21.15.157:39574) 2024-04-25T00:00:56.556 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.555+0000 7fbdecfb9640 1 -- 172.21.15.157:0/2554527749 learned_addr learned my addr 172.21.15.157:0/2554527749 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:00:56.556 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.555+0000 7fbdecfb9640 1 -- 172.21.15.157:0/2554527749 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbde814f3a0 msgr2=0x7fbde806d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:56.556 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.555+0000 7fbdecfb9640 1 --2- 172.21.15.157:0/2554527749 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbde814f3a0 0x7fbde806d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:56.556 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.555+0000 7fbdecfb9640 1 -- 172.21.15.157:0/2554527749 >> v1:172.21.15.169:6789/0 conn(0x7fbde80afd00 legacy=0x7fbde80b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:56.556 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.555+0000 7fbde6ffd640 1 -- 172.21.15.157:0/2554527749 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1677841015 0 0) 0x7fbde80c75f0 con 0x7fbde80afd00 2024-04-25T00:00:56.556 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.555+0000 7fbdecfb9640 1 -- 172.21.15.157:0/2554527749 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbde80c57d0 con 0x7fbde80b0870 2024-04-25T00:00:56.556 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.555+0000 7fbde7fff640 1 --2- 172.21.15.157:0/2554527749 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbde814f3a0 0x7fbde806d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:00:56.558 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.556+0000 7fbdecfb9640 1 --2- 172.21.15.157:0/2554527749 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbde80b0870 0x7fbde80c7860 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbde0009e50 tx=0x7fbde002a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a88b54900490e8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:56.558 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.557+0000 7fbde6ffd640 1 -- 172.21.15.157:0/2554527749 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde0036070 con 0x7fbde80b0870 2024-04-25T00:00:56.558 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.557+0000 7fbde6ffd640 1 -- 172.21.15.157:0/2554527749 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbde0031600 con 0x7fbde80b0870 2024-04-25T00:00:56.558 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.557+0000 7fbde6ffd640 1 -- 172.21.15.157:0/2554527749 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde00319d0 con 0x7fbde80b0870 2024-04-25T00:00:56.558 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.557+0000 7fbdeea1c640 1 -- 172.21.15.157:0/2554527749 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbde80b0870 msgr2=0x7fbde80c7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:56.558 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.557+0000 7fbdeea1c640 1 --2- 172.21.15.157:0/2554527749 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbde80b0870 0x7fbde80c7860 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbde0009e50 tx=0x7fbde002a710 comp rx=0 tx=0).stop 2024-04-25T00:00:56.558 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.557+0000 7fbdeea1c640 1 -- 172.21.15.157:0/2554527749 shutdown_connections 2024-04-25T00:00:56.558 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.557+0000 7fbdeea1c640 1 --2- 172.21.15.157:0/2554527749 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbde80b0870 0x7fbde80c7860 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:56.559 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.557+0000 7fbdeea1c640 1 --2- 172.21.15.157:0/2554527749 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbde814f3a0 0x7fbde806d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:56.559 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.557+0000 7fbdeea1c640 1 -- 172.21.15.157:0/2554527749 >> 172.21.15.157:0/2554527749 conn(0x7fbde8066ea0 msgr2=0x7fbde80672b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:56.559 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.557+0000 7fbdeea1c640 1 -- 172.21.15.157:0/2554527749 shutdown_connections 2024-04-25T00:00:56.559 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.557+0000 7fbdeea1c640 1 -- 172.21.15.157:0/2554527749 wait complete. 2024-04-25T00:00:56.559 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.558+0000 7fbdeea1c640 1 Processor -- start 2024-04-25T00:00:56.559 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.558+0000 7fbdeea1c640 1 -- start start 2024-04-25T00:00:56.561 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.559+0000 7fbdeea1c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbde80afd00 0x7fbde8158d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:56.561 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.559+0000 7fbdeea1c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbde814f3a0 0x7fbde80aa7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:56.561 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.559+0000 7fbdeea1c640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbde80ace80 0x7fbde81ee4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:56.561 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.559+0000 7fbdeea1c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbde814f780 con 0x7fbde80afd00 2024-04-25T00:00:56.561 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.559+0000 7fbdeea1c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbde80c5d90 con 0x7fbde814f3a0 2024-04-25T00:00:56.561 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.559+0000 7fbdeea1c640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbde80c7da0 con 0x7fbde80ace80 2024-04-25T00:00:56.561 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.559+0000 7fbde7fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbde80afd00 0x7fbde8158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:56.561 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.559+0000 7fbde7fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbde80afd00 0x7fbde8158d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37754/0 (socket says 172.21.15.157:37754) 2024-04-25T00:00:56.561 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.559+0000 7fbde7fff640 1 -- 172.21.15.157:0/2424842999 learned_addr learned my addr 172.21.15.157:0/2424842999 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:00:56.561 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.559+0000 7fbde77fe640 1 --2- 172.21.15.157:0/2424842999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbde814f3a0 0x7fbde80aa7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:56.562 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.559+0000 7fbdecfb9640 1 --2- 172.21.15.157:0/2424842999 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbde80ace80 0x7fbde81ee4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:56.562 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.559+0000 7fbde7fff640 1 -- 172.21.15.157:0/2424842999 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbde80ace80 msgr2=0x7fbde81ee4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:56.562 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.559+0000 7fbde7fff640 1 --2- 172.21.15.157:0/2424842999 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbde80ace80 0x7fbde81ee4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:56.562 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.559+0000 7fbde7fff640 1 -- 172.21.15.157:0/2424842999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbde814f3a0 msgr2=0x7fbde80aa7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:56.562 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.559+0000 7fbde7fff640 1 --2- 172.21.15.157:0/2424842999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbde814f3a0 0x7fbde80aa7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:56.562 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.560+0000 7fbde7fff640 1 -- 172.21.15.157:0/2424842999 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbde81525f0 con 0x7fbde80afd00 2024-04-25T00:00:56.562 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.560+0000 7fbde77fe640 1 --2- 172.21.15.157:0/2424842999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbde814f3a0 0x7fbde80aa7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:00:56.562 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.560+0000 7fbde7fff640 1 --2- 172.21.15.157:0/2424842999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbde80afd00 0x7fbde8158d90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbdd000e990 tx=0x7fbdd000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:56.562 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.560+0000 7fbde4ff9640 1 -- 172.21.15.157:0/2424842999 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd0017070 con 0x7fbde80afd00 2024-04-25T00:00:56.562 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.560+0000 7fbde4ff9640 1 -- 172.21.15.157:0/2424842999 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdd000c9e0 con 0x7fbde80afd00 2024-04-25T00:00:56.562 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.560+0000 7fbdeea1c640 1 -- 172.21.15.157:0/2424842999 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbde81eb120 con 0x7fbde80afd00 2024-04-25T00:00:56.562 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.560+0000 7fbde4ff9640 1 -- 172.21.15.157:0/2424842999 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd000cdb0 con 0x7fbde80afd00 2024-04-25T00:00:56.562 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.560+0000 7fbdeea1c640 1 -- 172.21.15.157:0/2424842999 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbde81eb6e0 con 0x7fbde80afd00 2024-04-25T00:00:56.562 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.561+0000 7fbdeea1c640 1 -- 172.21.15.157:0/2424842999 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbde81528d0 con 0x7fbde80afd00 2024-04-25T00:00:56.563 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.561+0000 7fbde4ff9640 1 -- 172.21.15.157:0/2424842999 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbdd001a070 con 0x7fbde80afd00 2024-04-25T00:00:56.563 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.562+0000 7fbde4ff9640 1 --2- 172.21.15.157:0/2424842999 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fbdc806be60 0x7fbdc806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:56.564 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.562+0000 7fbde4ff9640 1 -- 172.21.15.157:0/2424842999 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fbdd008b710 con 0x7fbde80afd00 2024-04-25T00:00:56.564 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.562+0000 7fbde77fe640 1 --2- 172.21.15.157:0/2424842999 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fbdc806be60 0x7fbdc806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:56.564 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.562+0000 7fbde4ff9640 1 --2- 172.21.15.157:0/2424842999 >> [v2:172.21.15.169:6808/3441420785,v1:172.21.15.169:6809/3441420785] conn(0x7fbdc8075880 0x7fbdc8077d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:56.564 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.562+0000 7fbde4ff9640 1 -- 172.21.15.157:0/2424842999 --> [v2:172.21.15.169:6808/3441420785,v1:172.21.15.169:6809/3441420785] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbdd0027ea0 con 0x7fbdc8075880 2024-04-25T00:00:56.564 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.562+0000 7fbde4ff9640 1 -- 172.21.15.157:0/2424842999 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbdd008bba0 con 0x7fbde80afd00 2024-04-25T00:00:56.564 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.562+0000 7fbdecfb9640 1 --2- 172.21.15.157:0/2424842999 >> [v2:172.21.15.169:6808/3441420785,v1:172.21.15.169:6809/3441420785] conn(0x7fbdc8075880 0x7fbdc8077d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:56.564 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.562+0000 7fbde77fe640 1 --2- 172.21.15.157:0/2424842999 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fbdc806be60 0x7fbdc806e320 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fbde80c7760 tx=0x7fbdd8007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:56.564 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.563+0000 7fbdecfb9640 1 --2- 172.21.15.157:0/2424842999 >> [v2:172.21.15.169:6808/3441420785,v1:172.21.15.169:6809/3441420785] conn(0x7fbdc8075880 0x7fbdc8077d20 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:56.564 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.563+0000 7fbde4ff9640 1 -- 172.21.15.157:0/2424842999 <== osd.6 v2:172.21.15.169:6808/3441420785 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbdd0027ea0 con 0x7fbdc8075880 2024-04-25T00:00:56.576 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.576+0000 7fbdeea1c640 1 -- 172.21.15.157:0/2424842999 --> [v2:172.21.15.169:6808/3441420785,v1:172.21.15.169:6809/3441420785] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fbde80681c0 con 0x7fbdc8075880 2024-04-25T00:00:56.576 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.576+0000 7fbde4ff9640 1 -- 172.21.15.157:0/2424842999 <== osd.6 v2:172.21.15.169:6808/3441420785 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fbde80681c0 con 0x7fbdc8075880 2024-04-25T00:00:56.576 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-04-25T00:00:56.576 INFO:teuthology.orchestra.run.smithi157.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T00:00:56.576 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-04-25T00:00:56.577 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.577+0000 7fbdeea1c640 1 -- 172.21.15.157:0/2424842999 >> [v2:172.21.15.169:6808/3441420785,v1:172.21.15.169:6809/3441420785] conn(0x7fbdc8075880 msgr2=0x7fbdc8077d20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:56.577 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.577+0000 7fbdeea1c640 1 --2- 172.21.15.157:0/2424842999 >> [v2:172.21.15.169:6808/3441420785,v1:172.21.15.169:6809/3441420785] conn(0x7fbdc8075880 0x7fbdc8077d20 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:56.577 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.577+0000 7fbdeea1c640 1 -- 172.21.15.157:0/2424842999 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fbdc806be60 msgr2=0x7fbdc806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:56.577 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.577+0000 7fbdeea1c640 1 --2- 172.21.15.157:0/2424842999 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fbdc806be60 0x7fbdc806e320 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fbde80c7760 tx=0x7fbdd8007d50 comp rx=0 tx=0).stop 2024-04-25T00:00:56.577 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.577+0000 7fbdeea1c640 1 -- 172.21.15.157:0/2424842999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbde80afd00 msgr2=0x7fbde8158d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:56.577 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.577+0000 7fbdeea1c640 1 --2- 172.21.15.157:0/2424842999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbde80afd00 0x7fbde8158d90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbdd000e990 tx=0x7fbdd000ee50 comp rx=0 tx=0).stop 2024-04-25T00:00:56.578 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.577+0000 7fbdecfb9640 1 -- 172.21.15.157:0/2424842999 reap_dead start 2024-04-25T00:00:56.578 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.577+0000 7fbdeea1c640 1 -- 172.21.15.157:0/2424842999 shutdown_connections 2024-04-25T00:00:56.578 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.577+0000 7fbdeea1c640 1 -- 172.21.15.157:0/2424842999 >> 172.21.15.157:0/2424842999 conn(0x7fbde8066ea0 msgr2=0x7fbde80b0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:56.578 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.577+0000 7fbdeea1c640 1 -- 172.21.15.157:0/2424842999 shutdown_connections 2024-04-25T00:00:56.578 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.577+0000 7fbdeea1c640 1 -- 172.21.15.157:0/2424842999 wait complete. 2024-04-25T00:00:56.587 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-25T00:00:56.687 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.687+0000 7fccdfc88640 1 Processor -- start 2024-04-25T00:00:56.687 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.688+0000 7fccdfc88640 1 -- start start 2024-04-25T00:00:56.688 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.688+0000 7fccdfc88640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fccd80c58a0 0x7fccd80c8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:56.689 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.688+0000 7fccdfc88640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fccd80c90e0 0x7fccd815fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:56.689 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.688+0000 7fccdfc88640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccd806bd80 con 0x7fccd80c68d0 2024-04-25T00:00:56.689 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.688+0000 7fccdfc88640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fccd80c6730 con 0x7fccd80c90e0 2024-04-25T00:00:56.689 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.688+0000 7fccdfc88640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fccd81561a0 con 0x7fccd80c58a0 2024-04-25T00:00:56.689 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.688+0000 7fccdd223640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fccd80c58a0 0x7fccd80c8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:56.689 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.689+0000 7fccdda24640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fccd80c68d0 0x7fccd80c6cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.157:39580/0 (socket says 172.21.15.157:39580) 2024-04-25T00:00:56.689 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.689+0000 7fccdda24640 1 -- 172.21.15.157:0/1372878344 learned_addr learned my addr 172.21.15.157:0/1372878344 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-25T00:00:56.689 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.689+0000 7fccde225640 1 --2- 172.21.15.157:0/1372878344 >> v2:172.21.15.169:3300/0 conn(0x7fccd80c90e0 0x7fccd815fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:56.689 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.689+0000 7fccdca22640 1 -- 172.21.15.157:0/1372878344 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3631527143 0 0) 0x7fccd806bd80 con 0x7fccd80c68d0 2024-04-25T00:00:56.689 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.689+0000 7fccdca22640 1 -- 172.21.15.157:0/1372878344 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fccc4003520 con 0x7fccd80c68d0 2024-04-25T00:00:56.689 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.689+0000 7fccdd223640 1 -- 172.21.15.157:0/1372878344 >> v2:172.21.15.169:3300/0 conn(0x7fccd80c90e0 msgr2=0x7fccd815fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:56.689 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.689+0000 7fccdd223640 1 --2- 172.21.15.157:0/1372878344 >> v2:172.21.15.169:3300/0 conn(0x7fccd80c90e0 0x7fccd815fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:56.689 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.689+0000 7fccdd223640 1 -- 172.21.15.157:0/1372878344 >> v1:172.21.15.169:6789/0 conn(0x7fccd80c68d0 legacy=0x7fccd80c6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:56.689 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.689+0000 7fccdd223640 1 -- 172.21.15.157:0/1372878344 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccd80c6370 con 0x7fccd80c58a0 2024-04-25T00:00:56.690 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.689+0000 7fccde225640 1 --2- 172.21.15.157:0/1372878344 >> v2:172.21.15.169:3300/0 conn(0x7fccd80c90e0 0x7fccd815fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:00:56.691 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.690+0000 7fccdd223640 1 --2- 172.21.15.157:0/1372878344 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fccd80c58a0 0x7fccd80c8ba0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fccc80097a0 tx=0x7fccc802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=830642bb93f78ea4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:56.691 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.690+0000 7fccdca22640 1 -- 172.21.15.157:0/1372878344 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc8037070 con 0x7fccd80c58a0 2024-04-25T00:00:56.691 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.690+0000 7fccdca22640 1 -- 172.21.15.157:0/1372878344 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccc8032440 con 0x7fccd80c58a0 2024-04-25T00:00:56.691 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.690+0000 7fccdca22640 1 -- 172.21.15.157:0/1372878344 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc8032810 con 0x7fccd80c58a0 2024-04-25T00:00:56.691 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.691+0000 7fccdfc88640 1 -- 172.21.15.157:0/1372878344 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fccd80c58a0 msgr2=0x7fccd80c8ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:56.691 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.691+0000 7fccdfc88640 1 --2- 172.21.15.157:0/1372878344 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fccd80c58a0 0x7fccd80c8ba0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fccc80097a0 tx=0x7fccc802a180 comp rx=0 tx=0).stop 2024-04-25T00:00:56.691 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.691+0000 7fccdfc88640 1 -- 172.21.15.157:0/1372878344 shutdown_connections 2024-04-25T00:00:56.691 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.691+0000 7fccdfc88640 1 --2- 172.21.15.157:0/1372878344 >> v2:172.21.15.169:3300/0 conn(0x7fccd80c90e0 0x7fccd815fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:56.691 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.691+0000 7fccdfc88640 1 --2- 172.21.15.157:0/1372878344 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fccd80c58a0 0x7fccd80c8ba0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:56.691 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.691+0000 7fccdfc88640 1 -- 172.21.15.157:0/1372878344 >> 172.21.15.157:0/1372878344 conn(0x7fccd80caca0 msgr2=0x7fccd80cb0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:56.691 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.691+0000 7fccdfc88640 1 -- 172.21.15.157:0/1372878344 shutdown_connections 2024-04-25T00:00:56.692 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.691+0000 7fccdfc88640 1 -- 172.21.15.157:0/1372878344 wait complete. 2024-04-25T00:00:56.692 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.693+0000 7fccdfc88640 1 Processor -- start 2024-04-25T00:00:56.692 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.693+0000 7fccdfc88640 1 -- start start 2024-04-25T00:00:56.694 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.694+0000 7fccdfc88640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fccd80c58a0 0x7fccd814f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:56.695 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.694+0000 7fccdfc88640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccd80c68d0 0x7fccd80afbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:56.695 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.694+0000 7fccdfc88640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccd80c90e0 0x7fccd814f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:56.695 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.694+0000 7fccdfc88640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fccd80c9570 con 0x7fccd80c90e0 2024-04-25T00:00:56.695 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.694+0000 7fccdfc88640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fccd8156480 con 0x7fccd80c68d0 2024-04-25T00:00:56.695 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.694+0000 7fccdfc88640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fccd806b830 con 0x7fccd80c58a0 2024-04-25T00:00:56.695 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.694+0000 7fccdda24640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fccd80c58a0 0x7fccd814f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:56.695 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.694+0000 7fccde225640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccd80c90e0 0x7fccd814f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:56.696 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.694+0000 7fccdda24640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fccd80c58a0 0x7fccd814f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.157:39906/0 (socket says 172.21.15.157:39906) 2024-04-25T00:00:56.696 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.694+0000 7fccdda24640 1 -- 172.21.15.157:0/4169324513 learned_addr learned my addr 172.21.15.157:0/4169324513 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:00:56.696 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.694+0000 7fccdd223640 1 --2- 172.21.15.157:0/4169324513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccd80c68d0 0x7fccd80afbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:56.696 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.694+0000 7fccdda24640 1 -- 172.21.15.157:0/4169324513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccd80c68d0 msgr2=0x7fccd80afbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:56.696 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.694+0000 7fccdda24640 1 --2- 172.21.15.157:0/4169324513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccd80c68d0 0x7fccd80afbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:56.696 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.694+0000 7fccdda24640 1 -- 172.21.15.157:0/4169324513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccd80c90e0 msgr2=0x7fccd814f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:56.696 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.694+0000 7fccdda24640 1 --2- 172.21.15.157:0/4169324513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccd80c90e0 0x7fccd814f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:56.696 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.694+0000 7fccdda24640 1 -- 172.21.15.157:0/4169324513 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccd814e360 con 0x7fccd80c58a0 2024-04-25T00:00:56.696 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.694+0000 7fccde225640 1 --2- 172.21.15.157:0/4169324513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccd80c90e0 0x7fccd814f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:00:56.696 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.694+0000 7fccdd223640 1 --2- 172.21.15.157:0/4169324513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccd80c68d0 0x7fccd80afbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:00:56.696 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.694+0000 7fccdda24640 1 --2- 172.21.15.157:0/4169324513 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fccd80c58a0 0x7fccd814f100 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fccd400c390 tx=0x7fccd400c790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:56.696 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.695+0000 7fccce7fc640 1 -- 172.21.15.157:0/4169324513 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd4017070 con 0x7fccd80c58a0 2024-04-25T00:00:56.697 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.695+0000 7fccce7fc640 1 -- 172.21.15.157:0/4169324513 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccd4005ce0 con 0x7fccd80c58a0 2024-04-25T00:00:56.697 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.695+0000 7fccce7fc640 1 -- 172.21.15.157:0/4169324513 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd40072f0 con 0x7fccd80c58a0 2024-04-25T00:00:56.697 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.695+0000 7fccdfc88640 1 -- 172.21.15.157:0/4169324513 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccd814e640 con 0x7fccd80c58a0 2024-04-25T00:00:56.697 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.695+0000 7fccdfc88640 1 -- 172.21.15.157:0/4169324513 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fccd814ec00 con 0x7fccd80c58a0 2024-04-25T00:00:56.698 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.696+0000 7fccdfc88640 1 -- 172.21.15.157:0/4169324513 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccd80b01e0 con 0x7fccd80c58a0 2024-04-25T00:00:56.698 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.696+0000 7fccce7fc640 1 -- 172.21.15.157:0/4169324513 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fccd401a070 con 0x7fccd80c58a0 2024-04-25T00:00:56.698 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.697+0000 7fccce7fc640 1 --2- 172.21.15.157:0/4169324513 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fccbc06be60 0x7fccbc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:56.698 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.697+0000 7fccdd223640 1 --2- 172.21.15.157:0/4169324513 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fccbc06be60 0x7fccbc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:56.698 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.697+0000 7fccce7fc640 1 -- 172.21.15.157:0/4169324513 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fccd408a560 con 0x7fccd80c58a0 2024-04-25T00:00:56.698 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.697+0000 7fccdd223640 1 --2- 172.21.15.157:0/4169324513 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fccbc06be60 0x7fccbc06e320 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fccc8034000 tx=0x7fccc802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:56.702 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.703+0000 7fccce7fc640 1 -- 172.21.15.157:0/4169324513 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fccd4058420 con 0x7fccd80c58a0 2024-04-25T00:00:56.882 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.882+0000 7fccdfc88640 1 -- 172.21.15.157:0/4169324513 --> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7fccd80681c0 con 0x7fccbc06be60 2024-04-25T00:00:56.883 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.883+0000 7fccce7fc640 1 -- 172.21.15.157:0/4169324513 <== mgr.4105 v2:172.21.15.169:6832/662378289 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fccd80681c0 con 0x7fccbc06be60 2024-04-25T00:00:56.883 INFO:teuthology.orchestra.run.smithi157.stderr:instructed osd(s) 6 to deep-scrub 2024-04-25T00:00:56.885 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.885+0000 7fccdfc88640 1 -- 172.21.15.157:0/4169324513 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fccbc06be60 msgr2=0x7fccbc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:56.885 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.885+0000 7fccdfc88640 1 --2- 172.21.15.157:0/4169324513 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fccbc06be60 0x7fccbc06e320 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fccc8034000 tx=0x7fccc802e000 comp rx=0 tx=0).stop 2024-04-25T00:00:56.885 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.885+0000 7fccdfc88640 1 -- 172.21.15.157:0/4169324513 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fccd80c58a0 msgr2=0x7fccd814f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:56.886 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.885+0000 7fccdfc88640 1 --2- 172.21.15.157:0/4169324513 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fccd80c58a0 0x7fccd814f100 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fccd400c390 tx=0x7fccd400c790 comp rx=0 tx=0).stop 2024-04-25T00:00:56.886 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.885+0000 7fccdfc88640 1 -- 172.21.15.157:0/4169324513 shutdown_connections 2024-04-25T00:00:56.886 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.885+0000 7fccdfc88640 1 --2- 172.21.15.157:0/4169324513 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fccbc06be60 0x7fccbc06e320 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:56.886 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.885+0000 7fccdfc88640 1 --2- 172.21.15.157:0/4169324513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccd80c90e0 0x7fccd814f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:56.886 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.885+0000 7fccdfc88640 1 --2- 172.21.15.157:0/4169324513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccd80c68d0 0x7fccd80afbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:56.886 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.885+0000 7fccdfc88640 1 --2- 172.21.15.157:0/4169324513 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fccd80c58a0 0x7fccd814f100 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:56.886 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.885+0000 7fccdfc88640 1 -- 172.21.15.157:0/4169324513 >> 172.21.15.157:0/4169324513 conn(0x7fccd80caca0 msgr2=0x7fccd8155420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:56.886 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.885+0000 7fccdfc88640 1 -- 172.21.15.157:0/4169324513 shutdown_connections 2024-04-25T00:00:56.886 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:56.885+0000 7fccdfc88640 1 -- 172.21.15.157:0/4169324513 wait complete. 2024-04-25T00:00:56.895 INFO:tasks.ceph:Scrubbing osd.7 2024-04-25T00:00:56.895 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-25T00:00:57.000 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.000+0000 7fd59b8f5640 1 Processor -- start 2024-04-25T00:00:57.000 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.001+0000 7fd59b8f5640 1 -- start start 2024-04-25T00:00:57.001 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.001+0000 7fd59b8f5640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd59406db60 0x7fd594072cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:57.002 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.001+0000 7fd59b8f5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd5940c7940 0x7fd59406df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:57.002 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.001+0000 7fd59b8f5640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd594155420 0x7fd594155890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:57.002 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.002+0000 7fd59b8f5640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fd59406eb70 con 0x7fd5940c7940 2024-04-25T00:00:57.002 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.002+0000 7fd59b8f5640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fd594158ab0 con 0x7fd59406db60 2024-04-25T00:00:57.002 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.002+0000 7fd59b8f5640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd594158d90 con 0x7fd594155420 2024-04-25T00:00:57.002 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.002+0000 7fd599691640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd59406db60 0x7fd594072cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:57.002 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.002+0000 7fd599e92640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd594155420 0x7fd594155890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:57.002 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.002+0000 7fd599691640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd59406db60 0x7fd594072cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.157:36314/0 (socket says 172.21.15.157:36314) 2024-04-25T00:00:57.002 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.002+0000 7fd599691640 1 -- 172.21.15.157:0/4183260081 learned_addr learned my addr 172.21.15.157:0/4183260081 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:00:57.002 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.002+0000 7fd598e90640 1 --2- 172.21.15.157:0/4183260081 >> v2:172.21.15.157:3300/0 conn(0x7fd5940c7940 0x7fd59406df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:57.002 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.002+0000 7fd599e92640 1 -- 172.21.15.157:0/4183260081 >> v2:172.21.15.169:3300/0 conn(0x7fd59406db60 msgr2=0x7fd594072cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:57.003 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.002+0000 7fd599e92640 1 --2- 172.21.15.157:0/4183260081 >> v2:172.21.15.169:3300/0 conn(0x7fd59406db60 0x7fd594072cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:57.003 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.002+0000 7fd599e92640 1 -- 172.21.15.157:0/4183260081 >> v2:172.21.15.157:3300/0 conn(0x7fd5940c7940 msgr2=0x7fd59406df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:57.003 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.002+0000 7fd599e92640 1 --2- 172.21.15.157:0/4183260081 >> v2:172.21.15.157:3300/0 conn(0x7fd5940c7940 0x7fd59406df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:57.003 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.002+0000 7fd599e92640 1 -- 172.21.15.157:0/4183260081 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd594159070 con 0x7fd594155420 2024-04-25T00:00:57.003 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.002+0000 7fd599691640 1 --2- 172.21.15.157:0/4183260081 >> v2:172.21.15.169:3300/0 conn(0x7fd59406db60 0x7fd594072cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:00:57.004 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.003+0000 7fd599e92640 1 --2- 172.21.15.157:0/4183260081 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd594155420 0x7fd594155890 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd594070630 tx=0x7fd58c02a540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26a3b029ee771513 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:57.004 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.003+0000 7fd58bfff640 1 -- 172.21.15.157:0/4183260081 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58c007740 con 0x7fd594155420 2024-04-25T00:00:57.004 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.003+0000 7fd58bfff640 1 -- 172.21.15.157:0/4183260081 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd58c0078e0 con 0x7fd594155420 2024-04-25T00:00:57.004 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.003+0000 7fd58bfff640 1 -- 172.21.15.157:0/4183260081 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58c007cb0 con 0x7fd594155420 2024-04-25T00:00:57.004 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.004+0000 7fd59b8f5640 1 -- 172.21.15.157:0/4183260081 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd594155420 msgr2=0x7fd594155890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:57.005 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.004+0000 7fd59b8f5640 1 --2- 172.21.15.157:0/4183260081 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd594155420 0x7fd594155890 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd594070630 tx=0x7fd58c02a540 comp rx=0 tx=0).stop 2024-04-25T00:00:57.005 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.004+0000 7fd59b8f5640 1 -- 172.21.15.157:0/4183260081 shutdown_connections 2024-04-25T00:00:57.005 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.004+0000 7fd59b8f5640 1 --2- 172.21.15.157:0/4183260081 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd594155420 0x7fd594155890 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:57.005 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.004+0000 7fd59b8f5640 1 --2- 172.21.15.157:0/4183260081 >> v2:172.21.15.157:3300/0 conn(0x7fd5940c7940 0x7fd59406df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:57.005 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.004+0000 7fd59b8f5640 1 --2- 172.21.15.157:0/4183260081 >> v2:172.21.15.169:3300/0 conn(0x7fd59406db60 0x7fd594072cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:57.005 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.004+0000 7fd59b8f5640 1 -- 172.21.15.157:0/4183260081 >> 172.21.15.157:0/4183260081 conn(0x7fd5940c6d90 msgr2=0x7fd5940c71a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:57.005 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.005+0000 7fd59b8f5640 1 -- 172.21.15.157:0/4183260081 shutdown_connections 2024-04-25T00:00:57.005 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.005+0000 7fd59b8f5640 1 -- 172.21.15.157:0/4183260081 wait complete. 2024-04-25T00:00:57.006 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.007+0000 7fd59b8f5640 1 Processor -- start 2024-04-25T00:00:57.006 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.007+0000 7fd59b8f5640 1 -- start start 2024-04-25T00:00:57.009 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.008+0000 7fd59b8f5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd59406db60 0x7fd594141b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:57.010 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.008+0000 7fd59b8f5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5940c7940 0x7fd59413c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:57.010 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.008+0000 7fd59b8f5640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd594155420 0x7fd59413c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:57.010 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.008+0000 7fd59b8f5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd594159870 con 0x7fd59406db60 2024-04-25T00:00:57.010 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.008+0000 7fd59b8f5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd594155dd0 con 0x7fd5940c7940 2024-04-25T00:00:57.010 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.008+0000 7fd59b8f5640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd594152d20 con 0x7fd594155420 2024-04-25T00:00:57.010 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.008+0000 7fd599691640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd59406db60 0x7fd594141b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:57.010 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.008+0000 7fd599691640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd59406db60 0x7fd594141b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37772/0 (socket says 172.21.15.157:37772) 2024-04-25T00:00:57.010 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.008+0000 7fd599691640 1 -- 172.21.15.157:0/2807200306 learned_addr learned my addr 172.21.15.157:0/2807200306 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:00:57.010 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.008+0000 7fd598e90640 1 --2- 172.21.15.157:0/2807200306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5940c7940 0x7fd59413c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:57.011 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.009+0000 7fd599691640 1 -- 172.21.15.157:0/2807200306 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd594155420 msgr2=0x7fd59413c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:57.011 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.009+0000 7fd599691640 1 --2- 172.21.15.157:0/2807200306 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd594155420 0x7fd59413c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:57.011 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.009+0000 7fd599691640 1 -- 172.21.15.157:0/2807200306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5940c7940 msgr2=0x7fd59413c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:57.011 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.009+0000 7fd599691640 1 --2- 172.21.15.157:0/2807200306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5940c7940 0x7fd59413c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:57.011 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.009+0000 7fd599691640 1 -- 172.21.15.157:0/2807200306 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5941ee890 con 0x7fd59406db60 2024-04-25T00:00:57.011 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.009+0000 7fd598e90640 1 --2- 172.21.15.157:0/2807200306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5940c7940 0x7fd59413c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:00:57.011 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.009+0000 7fd599691640 1 --2- 172.21.15.157:0/2807200306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd59406db60 0x7fd594141b30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd590006ca0 tx=0x7fd59000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:57.011 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.009+0000 7fd589ffb640 1 -- 172.21.15.157:0/2807200306 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd590004030 con 0x7fd59406db60 2024-04-25T00:00:57.011 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.009+0000 7fd589ffb640 1 -- 172.21.15.157:0/2807200306 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5900041d0 con 0x7fd59406db60 2024-04-25T00:00:57.011 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.009+0000 7fd59b8f5640 1 -- 172.21.15.157:0/2807200306 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5941eeb70 con 0x7fd59406db60 2024-04-25T00:00:57.011 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.009+0000 7fd589ffb640 1 -- 172.21.15.157:0/2807200306 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59000c940 con 0x7fd59406db60 2024-04-25T00:00:57.011 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.009+0000 7fd59b8f5640 1 -- 172.21.15.157:0/2807200306 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5941ef410 con 0x7fd59406db60 2024-04-25T00:00:57.013 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.011+0000 7fd59b8f5640 1 -- 172.21.15.157:0/2807200306 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd5940681c0 con 0x7fd59406db60 2024-04-25T00:00:57.013 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.012+0000 7fd589ffb640 1 -- 172.21.15.157:0/2807200306 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd59000cae0 con 0x7fd59406db60 2024-04-25T00:00:57.013 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.012+0000 7fd589ffb640 1 --2- 172.21.15.157:0/2807200306 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fd57806be30 0x7fd57806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:57.014 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.012+0000 7fd589ffb640 1 -- 172.21.15.157:0/2807200306 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fd59008b530 con 0x7fd59406db60 2024-04-25T00:00:57.014 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.012+0000 7fd589ffb640 1 --2- 172.21.15.157:0/2807200306 >> [v2:172.21.15.169:6816/1663179020,v1:172.21.15.169:6817/1663179020] conn(0x7fd578075840 0x7fd578077ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:57.014 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.012+0000 7fd589ffb640 1 -- 172.21.15.157:0/2807200306 --> [v2:172.21.15.169:6816/1663179020,v1:172.21.15.169:6817/1663179020] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd5780783b0 con 0x7fd578075840 2024-04-25T00:00:57.014 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.012+0000 7fd589ffb640 1 -- 172.21.15.157:0/2807200306 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd590092050 con 0x7fd59406db60 2024-04-25T00:00:57.014 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.013+0000 7fd598e90640 1 --2- 172.21.15.157:0/2807200306 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fd57806be30 0x7fd57806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:57.014 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.013+0000 7fd599e92640 1 --2- 172.21.15.157:0/2807200306 >> [v2:172.21.15.169:6816/1663179020,v1:172.21.15.169:6817/1663179020] conn(0x7fd578075840 0x7fd578077ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:57.014 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.013+0000 7fd598e90640 1 --2- 172.21.15.157:0/2807200306 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fd57806be30 0x7fd57806e2f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd5941ef950 tx=0x7fd584008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:57.014 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.013+0000 7fd599e92640 1 --2- 172.21.15.157:0/2807200306 >> [v2:172.21.15.169:6816/1663179020,v1:172.21.15.169:6817/1663179020] conn(0x7fd578075840 0x7fd578077ce0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:57.015 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.014+0000 7fd589ffb640 1 -- 172.21.15.157:0/2807200306 <== osd.7 v2:172.21.15.169:6816/1663179020 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd5780783b0 con 0x7fd578075840 2024-04-25T00:00:57.030 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.031+0000 7fd59b8f5640 1 -- 172.21.15.157:0/2807200306 --> [v2:172.21.15.169:6816/1663179020,v1:172.21.15.169:6817/1663179020] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fd5940acac0 con 0x7fd578075840 2024-04-25T00:00:57.030 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.031+0000 7fd589ffb640 1 -- 172.21.15.157:0/2807200306 <== osd.7 v2:172.21.15.169:6816/1663179020 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fd5940acac0 con 0x7fd578075840 2024-04-25T00:00:57.031 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-04-25T00:00:57.031 INFO:teuthology.orchestra.run.smithi157.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T00:00:57.031 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-04-25T00:00:57.032 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.031+0000 7fd59b8f5640 1 -- 172.21.15.157:0/2807200306 >> [v2:172.21.15.169:6816/1663179020,v1:172.21.15.169:6817/1663179020] conn(0x7fd578075840 msgr2=0x7fd578077ce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:57.033 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.031+0000 7fd59b8f5640 1 --2- 172.21.15.157:0/2807200306 >> [v2:172.21.15.169:6816/1663179020,v1:172.21.15.169:6817/1663179020] conn(0x7fd578075840 0x7fd578077ce0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:57.033 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.032+0000 7fd59b8f5640 1 -- 172.21.15.157:0/2807200306 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fd57806be30 msgr2=0x7fd57806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:57.033 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.032+0000 7fd59b8f5640 1 --2- 172.21.15.157:0/2807200306 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fd57806be30 0x7fd57806e2f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd5941ef950 tx=0x7fd584008040 comp rx=0 tx=0).stop 2024-04-25T00:00:57.033 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.032+0000 7fd59b8f5640 1 -- 172.21.15.157:0/2807200306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd59406db60 msgr2=0x7fd594141b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:57.033 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.032+0000 7fd59b8f5640 1 --2- 172.21.15.157:0/2807200306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd59406db60 0x7fd594141b30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd590006ca0 tx=0x7fd59000ed60 comp rx=0 tx=0).stop 2024-04-25T00:00:57.033 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.032+0000 7fd599e92640 1 -- 172.21.15.157:0/2807200306 reap_dead start 2024-04-25T00:00:57.033 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.032+0000 7fd59b8f5640 1 -- 172.21.15.157:0/2807200306 shutdown_connections 2024-04-25T00:00:57.033 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.032+0000 7fd59b8f5640 1 -- 172.21.15.157:0/2807200306 >> 172.21.15.157:0/2807200306 conn(0x7fd5940c6d90 msgr2=0x7fd5940c3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:57.033 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.032+0000 7fd59b8f5640 1 -- 172.21.15.157:0/2807200306 shutdown_connections 2024-04-25T00:00:57.033 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.032+0000 7fd59b8f5640 1 -- 172.21.15.157:0/2807200306 wait complete. 2024-04-25T00:00:57.043 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-25T00:00:57.148 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.148+0000 7fb07094f640 1 Processor -- start 2024-04-25T00:00:57.149 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.148+0000 7fb07094f640 1 -- start start 2024-04-25T00:00:57.150 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.149+0000 7fb07094f640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb06c0c58a0 0x7fb06c0c8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:57.150 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.149+0000 7fb07094f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb06c0c90e0 0x7fb06c15fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:57.150 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.149+0000 7fb07094f640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb06c06bd80 con 0x7fb06c0c68d0 2024-04-25T00:00:57.151 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.149+0000 7fb07094f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fb06c0c6730 con 0x7fb06c0c90e0 2024-04-25T00:00:57.151 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.149+0000 7fb07094f640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb06c1561a0 con 0x7fb06c0c58a0 2024-04-25T00:00:57.151 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.149+0000 7fb06ad9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb06c0c90e0 0x7fb06c15fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:57.151 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.149+0000 7fb06ad9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb06c0c90e0 0x7fb06c15fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37782/0 (socket says 172.21.15.157:37782) 2024-04-25T00:00:57.151 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.149+0000 7fb06ad9d640 1 -- 172.21.15.157:0/321066979 learned_addr learned my addr 172.21.15.157:0/321066979 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:00:57.151 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.149+0000 7fb063fff640 1 --2- 172.21.15.157:0/321066979 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb06c0c58a0 0x7fb06c0c8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:57.151 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.149+0000 7fb06ad9d640 1 -- 172.21.15.157:0/321066979 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb06c0c58a0 msgr2=0x7fb06c0c8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:57.151 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.149+0000 7fb06ad9d640 1 --2- 172.21.15.157:0/321066979 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb06c0c58a0 0x7fb06c0c8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:57.151 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.149+0000 7fb06ad9d640 1 -- 172.21.15.157:0/321066979 >> v1:172.21.15.169:6789/0 conn(0x7fb06c0c68d0 legacy=0x7fb06c0c6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:57.151 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.149+0000 7fb06ad9d640 1 -- 172.21.15.157:0/321066979 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb06c0c6370 con 0x7fb06c0c90e0 2024-04-25T00:00:57.151 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.149+0000 7fb069d9b640 1 -- 172.21.15.157:0/321066979 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 45215239 0 0) 0x7fb06c06bd80 con 0x7fb06c0c68d0 2024-04-25T00:00:57.151 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.150+0000 7fb06ad9d640 1 --2- 172.21.15.157:0/321066979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb06c0c90e0 0x7fb06c15fe40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb064009e50 tx=0x7fb06402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b6c87a6f6cc163e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:57.151 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.151+0000 7fb069d9b640 1 -- 172.21.15.157:0/321066979 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb064036070 con 0x7fb06c0c90e0 2024-04-25T00:00:57.151 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.151+0000 7fb069d9b640 1 -- 172.21.15.157:0/321066979 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb064031600 con 0x7fb06c0c90e0 2024-04-25T00:00:57.152 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.151+0000 7fb069d9b640 1 -- 172.21.15.157:0/321066979 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0640319d0 con 0x7fb06c0c90e0 2024-04-25T00:00:57.152 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.151+0000 7fb07094f640 1 -- 172.21.15.157:0/321066979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb06c0c90e0 msgr2=0x7fb06c15fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:57.152 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.151+0000 7fb07094f640 1 --2- 172.21.15.157:0/321066979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb06c0c90e0 0x7fb06c15fe40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb064009e50 tx=0x7fb06402a710 comp rx=0 tx=0).stop 2024-04-25T00:00:57.153 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.151+0000 7fb07094f640 1 -- 172.21.15.157:0/321066979 shutdown_connections 2024-04-25T00:00:57.153 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.151+0000 7fb07094f640 1 --2- 172.21.15.157:0/321066979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb06c0c90e0 0x7fb06c15fe40 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:57.153 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.151+0000 7fb07094f640 1 --2- 172.21.15.157:0/321066979 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb06c0c58a0 0x7fb06c0c8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:57.153 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.151+0000 7fb07094f640 1 -- 172.21.15.157:0/321066979 >> 172.21.15.157:0/321066979 conn(0x7fb06c0caca0 msgr2=0x7fb06c0cb0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:57.153 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.151+0000 7fb07094f640 1 -- 172.21.15.157:0/321066979 shutdown_connections 2024-04-25T00:00:57.153 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.151+0000 7fb07094f640 1 -- 172.21.15.157:0/321066979 wait complete. 2024-04-25T00:00:57.156 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.153+0000 7fb07094f640 1 Processor -- start 2024-04-25T00:00:57.156 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.153+0000 7fb07094f640 1 -- start start 2024-04-25T00:00:57.156 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.153+0000 7fb07094f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb06c0c58a0 0x7fb06c1ec010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:57.157 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.153+0000 7fb07094f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb06c0c68d0 0x7fb06c14f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:57.157 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.153+0000 7fb07094f640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb06c0c90e0 0x7fb06c1ee110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:57.157 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.153+0000 7fb07094f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c0c9570 con 0x7fb06c0c58a0 2024-04-25T00:00:57.157 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.153+0000 7fb07094f640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c156480 con 0x7fb06c0c68d0 2024-04-25T00:00:57.157 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.153+0000 7fb07094f640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb06c06b830 con 0x7fb06c0c90e0 2024-04-25T00:00:57.157 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.153+0000 7fb06a59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb06c0c58a0 0x7fb06c1ec010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:57.157 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.153+0000 7fb06a59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb06c0c58a0 0x7fb06c1ec010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37790/0 (socket says 172.21.15.157:37790) 2024-04-25T00:00:57.157 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.153+0000 7fb06a59c640 1 -- 172.21.15.157:0/1197066973 learned_addr learned my addr 172.21.15.157:0/1197066973 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:00:57.157 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.153+0000 7fb063fff640 1 --2- 172.21.15.157:0/1197066973 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb06c0c68d0 0x7fb06c14f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:57.157 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.153+0000 7fb06ad9d640 1 --2- 172.21.15.157:0/1197066973 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb06c0c90e0 0x7fb06c1ee110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:57.157 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.154+0000 7fb06a59c640 1 -- 172.21.15.157:0/1197066973 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb06c0c90e0 msgr2=0x7fb06c1ee110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:57.157 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.154+0000 7fb06a59c640 1 --2- 172.21.15.157:0/1197066973 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb06c0c90e0 0x7fb06c1ee110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:57.157 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.154+0000 7fb06a59c640 1 -- 172.21.15.157:0/1197066973 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb06c0c68d0 msgr2=0x7fb06c14f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:57.157 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.154+0000 7fb06a59c640 1 --2- 172.21.15.157:0/1197066973 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb06c0c68d0 0x7fb06c14f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:57.158 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.154+0000 7fb06a59c640 1 -- 172.21.15.157:0/1197066973 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb06c14dc20 con 0x7fb06c0c58a0 2024-04-25T00:00:57.158 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.154+0000 7fb06a59c640 1 --2- 172.21.15.157:0/1197066973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb06c0c58a0 0x7fb06c1ec010 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb05c00c440 tx=0x7fb05c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:57.158 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.154+0000 7fb062ffd640 1 -- 172.21.15.157:0/1197066973 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05c017070 con 0x7fb06c0c58a0 2024-04-25T00:00:57.158 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.154+0000 7fb062ffd640 1 -- 172.21.15.157:0/1197066973 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb05c005480 con 0x7fb06c0c58a0 2024-04-25T00:00:57.158 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.154+0000 7fb07094f640 1 -- 172.21.15.157:0/1197066973 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb06c14dec0 con 0x7fb06c0c58a0 2024-04-25T00:00:57.158 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.154+0000 7fb062ffd640 1 -- 172.21.15.157:0/1197066973 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05c003830 con 0x7fb06c0c58a0 2024-04-25T00:00:57.158 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.154+0000 7fb07094f640 1 -- 172.21.15.157:0/1197066973 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb06c151820 con 0x7fb06c0c58a0 2024-04-25T00:00:57.158 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.155+0000 7fb07094f640 1 -- 172.21.15.157:0/1197066973 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb06c0681c0 con 0x7fb06c0c58a0 2024-04-25T00:00:57.158 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.155+0000 7fb062ffd640 1 -- 172.21.15.157:0/1197066973 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb05c01a070 con 0x7fb06c0c58a0 2024-04-25T00:00:57.158 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.156+0000 7fb062ffd640 1 --2- 172.21.15.157:0/1197066973 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fb04c06be60 0x7fb04c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:57.158 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.156+0000 7fb062ffd640 1 -- 172.21.15.157:0/1197066973 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fb05c08a3d0 con 0x7fb06c0c58a0 2024-04-25T00:00:57.158 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.156+0000 7fb063fff640 1 --2- 172.21.15.157:0/1197066973 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fb04c06be60 0x7fb04c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:57.158 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.156+0000 7fb063fff640 1 --2- 172.21.15.157:0/1197066973 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fb04c06be60 0x7fb04c06e320 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb054009aa0 tx=0x7fb054009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:57.159 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.160+0000 7fb062ffd640 1 -- 172.21.15.157:0/1197066973 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb05c0583c0 con 0x7fb06c0c58a0 2024-04-25T00:00:57.343 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.343+0000 7fb07094f640 1 -- 172.21.15.157:0/1197066973 --> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7fb06c0c5680 con 0x7fb04c06be60 2024-04-25T00:00:57.343 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.343+0000 7fb062ffd640 1 -- 172.21.15.157:0/1197066973 <== mgr.4105 v2:172.21.15.169:6832/662378289 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fb06c0c5680 con 0x7fb04c06be60 2024-04-25T00:00:57.343 INFO:teuthology.orchestra.run.smithi157.stderr:instructed osd(s) 7 to deep-scrub 2024-04-25T00:00:57.346 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.345+0000 7fb07094f640 1 -- 172.21.15.157:0/1197066973 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fb04c06be60 msgr2=0x7fb04c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:57.346 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.345+0000 7fb07094f640 1 --2- 172.21.15.157:0/1197066973 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fb04c06be60 0x7fb04c06e320 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb054009aa0 tx=0x7fb054009190 comp rx=0 tx=0).stop 2024-04-25T00:00:57.346 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.345+0000 7fb07094f640 1 -- 172.21.15.157:0/1197066973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb06c0c58a0 msgr2=0x7fb06c1ec010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:57.346 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.345+0000 7fb07094f640 1 --2- 172.21.15.157:0/1197066973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb06c0c58a0 0x7fb06c1ec010 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb05c00c440 tx=0x7fb05c00c900 comp rx=0 tx=0).stop 2024-04-25T00:00:57.346 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.345+0000 7fb07094f640 1 -- 172.21.15.157:0/1197066973 shutdown_connections 2024-04-25T00:00:57.347 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.345+0000 7fb07094f640 1 --2- 172.21.15.157:0/1197066973 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fb04c06be60 0x7fb04c06e320 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:57.347 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.345+0000 7fb07094f640 1 --2- 172.21.15.157:0/1197066973 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb06c0c90e0 0x7fb06c1ee110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:57.347 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.345+0000 7fb07094f640 1 --2- 172.21.15.157:0/1197066973 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb06c0c68d0 0x7fb06c14f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:57.347 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.345+0000 7fb07094f640 1 --2- 172.21.15.157:0/1197066973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb06c0c58a0 0x7fb06c1ec010 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:57.347 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.345+0000 7fb07094f640 1 -- 172.21.15.157:0/1197066973 >> 172.21.15.157:0/1197066973 conn(0x7fb06c0caca0 msgr2=0x7fb06c0abfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:57.347 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.346+0000 7fb07094f640 1 -- 172.21.15.157:0/1197066973 shutdown_connections 2024-04-25T00:00:57.347 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.346+0000 7fb07094f640 1 -- 172.21.15.157:0/1197066973 wait complete. 2024-04-25T00:00:57.359 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:00:57.458 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.459+0000 7f93b8d34640 1 Processor -- start 2024-04-25T00:00:57.458 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.459+0000 7f93b8d34640 1 -- start start 2024-04-25T00:00:57.461 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.460+0000 7f93b8d34640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f93b4152b40 0x7f93b40c5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:57.461 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.460+0000 7f93b8d34640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f93b40c5b70 0x7f93b4155a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:57.461 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.460+0000 7f93b8d34640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93b40c8bb0 con 0x7f93b40be360 2024-04-25T00:00:57.461 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.460+0000 7f93b8d34640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f93b40c8e90 con 0x7f93b40c5b70 2024-04-25T00:00:57.461 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.460+0000 7f93b8d34640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f93b4159480 con 0x7f93b4152b40 2024-04-25T00:00:57.461 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.460+0000 7f93b2d9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f93b40c5b70 0x7f93b4155a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:57.461 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.460+0000 7f93b2d9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f93b40c5b70 0x7f93b4155a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37800/0 (socket says 172.21.15.157:37800) 2024-04-25T00:00:57.462 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.460+0000 7f93b2d9d640 1 -- 172.21.15.157:0/3662841747 learned_addr learned my addr 172.21.15.157:0/3662841747 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:00:57.462 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.460+0000 7f93b259c640 1 --2- 172.21.15.157:0/3662841747 >> v2:172.21.15.169:3300/0 conn(0x7f93b4152b40 0x7f93b40c5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:57.462 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.460+0000 7f93b2d9d640 1 -- 172.21.15.157:0/3662841747 >> v2:172.21.15.169:3300/0 conn(0x7f93b4152b40 msgr2=0x7f93b40c5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:57.462 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.460+0000 7f93b2d9d640 1 --2- 172.21.15.157:0/3662841747 >> v2:172.21.15.169:3300/0 conn(0x7f93b4152b40 0x7f93b40c5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:57.462 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.460+0000 7f93b2d9d640 1 -- 172.21.15.157:0/3662841747 >> v1:172.21.15.169:6789/0 conn(0x7f93b40be360 legacy=0x7f93b4155350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:57.462 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.460+0000 7f93b2d9d640 1 -- 172.21.15.157:0/3662841747 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93b4159760 con 0x7f93b40c5b70 2024-04-25T00:00:57.462 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.461+0000 7f93b2d9d640 1 --2- 172.21.15.157:0/3662841747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93b40c5b70 0x7f93b4155a60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f93b406cce0 tx=0x7f93ac02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f6b69356be42207 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:57.462 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.461+0000 7f93b159a640 1 -- 172.21.15.157:0/3662841747 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ac036070 con 0x7f93b40c5b70 2024-04-25T00:00:57.462 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.461+0000 7f93b159a640 1 -- 172.21.15.157:0/3662841747 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93ac031600 con 0x7f93b40c5b70 2024-04-25T00:00:57.462 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.461+0000 7f93b159a640 1 -- 172.21.15.157:0/3662841747 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ac0319d0 con 0x7f93b40c5b70 2024-04-25T00:00:57.462 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.461+0000 7f93b8d34640 1 -- 172.21.15.157:0/3662841747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93b40c5b70 msgr2=0x7f93b4155a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:57.462 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.461+0000 7f93b8d34640 1 --2- 172.21.15.157:0/3662841747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93b40c5b70 0x7f93b4155a60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f93b406cce0 tx=0x7f93ac02a610 comp rx=0 tx=0).stop 2024-04-25T00:00:57.462 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.462+0000 7f93b8d34640 1 -- 172.21.15.157:0/3662841747 shutdown_connections 2024-04-25T00:00:57.462 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.462+0000 7f93b8d34640 1 --2- 172.21.15.157:0/3662841747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93b40c5b70 0x7f93b4155a60 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:57.463 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.462+0000 7f93b8d34640 1 --2- 172.21.15.157:0/3662841747 >> v2:172.21.15.169:3300/0 conn(0x7f93b4152b40 0x7f93b40c5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:57.463 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.462+0000 7f93b8d34640 1 -- 172.21.15.157:0/3662841747 >> 172.21.15.157:0/3662841747 conn(0x7f93b40672c0 msgr2=0x7f93b40676d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:57.463 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.462+0000 7f93b8d34640 1 -- 172.21.15.157:0/3662841747 shutdown_connections 2024-04-25T00:00:57.463 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.462+0000 7f93b8d34640 1 -- 172.21.15.157:0/3662841747 wait complete. 2024-04-25T00:00:57.466 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.463+0000 7f93b8d34640 1 Processor -- start 2024-04-25T00:00:57.466 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.463+0000 7f93b8d34640 1 -- start start 2024-04-25T00:00:57.466 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.463+0000 7f93b8d34640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f93b40be360 0x7f93b41483e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:57.466 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.463+0000 7f93b8d34640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93b40c5b70 0x7f93b4157870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:57.466 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.463+0000 7f93b8d34640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93b4152b40 0x7f93b4147820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:57.466 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.463+0000 7f93b8d34640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f93b40caa00 con 0x7f93b4152b40 2024-04-25T00:00:57.466 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.463+0000 7f93b8d34640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f93b40cb210 con 0x7f93b40c5b70 2024-04-25T00:00:57.466 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.463+0000 7f93b8d34640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f93b40cba40 con 0x7f93b40be360 2024-04-25T00:00:57.466 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.464+0000 7f93b2d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93b4152b40 0x7f93b4147820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:57.466 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.464+0000 7f93b2d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93b4152b40 0x7f93b4147820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37808/0 (socket says 172.21.15.157:37808) 2024-04-25T00:00:57.466 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.464+0000 7f93b2d9d640 1 -- 172.21.15.157:0/2931349043 learned_addr learned my addr 172.21.15.157:0/2931349043 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:00:57.467 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.464+0000 7f93b1d9b640 1 --2- 172.21.15.157:0/2931349043 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93b40c5b70 0x7f93b4157870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:57.467 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.464+0000 7f93b259c640 1 --2- 172.21.15.157:0/2931349043 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f93b40be360 0x7f93b41483e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:57.467 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.464+0000 7f93b2d9d640 1 -- 172.21.15.157:0/2931349043 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f93b40be360 msgr2=0x7f93b41483e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:57.467 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.464+0000 7f93b2d9d640 1 --2- 172.21.15.157:0/2931349043 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f93b40be360 0x7f93b41483e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:57.467 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.464+0000 7f93b2d9d640 1 -- 172.21.15.157:0/2931349043 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93b40c5b70 msgr2=0x7f93b4157870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:57.467 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.464+0000 7f93b2d9d640 1 --2- 172.21.15.157:0/2931349043 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93b40c5b70 0x7f93b4157870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:57.467 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.464+0000 7f93b2d9d640 1 -- 172.21.15.157:0/2931349043 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93b41eb240 con 0x7f93b4152b40 2024-04-25T00:00:57.467 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.464+0000 7f93b2d9d640 1 --2- 172.21.15.157:0/2931349043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93b4152b40 0x7f93b4147820 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f93ac009f40 tx=0x7f93ac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:57.467 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.464+0000 7f93a2ffd640 1 -- 172.21.15.157:0/2931349043 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ac03f020 con 0x7f93b4152b40 2024-04-25T00:00:57.467 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.464+0000 7f93a2ffd640 1 -- 172.21.15.157:0/2931349043 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93ac037040 con 0x7f93b4152b40 2024-04-25T00:00:57.467 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.464+0000 7f93b8d34640 1 -- 172.21.15.157:0/2931349043 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93b41eb520 con 0x7f93b4152b40 2024-04-25T00:00:57.467 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.464+0000 7f93a2ffd640 1 -- 172.21.15.157:0/2931349043 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ac036070 con 0x7f93b4152b40 2024-04-25T00:00:57.467 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.464+0000 7f93b8d34640 1 -- 172.21.15.157:0/2931349043 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93b41ebae0 con 0x7f93b4152b40 2024-04-25T00:00:57.467 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.465+0000 7f93b8d34640 1 -- 172.21.15.157:0/2931349043 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93b40681c0 con 0x7f93b4152b40 2024-04-25T00:00:57.468 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.466+0000 7f93a2ffd640 1 -- 172.21.15.157:0/2931349043 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f93ac042020 con 0x7f93b4152b40 2024-04-25T00:00:57.468 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.466+0000 7f93a2ffd640 1 --2- 172.21.15.157:0/2931349043 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f939406be30 0x7f939406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:57.468 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.466+0000 7f93a2ffd640 1 -- 172.21.15.157:0/2931349043 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f93ac0ab5d0 con 0x7f93b4152b40 2024-04-25T00:00:57.468 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.466+0000 7f93b259c640 1 --2- 172.21.15.157:0/2931349043 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f939406be30 0x7f939406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:57.468 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.467+0000 7f93b259c640 1 --2- 172.21.15.157:0/2931349043 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f939406be30 0x7f939406e2f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f93a4009b40 tx=0x7f93a4009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:57.469 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.470+0000 7f93a2ffd640 1 -- 172.21.15.157:0/2931349043 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f93ac049030 con 0x7f93b4152b40 2024-04-25T00:00:57.646 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.646+0000 7f93b8d34640 1 -- 172.21.15.157:0/2931349043 --> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f93b40c1b40 con 0x7f939406be30 2024-04-25T00:00:57.653 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.654+0000 7f93a2ffd640 1 -- 172.21.15.157:0/2931349043 <== mgr.4105 v2:172.21.15.169:6832/662378289 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+334214 (secure 0 0 0) 0x7f93b40c1b40 con 0x7f939406be30 2024-04-25T00:00:57.658 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-25T00:00:57.660 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2024-04-25T00:00:57.662 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.658+0000 7f93b8d34640 1 -- 172.21.15.157:0/2931349043 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f939406be30 msgr2=0x7f939406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:57.662 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.658+0000 7f93b8d34640 1 --2- 172.21.15.157:0/2931349043 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f939406be30 0x7f939406e2f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f93a4009b40 tx=0x7f93a4009190 comp rx=0 tx=0).stop 2024-04-25T00:00:57.662 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.658+0000 7f93b8d34640 1 -- 172.21.15.157:0/2931349043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93b4152b40 msgr2=0x7f93b4147820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:57.662 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.658+0000 7f93b8d34640 1 --2- 172.21.15.157:0/2931349043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93b4152b40 0x7f93b4147820 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f93ac009f40 tx=0x7f93ac004a40 comp rx=0 tx=0).stop 2024-04-25T00:00:57.662 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.658+0000 7f93b8d34640 1 -- 172.21.15.157:0/2931349043 shutdown_connections 2024-04-25T00:00:57.662 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.658+0000 7f93b8d34640 1 --2- 172.21.15.157:0/2931349043 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f939406be30 0x7f939406e2f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:57.662 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.658+0000 7f93b8d34640 1 --2- 172.21.15.157:0/2931349043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93b4152b40 0x7f93b4147820 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:57.662 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.658+0000 7f93b8d34640 1 --2- 172.21.15.157:0/2931349043 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93b40c5b70 0x7f93b4157870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:57.662 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.658+0000 7f93b8d34640 1 --2- 172.21.15.157:0/2931349043 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f93b40be360 0x7f93b41483e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:57.662 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.658+0000 7f93b8d34640 1 -- 172.21.15.157:0/2931349043 >> 172.21.15.157:0/2931349043 conn(0x7f93b40672c0 msgr2=0x7f93b414d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:57.663 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.658+0000 7f93b8d34640 1 -- 172.21.15.157:0/2931349043 shutdown_connections 2024-04-25T00:00:57.663 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:00:57.658+0000 7f93b8d34640 1 -- 172.21.15.157:0/2931349043 wait complete. 2024-04-25T00:00:57.669 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":805,"stamp":"2024-04-25T00:00:56.173915+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":237011790,"num_objects":81,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":0,"num_read":15578,"num_read_kb":3146093,"num_write":423917,"num_write_kb":32349138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":589285,"ondisk_log_size":589285,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":7862608,"kb_used_data":283920,"kb_used_omap":12,"kb_used_meta":7578611,"kb_avail":747112112,"statfs":{"total":773094113280,"available":765042802688,"internally_reserved":0,"allocated":290734080,"data_stored":477824924,"data_compressed":55941184,"data_compressed_allocated":81788928,"data_compressed_original":270307328,"omap_allocated":12752,"internal_metadata":7760498224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,1,0,22,3,60],"upper_bound":64},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-234688074,"num_objects":-1555,"num_object_clones":0,"num_object_copies":-4870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1555,"num_whiteouts":0,"num_read":47,"num_read_kb":7347,"num_write":3865,"num_write_kb":75957,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-113,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008021"},"pg_stats":[{"pgid":"2.1f","version":"27'2678","reported_seq":2717,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.344080+0000","last_change":"2024-04-24T23:34:53.362409+0000","last_active":"2024-04-25T00:00:47.344080+0000","last_peered":"2024-04-25T00:00:47.344080+0000","last_clean":"2024-04-25T00:00:47.344080+0000","last_became_active":"2024-04-24T23:34:53.362104+0000","last_became_peered":"2024-04-24T23:34:53.362104+0000","last_unstale":"2024-04-25T00:00:47.344080+0000","last_undegraded":"2024-04-25T00:00:47.344080+0000","last_fullsized":"2024-04-25T00:00:47.344080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":2678,"log_dups_size":0,"ondisk_log_size":2678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":2687,"num_write_kb":18218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"27'2551","reported_seq":2588,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.988885+0000","last_change":"2024-04-24T23:34:54.373560+0000","last_active":"2024-04-25T00:00:50.988885+0000","last_peered":"2024-04-25T00:00:50.988885+0000","last_clean":"2024-04-25T00:00:50.988885+0000","last_became_active":"2024-04-24T23:34:54.373289+0000","last_became_peered":"2024-04-24T23:34:54.373289+0000","last_unstale":"2024-04-25T00:00:50.988885+0000","last_undegraded":"2024-04-25T00:00:50.988885+0000","last_fullsized":"2024-04-25T00:00:50.988885+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2551,"log_dups_size":0,"ondisk_log_size":2551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"27'9462","reported_seq":9810,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:49.950986+0000","last_change":"2024-04-24T23:34:58.730165+0000","last_active":"2024-04-25T00:00:49.950986+0000","last_peered":"2024-04-25T00:00:49.950986+0000","last_clean":"2024-04-25T00:00:49.950986+0000","last_became_active":"2024-04-24T23:34:58.729816+0000","last_became_peered":"2024-04-24T23:34:58.729816+0000","last_unstale":"2024-04-25T00:00:49.950986+0000","last_undegraded":"2024-04-25T00:00:49.950986+0000","last_fullsized":"2024-04-25T00:00:49.950986+0000","mapping_epoch":23,"log_start":"27'3200","ondisk_log_start":"27'3200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":6262,"log_dups_size":0,"ondisk_log_size":6262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":319,"num_read_kb":81026,"num_write":6761,"num_write_kb":1031213,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"27'6592","reported_seq":6631,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.349319+0000","last_change":"2024-04-24T23:34:53.361650+0000","last_active":"2024-04-25T00:00:47.349319+0000","last_peered":"2024-04-25T00:00:47.349319+0000","last_clean":"2024-04-25T00:00:47.349319+0000","last_became_active":"2024-04-24T23:34:53.361366+0000","last_became_peered":"2024-04-24T23:34:53.361366+0000","last_unstale":"2024-04-25T00:00:47.349319+0000","last_undegraded":"2024-04-25T00:00:47.349319+0000","last_fullsized":"2024-04-25T00:00:47.349319+0000","mapping_epoch":18,"log_start":"27'300","ondisk_log_start":"27'300","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":6292,"log_dups_size":0,"ondisk_log_size":6292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6593,"num_write_kb":53610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"27'2529","reported_seq":2566,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:51.025210+0000","last_change":"2024-04-24T23:34:54.383602+0000","last_active":"2024-04-25T00:00:51.025210+0000","last_peered":"2024-04-25T00:00:51.025210+0000","last_clean":"2024-04-25T00:00:51.025210+0000","last_became_active":"2024-04-24T23:34:54.383160+0000","last_became_peered":"2024-04-24T23:34:54.383160+0000","last_unstale":"2024-04-25T00:00:51.025210+0000","last_undegraded":"2024-04-25T00:00:51.025210+0000","last_fullsized":"2024-04-25T00:00:51.025210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2529,"log_dups_size":0,"ondisk_log_size":2529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"27'9228","reported_seq":9425,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.815605+0000","last_change":"2024-04-24T23:34:58.404416+0000","last_active":"2024-04-25T00:00:50.815605+0000","last_peered":"2024-04-25T00:00:50.815605+0000","last_clean":"2024-04-25T00:00:50.815605+0000","last_became_active":"2024-04-24T23:34:58.403984+0000","last_became_peered":"2024-04-24T23:34:58.403984+0000","last_unstale":"2024-04-25T00:00:50.815605+0000","last_undegraded":"2024-04-25T00:00:50.815605+0000","last_fullsized":"2024-04-25T00:00:50.815605+0000","mapping_epoch":23,"log_start":"27'2060","ondisk_log_start":"27'2060","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7168,"log_dups_size":0,"ondisk_log_size":7168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:46.237082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":169,"num_read_kb":42926,"num_write":6485,"num_write_kb":1085292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"27'10053","reported_seq":10208,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:49.138658+0000","last_change":"2024-04-24T23:34:53.366730+0000","last_active":"2024-04-25T00:00:49.138658+0000","last_peered":"2024-04-25T00:00:49.138658+0000","last_clean":"2024-04-25T00:00:49.138658+0000","last_became_active":"2024-04-24T23:34:53.366430+0000","last_became_peered":"2024-04-24T23:34:53.366430+0000","last_unstale":"2024-04-25T00:00:49.138658+0000","last_undegraded":"2024-04-25T00:00:49.138658+0000","last_fullsized":"2024-04-25T00:00:49.138658+0000","mapping_epoch":18,"log_start":"27'3200","ondisk_log_start":"27'3200","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":6853,"log_dups_size":0,"ondisk_log_size":6853,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:41.422591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5022938,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":120,"num_read_kb":893,"num_write":10045,"num_write_kb":77054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"27'2638","reported_seq":2672,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:51.002770+0000","last_change":"2024-04-24T23:34:54.373592+0000","last_active":"2024-04-25T00:00:51.002770+0000","last_peered":"2024-04-25T00:00:51.002770+0000","last_clean":"2024-04-25T00:00:51.002770+0000","last_became_active":"2024-04-24T23:34:54.372844+0000","last_became_peered":"2024-04-24T23:34:54.372844+0000","last_unstale":"2024-04-25T00:00:51.002770+0000","last_undegraded":"2024-04-25T00:00:51.002770+0000","last_fullsized":"2024-04-25T00:00:51.002770+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2638,"log_dups_size":0,"ondisk_log_size":2638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:49.372040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"27'9403","reported_seq":10019,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.785455+0000","last_change":"2024-04-24T23:34:58.399224+0000","last_active":"2024-04-25T00:00:50.785455+0000","last_peered":"2024-04-25T00:00:50.785455+0000","last_clean":"2024-04-25T00:00:50.785455+0000","last_became_active":"2024-04-24T23:34:58.397817+0000","last_became_peered":"2024-04-24T23:34:58.397817+0000","last_unstale":"2024-04-25T00:00:50.785455+0000","last_undegraded":"2024-04-25T00:00:50.785455+0000","last_fullsized":"2024-04-25T00:00:50.785455+0000","mapping_epoch":23,"log_start":"27'2212","ondisk_log_start":"27'2212","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7191,"log_dups_size":0,"ondisk_log_size":7191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:53.085366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":588,"num_read_kb":149352,"num_write":6710,"num_write_kb":891395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"27'5158","reported_seq":5193,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.195334+0000","last_change":"2024-04-24T23:34:53.362994+0000","last_active":"2024-04-25T00:00:47.195334+0000","last_peered":"2024-04-25T00:00:47.195334+0000","last_clean":"2024-04-25T00:00:47.195334+0000","last_became_active":"2024-04-24T23:34:53.362337+0000","last_became_peered":"2024-04-24T23:34:53.362337+0000","last_unstale":"2024-04-25T00:00:47.195334+0000","last_undegraded":"2024-04-25T00:00:47.195334+0000","last_fullsized":"2024-04-25T00:00:47.195334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":5158,"log_dups_size":0,"ondisk_log_size":5158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:12.258211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":5165,"num_write_kb":44324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"27'2566","reported_seq":2600,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:51.015930+0000","last_change":"2024-04-24T23:34:54.379218+0000","last_active":"2024-04-25T00:00:51.015930+0000","last_peered":"2024-04-25T00:00:51.015930+0000","last_clean":"2024-04-25T00:00:51.015930+0000","last_became_active":"2024-04-24T23:34:54.376019+0000","last_became_peered":"2024-04-24T23:34:54.376019+0000","last_unstale":"2024-04-25T00:00:51.015930+0000","last_undegraded":"2024-04-25T00:00:51.015930+0000","last_fullsized":"2024-04-25T00:00:51.015930+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2566,"log_dups_size":0,"ondisk_log_size":2566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:41:48.207542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"27'8765","reported_seq":9265,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.714347+0000","last_change":"2024-04-24T23:34:58.397152+0000","last_active":"2024-04-25T00:00:50.714347+0000","last_peered":"2024-04-25T00:00:50.714347+0000","last_clean":"2024-04-25T00:00:50.714347+0000","last_became_active":"2024-04-24T23:34:58.396658+0000","last_became_peered":"2024-04-24T23:34:58.396658+0000","last_unstale":"2024-04-25T00:00:50.714347+0000","last_undegraded":"2024-04-25T00:00:50.714347+0000","last_fullsized":"2024-04-25T00:00:50.714347+0000","mapping_epoch":23,"log_start":"27'1609","ondisk_log_start":"27'1609","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7156,"log_dups_size":0,"ondisk_log_size":7156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:41.360776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":472,"num_read_kb":119888,"num_write":5982,"num_write_kb":849049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"27'2561","reported_seq":2596,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.975199+0000","last_change":"2024-04-24T23:34:54.377859+0000","last_active":"2024-04-25T00:00:50.975199+0000","last_peered":"2024-04-25T00:00:50.975199+0000","last_clean":"2024-04-25T00:00:50.975199+0000","last_became_active":"2024-04-24T23:34:54.377576+0000","last_became_peered":"2024-04-24T23:34:54.377576+0000","last_unstale":"2024-04-25T00:00:50.975199+0000","last_undegraded":"2024-04-25T00:00:50.975199+0000","last_fullsized":"2024-04-25T00:00:50.975199+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2561,"log_dups_size":0,"ondisk_log_size":2561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:47.446459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"27'6834","reported_seq":6870,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.226938+0000","last_change":"2024-04-24T23:34:53.360903+0000","last_active":"2024-04-25T00:00:47.226938+0000","last_peered":"2024-04-25T00:00:47.226938+0000","last_clean":"2024-04-25T00:00:47.226938+0000","last_became_active":"2024-04-24T23:34:53.360635+0000","last_became_peered":"2024-04-24T23:34:53.360635+0000","last_unstale":"2024-04-25T00:00:47.226938+0000","last_undegraded":"2024-04-25T00:00:47.226938+0000","last_fullsized":"2024-04-25T00:00:47.226938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":6834,"log_dups_size":0,"ondisk_log_size":6834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:42.205997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6840,"num_write_kb":53782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"27'10213","reported_seq":10495,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.805890+0000","last_change":"2024-04-24T23:34:58.407146+0000","last_active":"2024-04-25T00:00:50.805890+0000","last_peered":"2024-04-25T00:00:50.805890+0000","last_clean":"2024-04-25T00:00:50.805890+0000","last_became_active":"2024-04-24T23:34:58.406844+0000","last_became_peered":"2024-04-24T23:34:58.406844+0000","last_unstale":"2024-04-25T00:00:50.805890+0000","last_undegraded":"2024-04-25T00:00:50.805890+0000","last_fullsized":"2024-04-25T00:00:50.805890+0000","mapping_epoch":23,"log_start":"27'3041","ondisk_log_start":"27'3041","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7172,"log_dups_size":0,"ondisk_log_size":7172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:41:52.244767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":253,"num_read_kb":64262,"num_write":7465,"num_write_kb":1030177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"27'2579","reported_seq":2615,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.977692+0000","last_change":"2024-04-24T23:34:54.366724+0000","last_active":"2024-04-25T00:00:50.977692+0000","last_peered":"2024-04-25T00:00:50.977692+0000","last_clean":"2024-04-25T00:00:50.977692+0000","last_became_active":"2024-04-24T23:34:54.366426+0000","last_became_peered":"2024-04-24T23:34:54.366426+0000","last_unstale":"2024-04-25T00:00:50.977692+0000","last_undegraded":"2024-04-25T00:00:50.977692+0000","last_fullsized":"2024-04-25T00:00:50.977692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2579,"log_dups_size":0,"ondisk_log_size":2579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"27'7913","reported_seq":7951,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.214433+0000","last_change":"2024-04-24T23:34:53.359140+0000","last_active":"2024-04-25T00:00:47.214433+0000","last_peered":"2024-04-25T00:00:47.214433+0000","last_clean":"2024-04-25T00:00:47.214433+0000","last_became_active":"2024-04-24T23:34:53.358841+0000","last_became_peered":"2024-04-24T23:34:53.358841+0000","last_unstale":"2024-04-25T00:00:47.214433+0000","last_undegraded":"2024-04-25T00:00:47.214433+0000","last_fullsized":"2024-04-25T00:00:47.214433+0000","mapping_epoch":18,"log_start":"27'500","ondisk_log_start":"27'500","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":7413,"log_dups_size":0,"ondisk_log_size":7413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7912,"num_write_kb":58475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"27'9714","reported_seq":10146,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.696313+0000","last_change":"2024-04-24T23:34:58.400128+0000","last_active":"2024-04-25T00:00:50.696313+0000","last_peered":"2024-04-25T00:00:50.696313+0000","last_clean":"2024-04-25T00:00:50.696313+0000","last_became_active":"2024-04-24T23:34:58.399375+0000","last_became_peered":"2024-04-24T23:34:58.399375+0000","last_unstale":"2024-04-25T00:00:50.696313+0000","last_undegraded":"2024-04-25T00:00:50.696313+0000","last_fullsized":"2024-04-25T00:00:50.696313+0000","mapping_epoch":23,"log_start":"27'2556","ondisk_log_start":"27'2556","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7158,"log_dups_size":0,"ondisk_log_size":7158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:53.409265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":403,"num_read_kb":102362,"num_write":6994,"num_write_kb":969161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"27'2649","reported_seq":2685,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:51.002091+0000","last_change":"2024-04-24T23:34:54.365012+0000","last_active":"2024-04-25T00:00:51.002091+0000","last_peered":"2024-04-25T00:00:51.002091+0000","last_clean":"2024-04-25T00:00:51.002091+0000","last_became_active":"2024-04-24T23:34:54.364744+0000","last_became_peered":"2024-04-24T23:34:54.364744+0000","last_unstale":"2024-04-25T00:00:51.002091+0000","last_undegraded":"2024-04-25T00:00:51.002091+0000","last_fullsized":"2024-04-25T00:00:51.002091+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2649,"log_dups_size":0,"ondisk_log_size":2649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"27'4022","reported_seq":4061,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:54.633360+0000","last_change":"2024-04-24T23:34:53.360843+0000","last_active":"2024-04-25T00:00:54.633360+0000","last_peered":"2024-04-25T00:00:54.633360+0000","last_clean":"2024-04-25T00:00:54.633360+0000","last_became_active":"2024-04-24T23:34:53.360570+0000","last_became_peered":"2024-04-24T23:34:53.360570+0000","last_unstale":"2024-04-25T00:00:54.633360+0000","last_undegraded":"2024-04-25T00:00:54.633360+0000","last_fullsized":"2024-04-25T00:00:54.633360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":4022,"log_dups_size":0,"ondisk_log_size":4022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4022,"num_write_kb":27320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"27'9656","reported_seq":10078,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.007841+0000","last_change":"2024-04-24T23:34:58.731835+0000","last_active":"2024-04-25T00:00:50.007841+0000","last_peered":"2024-04-25T00:00:50.007841+0000","last_clean":"2024-04-25T00:00:50.007841+0000","last_became_active":"2024-04-24T23:34:58.731485+0000","last_became_peered":"2024-04-24T23:34:58.731485+0000","last_unstale":"2024-04-25T00:00:50.007841+0000","last_undegraded":"2024-04-25T00:00:50.007841+0000","last_fullsized":"2024-04-25T00:00:50.007841+0000","mapping_epoch":23,"log_start":"27'2316","ondisk_log_start":"27'2316","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7340,"log_dups_size":0,"ondisk_log_size":7340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":392,"num_read_kb":99568,"num_write":7039,"num_write_kb":904216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"27'2554","reported_seq":2589,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:51.027913+0000","last_change":"2024-04-24T23:34:54.362447+0000","last_active":"2024-04-25T00:00:51.027913+0000","last_peered":"2024-04-25T00:00:51.027913+0000","last_clean":"2024-04-25T00:00:51.027913+0000","last_became_active":"2024-04-24T23:34:54.362141+0000","last_became_peered":"2024-04-24T23:34:54.362141+0000","last_unstale":"2024-04-25T00:00:51.027913+0000","last_undegraded":"2024-04-25T00:00:51.027913+0000","last_fullsized":"2024-04-25T00:00:51.027913+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2554,"log_dups_size":0,"ondisk_log_size":2554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:51.227333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"27'2570","reported_seq":2607,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.182733+0000","last_change":"2024-04-24T23:34:53.366452+0000","last_active":"2024-04-25T00:00:47.182733+0000","last_peered":"2024-04-25T00:00:47.182733+0000","last_clean":"2024-04-25T00:00:47.182733+0000","last_became_active":"2024-04-24T23:34:53.366110+0000","last_became_peered":"2024-04-24T23:34:53.366110+0000","last_unstale":"2024-04-25T00:00:47.182733+0000","last_undegraded":"2024-04-25T00:00:47.182733+0000","last_fullsized":"2024-04-25T00:00:47.182733+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":2570,"log_dups_size":0,"ondisk_log_size":2570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:14.716741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2576,"num_write_kb":22186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"27'9201","reported_seq":9381,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.195641+0000","last_change":"2024-04-24T23:34:58.402849+0000","last_active":"2024-04-25T00:00:50.195641+0000","last_peered":"2024-04-25T00:00:50.195641+0000","last_clean":"2024-04-25T00:00:50.195641+0000","last_became_active":"2024-04-24T23:34:58.402365+0000","last_became_peered":"2024-04-24T23:34:58.402365+0000","last_unstale":"2024-04-25T00:00:50.195641+0000","last_undegraded":"2024-04-25T00:00:50.195641+0000","last_fullsized":"2024-04-25T00:00:50.195641+0000","mapping_epoch":23,"log_start":"27'2049","ondisk_log_start":"27'2049","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7152,"log_dups_size":0,"ondisk_log_size":7152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:07:32.822427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":151,"num_read_kb":38354,"num_write":6553,"num_write_kb":977677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"27'2524","reported_seq":2561,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:54.187922+0000","last_change":"2024-04-24T23:34:54.370300+0000","last_active":"2024-04-25T00:00:54.187922+0000","last_peered":"2024-04-25T00:00:54.187922+0000","last_clean":"2024-04-25T00:00:54.187922+0000","last_became_active":"2024-04-24T23:34:54.369835+0000","last_became_peered":"2024-04-24T23:34:54.369835+0000","last_unstale":"2024-04-25T00:00:54.187922+0000","last_undegraded":"2024-04-25T00:00:54.187922+0000","last_fullsized":"2024-04-25T00:00:54.187922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2524,"log_dups_size":0,"ondisk_log_size":2524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"27'4484","reported_seq":4522,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.180061+0000","last_change":"2024-04-24T23:34:53.366110+0000","last_active":"2024-04-25T00:00:47.180061+0000","last_peered":"2024-04-25T00:00:47.180061+0000","last_clean":"2024-04-25T00:00:47.180061+0000","last_became_active":"2024-04-24T23:34:53.365783+0000","last_became_peered":"2024-04-24T23:34:53.365783+0000","last_unstale":"2024-04-25T00:00:47.180061+0000","last_undegraded":"2024-04-25T00:00:47.180061+0000","last_fullsized":"2024-04-25T00:00:47.180061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":4484,"log_dups_size":0,"ondisk_log_size":4484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:41.422591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":4494,"num_write_kb":39872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"27'9221","reported_seq":9441,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:49.970638+0000","last_change":"2024-04-24T23:34:58.728191+0000","last_active":"2024-04-25T00:00:49.970638+0000","last_peered":"2024-04-25T00:00:49.970638+0000","last_clean":"2024-04-25T00:00:49.970638+0000","last_became_active":"2024-04-24T23:34:58.727653+0000","last_became_peered":"2024-04-24T23:34:58.727653+0000","last_unstale":"2024-04-25T00:00:49.970638+0000","last_undegraded":"2024-04-25T00:00:49.970638+0000","last_fullsized":"2024-04-25T00:00:49.970638+0000","mapping_epoch":23,"log_start":"27'2006","ondisk_log_start":"27'2006","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7215,"log_dups_size":0,"ondisk_log_size":7215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:30.299011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":190,"num_read_kb":48260,"num_write":6573,"num_write_kb":840416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"27'2508","reported_seq":2544,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:51.015678+0000","last_change":"2024-04-24T23:34:54.368278+0000","last_active":"2024-04-25T00:00:51.015678+0000","last_peered":"2024-04-25T00:00:51.015678+0000","last_clean":"2024-04-25T00:00:51.015678+0000","last_became_active":"2024-04-24T23:34:54.367991+0000","last_became_peered":"2024-04-24T23:34:54.367991+0000","last_unstale":"2024-04-25T00:00:51.015678+0000","last_undegraded":"2024-04-25T00:00:51.015678+0000","last_fullsized":"2024-04-25T00:00:51.015678+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2508,"log_dups_size":0,"ondisk_log_size":2508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"27'7475","reported_seq":8139,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:51.653186+0000","last_change":"2024-04-24T23:34:53.364189+0000","last_active":"2024-04-25T00:00:51.653186+0000","last_peered":"2024-04-25T00:00:51.653186+0000","last_clean":"2024-04-25T00:00:51.653186+0000","last_became_active":"2024-04-24T23:34:53.363842+0000","last_became_peered":"2024-04-24T23:34:53.363842+0000","last_unstale":"2024-04-25T00:00:51.653186+0000","last_undegraded":"2024-04-25T00:00:51.653186+0000","last_fullsized":"2024-04-25T00:00:51.653186+0000","mapping_epoch":18,"log_start":"27'100","ondisk_log_start":"27'100","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":7375,"log_dups_size":0,"ondisk_log_size":7375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":626,"num_read_kb":4509,"num_write":10321,"num_write_kb":61532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"27'8920","reported_seq":9229,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:49.634374+0000","last_change":"2024-04-24T23:34:58.729450+0000","last_active":"2024-04-25T00:00:49.634374+0000","last_peered":"2024-04-25T00:00:49.634374+0000","last_clean":"2024-04-25T00:00:49.634374+0000","last_became_active":"2024-04-24T23:34:58.729082+0000","last_became_peered":"2024-04-24T23:34:58.729082+0000","last_unstale":"2024-04-25T00:00:49.634374+0000","last_undegraded":"2024-04-25T00:00:49.634374+0000","last_fullsized":"2024-04-25T00:00:49.634374+0000","mapping_epoch":23,"log_start":"27'2609","ondisk_log_start":"27'2609","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":6311,"log_dups_size":0,"ondisk_log_size":6311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":278,"num_read_kb":70612,"num_write":6238,"num_write_kb":899702,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"27'2598","reported_seq":2635,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.966475+0000","last_change":"2024-04-24T23:34:54.357246+0000","last_active":"2024-04-25T00:00:50.966475+0000","last_peered":"2024-04-25T00:00:50.966475+0000","last_clean":"2024-04-25T00:00:50.966475+0000","last_became_active":"2024-04-24T23:34:54.356560+0000","last_became_peered":"2024-04-24T23:34:54.356560+0000","last_unstale":"2024-04-25T00:00:50.966475+0000","last_undegraded":"2024-04-25T00:00:50.966475+0000","last_fullsized":"2024-04-25T00:00:50.966475+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2598,"log_dups_size":0,"ondisk_log_size":2598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"27'6261","reported_seq":6298,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.213097+0000","last_change":"2024-04-24T23:34:53.366296+0000","last_active":"2024-04-25T00:00:47.213097+0000","last_peered":"2024-04-25T00:00:47.213097+0000","last_clean":"2024-04-25T00:00:47.213097+0000","last_became_active":"2024-04-24T23:34:53.365636+0000","last_became_peered":"2024-04-24T23:34:53.365636+0000","last_unstale":"2024-04-25T00:00:47.213097+0000","last_undegraded":"2024-04-25T00:00:47.213097+0000","last_fullsized":"2024-04-25T00:00:47.213097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":6261,"log_dups_size":0,"ondisk_log_size":6261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:50.222015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":6274,"num_write_kb":49208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"27'9152","reported_seq":9687,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:48.942043+0000","last_change":"2024-04-24T23:34:58.728271+0000","last_active":"2024-04-25T00:00:48.942043+0000","last_peered":"2024-04-25T00:00:48.942043+0000","last_clean":"2024-04-25T00:00:48.942043+0000","last_became_active":"2024-04-24T23:34:58.727885+0000","last_became_peered":"2024-04-24T23:34:58.727885+0000","last_unstale":"2024-04-25T00:00:48.942043+0000","last_undegraded":"2024-04-25T00:00:48.942043+0000","last_fullsized":"2024-04-25T00:00:48.942043+0000","mapping_epoch":23,"log_start":"27'1968","ondisk_log_start":"27'1968","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7184,"log_dups_size":0,"ondisk_log_size":7184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":504,"num_read_kb":128016,"num_write":6457,"num_write_kb":842362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"27'2505","reported_seq":2541,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:51.022623+0000","last_change":"2024-04-24T23:34:54.372245+0000","last_active":"2024-04-25T00:00:51.022623+0000","last_peered":"2024-04-25T00:00:51.022623+0000","last_clean":"2024-04-25T00:00:51.022623+0000","last_became_active":"2024-04-24T23:34:54.371947+0000","last_became_peered":"2024-04-24T23:34:54.371947+0000","last_unstale":"2024-04-25T00:00:51.022623+0000","last_undegraded":"2024-04-25T00:00:51.022623+0000","last_fullsized":"2024-04-25T00:00:51.022623+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2505,"log_dups_size":0,"ondisk_log_size":2505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:47:52.460508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"27'7226","reported_seq":7265,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:51.044979+0000","last_change":"2024-04-24T23:34:53.360352+0000","last_active":"2024-04-25T00:00:51.044979+0000","last_peered":"2024-04-25T00:00:51.044979+0000","last_clean":"2024-04-25T00:00:51.044979+0000","last_became_active":"2024-04-24T23:34:53.360082+0000","last_became_peered":"2024-04-24T23:34:53.360082+0000","last_unstale":"2024-04-25T00:00:51.044979+0000","last_undegraded":"2024-04-25T00:00:51.044979+0000","last_fullsized":"2024-04-25T00:00:51.044979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":7226,"log_dups_size":0,"ondisk_log_size":7226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17487510,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":7224,"num_write_kb":70858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"27'9197","reported_seq":9383,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.458101+0000","last_change":"2024-04-24T23:34:58.404176+0000","last_active":"2024-04-25T00:00:50.458101+0000","last_peered":"2024-04-25T00:00:50.458101+0000","last_clean":"2024-04-25T00:00:50.458101+0000","last_became_active":"2024-04-24T23:34:58.403735+0000","last_became_peered":"2024-04-24T23:34:58.403735+0000","last_unstale":"2024-04-25T00:00:50.458101+0000","last_undegraded":"2024-04-25T00:00:50.458101+0000","last_fullsized":"2024-04-25T00:00:50.458101+0000","mapping_epoch":23,"log_start":"27'1819","ondisk_log_start":"27'1819","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7378,"log_dups_size":0,"ondisk_log_size":7378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":155,"num_read_kb":39370,"num_write":6529,"num_write_kb":924891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"27'2542","reported_seq":2578,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:51.030774+0000","last_change":"2024-04-24T23:34:54.367638+0000","last_active":"2024-04-25T00:00:51.030774+0000","last_peered":"2024-04-25T00:00:51.030774+0000","last_clean":"2024-04-25T00:00:51.030774+0000","last_became_active":"2024-04-24T23:34:54.367355+0000","last_became_peered":"2024-04-24T23:34:54.367355+0000","last_unstale":"2024-04-25T00:00:51.030774+0000","last_undegraded":"2024-04-25T00:00:51.030774+0000","last_fullsized":"2024-04-25T00:00:51.030774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2542,"log_dups_size":0,"ondisk_log_size":2542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:15.722059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"27'8389","reported_seq":8427,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.171711+0000","last_change":"2024-04-24T23:34:53.356111+0000","last_active":"2024-04-25T00:00:47.171711+0000","last_peered":"2024-04-25T00:00:47.171711+0000","last_clean":"2024-04-25T00:00:47.171711+0000","last_became_active":"2024-04-24T23:34:53.355710+0000","last_became_peered":"2024-04-24T23:34:53.355710+0000","last_unstale":"2024-04-25T00:00:47.171711+0000","last_undegraded":"2024-04-25T00:00:47.171711+0000","last_fullsized":"2024-04-25T00:00:47.171711+0000","mapping_epoch":18,"log_start":"27'600","ondisk_log_start":"27'600","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":7789,"log_dups_size":0,"ondisk_log_size":7789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:21.089467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":8401,"num_write_kb":63182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"27'9677","reported_seq":10042,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.503918+0000","last_change":"2024-04-24T23:34:58.403016+0000","last_active":"2024-04-25T00:00:50.503918+0000","last_peered":"2024-04-25T00:00:50.503918+0000","last_clean":"2024-04-25T00:00:50.503918+0000","last_became_active":"2024-04-24T23:34:58.402598+0000","last_became_peered":"2024-04-24T23:34:58.402598+0000","last_unstale":"2024-04-25T00:00:50.503918+0000","last_undegraded":"2024-04-25T00:00:50.503918+0000","last_fullsized":"2024-04-25T00:00:50.503918+0000","mapping_epoch":23,"log_start":"27'2299","ondisk_log_start":"27'2299","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7378,"log_dups_size":0,"ondisk_log_size":7378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":334,"num_read_kb":84836,"num_write":6883,"num_write_kb":886687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"27'2549","reported_seq":2586,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.975104+0000","last_change":"2024-04-24T23:34:54.356601+0000","last_active":"2024-04-25T00:00:50.975104+0000","last_peered":"2024-04-25T00:00:50.975104+0000","last_clean":"2024-04-25T00:00:50.975104+0000","last_became_active":"2024-04-24T23:34:54.356326+0000","last_became_peered":"2024-04-24T23:34:54.356326+0000","last_unstale":"2024-04-25T00:00:50.975104+0000","last_undegraded":"2024-04-25T00:00:50.975104+0000","last_fullsized":"2024-04-25T00:00:50.975104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2549,"log_dups_size":0,"ondisk_log_size":2549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"27'7177","reported_seq":8144,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.164180+0000","last_change":"2024-04-24T23:34:53.359238+0000","last_active":"2024-04-25T00:00:47.164180+0000","last_peered":"2024-04-25T00:00:47.164180+0000","last_clean":"2024-04-25T00:00:47.164180+0000","last_became_active":"2024-04-24T23:34:53.358937+0000","last_became_peered":"2024-04-24T23:34:53.358937+0000","last_unstale":"2024-04-25T00:00:47.164180+0000","last_undegraded":"2024-04-25T00:00:47.164180+0000","last_fullsized":"2024-04-25T00:00:47.164180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":7177,"log_dups_size":0,"ondisk_log_size":7177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:50.222015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":929,"num_read_kb":4700,"num_write":7175,"num_write_kb":61982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"27'9030","reported_seq":9624,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:49.961912+0000","last_change":"2024-04-24T23:34:58.403258+0000","last_active":"2024-04-25T00:00:49.961912+0000","last_peered":"2024-04-25T00:00:49.961912+0000","last_clean":"2024-04-25T00:00:49.961912+0000","last_became_active":"2024-04-24T23:34:58.402569+0000","last_became_peered":"2024-04-24T23:34:58.402569+0000","last_unstale":"2024-04-25T00:00:49.961912+0000","last_undegraded":"2024-04-25T00:00:49.961912+0000","last_fullsized":"2024-04-25T00:00:49.961912+0000","mapping_epoch":23,"log_start":"27'1638","ondisk_log_start":"27'1638","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7392,"log_dups_size":0,"ondisk_log_size":7392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":564,"num_read_kb":143256,"num_write":6307,"num_write_kb":911438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"27'2483","reported_seq":2520,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.953377+0000","last_change":"2024-04-24T23:34:54.377543+0000","last_active":"2024-04-25T00:00:50.953377+0000","last_peered":"2024-04-25T00:00:50.953377+0000","last_clean":"2024-04-25T00:00:50.953377+0000","last_became_active":"2024-04-24T23:34:54.377228+0000","last_became_peered":"2024-04-24T23:34:54.377228+0000","last_unstale":"2024-04-25T00:00:50.953377+0000","last_undegraded":"2024-04-25T00:00:50.953377+0000","last_fullsized":"2024-04-25T00:00:50.953377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2483,"log_dups_size":0,"ondisk_log_size":2483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"27'7206","reported_seq":7243,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.222770+0000","last_change":"2024-04-24T23:34:53.362675+0000","last_active":"2024-04-25T00:00:47.222770+0000","last_peered":"2024-04-25T00:00:47.222770+0000","last_clean":"2024-04-25T00:00:47.222770+0000","last_became_active":"2024-04-24T23:34:53.362425+0000","last_became_peered":"2024-04-24T23:34:53.362425+0000","last_unstale":"2024-04-25T00:00:47.222770+0000","last_undegraded":"2024-04-25T00:00:47.222770+0000","last_fullsized":"2024-04-25T00:00:47.222770+0000","mapping_epoch":18,"log_start":"27'300","ondisk_log_start":"27'300","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":6906,"log_dups_size":0,"ondisk_log_size":6906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:21.089467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":7208,"num_write_kb":58030,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"27'9329","reported_seq":9853,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:54.188093+0000","last_change":"2024-04-24T23:34:58.728573+0000","last_active":"2024-04-25T00:00:54.188093+0000","last_peered":"2024-04-25T00:00:54.188093+0000","last_clean":"2024-04-25T00:00:54.188093+0000","last_became_active":"2024-04-24T23:34:58.727284+0000","last_became_peered":"2024-04-24T23:34:58.727284+0000","last_unstale":"2024-04-25T00:00:54.188093+0000","last_undegraded":"2024-04-25T00:00:54.188093+0000","last_fullsized":"2024-04-25T00:00:54.188093+0000","mapping_epoch":23,"log_start":"27'2114","ondisk_log_start":"27'2114","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7215,"log_dups_size":0,"ondisk_log_size":7215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":493,"num_read_kb":125222,"num_write":6691,"num_write_kb":1110316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"27'2597","reported_seq":2634,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:51.022063+0000","last_change":"2024-04-24T23:34:54.382820+0000","last_active":"2024-04-25T00:00:51.022063+0000","last_peered":"2024-04-25T00:00:51.022063+0000","last_clean":"2024-04-25T00:00:51.022063+0000","last_became_active":"2024-04-24T23:34:54.382564+0000","last_became_peered":"2024-04-24T23:34:54.382564+0000","last_unstale":"2024-04-25T00:00:51.022063+0000","last_undegraded":"2024-04-25T00:00:51.022063+0000","last_fullsized":"2024-04-25T00:00:51.022063+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2597,"log_dups_size":0,"ondisk_log_size":2597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"27'5189","reported_seq":5227,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.187361+0000","last_change":"2024-04-24T23:34:53.357743+0000","last_active":"2024-04-25T00:00:47.187361+0000","last_peered":"2024-04-25T00:00:47.187361+0000","last_clean":"2024-04-25T00:00:47.187361+0000","last_became_active":"2024-04-24T23:34:53.357456+0000","last_became_peered":"2024-04-24T23:34:53.357456+0000","last_unstale":"2024-04-25T00:00:47.187361+0000","last_undegraded":"2024-04-25T00:00:47.187361+0000","last_fullsized":"2024-04-25T00:00:47.187361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":5189,"log_dups_size":0,"ondisk_log_size":5189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:57:14.381644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5191,"num_write_kb":40339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":193,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-24T23:56:58.615940+0000","last_change":"2024-04-24T23:34:49.778966+0000","last_active":"2024-04-24T23:56:58.615940+0000","last_peered":"2024-04-24T23:56:58.615940+0000","last_clean":"2024-04-24T23:56:58.615940+0000","last_became_active":"2024-04-24T23:34:49.778083+0000","last_became_peered":"2024-04-24T23:34:49.778083+0000","last_unstale":"2024-04-24T23:56:58.615940+0000","last_undegraded":"2024-04-24T23:56:58.615940+0000","last_fullsized":"2024-04-24T23:56:58.615940+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:48.587078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:48.587078+0000","last_clean_scrub_stamp":"2024-04-24T23:34:48.587078+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:46:59.471089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"27'8889","reported_seq":9406,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:49.282602+0000","last_change":"2024-04-24T23:34:58.729779+0000","last_active":"2024-04-25T00:00:49.282602+0000","last_peered":"2024-04-25T00:00:49.282602+0000","last_clean":"2024-04-25T00:00:49.282602+0000","last_became_active":"2024-04-24T23:34:58.729370+0000","last_became_peered":"2024-04-24T23:34:58.729370+0000","last_unstale":"2024-04-25T00:00:49.282602+0000","last_undegraded":"2024-04-25T00:00:49.282602+0000","last_fullsized":"2024-04-25T00:00:49.282602+0000","mapping_epoch":23,"log_start":"27'902","ondisk_log_start":"27'902","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7987,"log_dups_size":0,"ondisk_log_size":7987,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:30.299011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":487,"num_read_kb":123698,"num_write":6268,"num_write_kb":912685,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"27'2533","reported_seq":2567,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:51.036449+0000","last_change":"2024-04-24T23:34:54.384106+0000","last_active":"2024-04-25T00:00:51.036449+0000","last_peered":"2024-04-25T00:00:51.036449+0000","last_clean":"2024-04-25T00:00:51.036449+0000","last_became_active":"2024-04-24T23:34:54.383589+0000","last_became_peered":"2024-04-24T23:34:54.383589+0000","last_unstale":"2024-04-25T00:00:51.036449+0000","last_undegraded":"2024-04-25T00:00:51.036449+0000","last_fullsized":"2024-04-25T00:00:51.036449+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2533,"log_dups_size":0,"ondisk_log_size":2533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:41:48.207542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"27'8231","reported_seq":8268,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.191600+0000","last_change":"2024-04-24T23:34:53.354077+0000","last_active":"2024-04-25T00:00:47.191600+0000","last_peered":"2024-04-25T00:00:47.191600+0000","last_clean":"2024-04-25T00:00:47.191600+0000","last_became_active":"2024-04-24T23:34:53.353804+0000","last_became_peered":"2024-04-24T23:34:53.353804+0000","last_unstale":"2024-04-25T00:00:47.191600+0000","last_undegraded":"2024-04-25T00:00:47.191600+0000","last_fullsized":"2024-04-25T00:00:47.191600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":8231,"log_dups_size":0,"ondisk_log_size":8231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:41.422591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8225,"num_write_kb":58738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"27'9238","reported_seq":9425,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:48.570191+0000","last_change":"2024-04-24T23:34:58.728722+0000","last_active":"2024-04-25T00:00:48.570191+0000","last_peered":"2024-04-25T00:00:48.570191+0000","last_clean":"2024-04-25T00:00:48.570191+0000","last_became_active":"2024-04-24T23:34:58.727649+0000","last_became_peered":"2024-04-24T23:34:58.727649+0000","last_unstale":"2024-04-25T00:00:48.570191+0000","last_undegraded":"2024-04-25T00:00:48.570191+0000","last_fullsized":"2024-04-25T00:00:48.570191+0000","mapping_epoch":23,"log_start":"27'1312","ondisk_log_start":"27'1312","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7926,"log_dups_size":0,"ondisk_log_size":7926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:41:52.244767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":158,"num_read_kb":40132,"num_write":6566,"num_write_kb":960960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"27'2561","reported_seq":2596,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.991982+0000","last_change":"2024-04-24T23:34:54.373371+0000","last_active":"2024-04-25T00:00:50.991982+0000","last_peered":"2024-04-25T00:00:50.991982+0000","last_clean":"2024-04-25T00:00:50.991982+0000","last_became_active":"2024-04-24T23:34:54.372553+0000","last_became_peered":"2024-04-24T23:34:54.372553+0000","last_unstale":"2024-04-25T00:00:50.991982+0000","last_undegraded":"2024-04-25T00:00:50.991982+0000","last_fullsized":"2024-04-25T00:00:50.991982+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2561,"log_dups_size":0,"ondisk_log_size":2561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:26.261786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"27'8394","reported_seq":8432,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.298711+0000","last_change":"2024-04-24T23:34:53.353505+0000","last_active":"2024-04-25T00:00:47.298711+0000","last_peered":"2024-04-25T00:00:47.298711+0000","last_clean":"2024-04-25T00:00:47.298711+0000","last_became_active":"2024-04-24T23:34:53.353186+0000","last_became_peered":"2024-04-24T23:34:53.353186+0000","last_unstale":"2024-04-25T00:00:47.298711+0000","last_undegraded":"2024-04-25T00:00:47.298711+0000","last_fullsized":"2024-04-25T00:00:47.298711+0000","mapping_epoch":18,"log_start":"27'1000","ondisk_log_start":"27'1000","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":7394,"log_dups_size":0,"ondisk_log_size":7394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388678,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8393,"num_write_kb":58880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"27'9582","reported_seq":9938,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:48.891335+0000","last_change":"2024-04-24T23:34:58.400988+0000","last_active":"2024-04-25T00:00:48.891335+0000","last_peered":"2024-04-25T00:00:48.891335+0000","last_clean":"2024-04-25T00:00:48.891335+0000","last_became_active":"2024-04-24T23:34:58.400549+0000","last_became_peered":"2024-04-24T23:34:58.400549+0000","last_unstale":"2024-04-25T00:00:48.891335+0000","last_undegraded":"2024-04-25T00:00:48.891335+0000","last_fullsized":"2024-04-25T00:00:48.891335+0000","mapping_epoch":23,"log_start":"27'2356","ondisk_log_start":"27'2356","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7226,"log_dups_size":0,"ondisk_log_size":7226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":326,"num_read_kb":82804,"num_write":6847,"num_write_kb":886980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"27'2433","reported_seq":2467,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:51.036829+0000","last_change":"2024-04-24T23:34:54.375234+0000","last_active":"2024-04-25T00:00:51.036829+0000","last_peered":"2024-04-25T00:00:51.036829+0000","last_clean":"2024-04-25T00:00:51.036829+0000","last_became_active":"2024-04-24T23:34:54.374963+0000","last_became_peered":"2024-04-24T23:34:54.374963+0000","last_unstale":"2024-04-25T00:00:51.036829+0000","last_undegraded":"2024-04-25T00:00:51.036829+0000","last_fullsized":"2024-04-25T00:00:51.036829+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2433,"log_dups_size":0,"ondisk_log_size":2433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:47:52.460508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"27'8937","reported_seq":8974,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.146208+0000","last_change":"2024-04-24T23:34:53.360531+0000","last_active":"2024-04-25T00:00:47.146208+0000","last_peered":"2024-04-25T00:00:47.146208+0000","last_clean":"2024-04-25T00:00:47.146208+0000","last_became_active":"2024-04-24T23:34:53.360247+0000","last_became_peered":"2024-04-24T23:34:53.360247+0000","last_unstale":"2024-04-25T00:00:47.146208+0000","last_undegraded":"2024-04-25T00:00:47.146208+0000","last_fullsized":"2024-04-25T00:00:47.146208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":8937,"log_dups_size":0,"ondisk_log_size":8937,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:57:14.381644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8935,"num_write_kb":79464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"27'9258","reported_seq":10441,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:49.231199+0000","last_change":"2024-04-24T23:34:58.729798+0000","last_active":"2024-04-25T00:00:49.231199+0000","last_peered":"2024-04-25T00:00:49.231199+0000","last_clean":"2024-04-25T00:00:49.231199+0000","last_became_active":"2024-04-24T23:34:58.729345+0000","last_became_peered":"2024-04-24T23:34:58.729345+0000","last_unstale":"2024-04-25T00:00:49.231199+0000","last_undegraded":"2024-04-25T00:00:49.231199+0000","last_fullsized":"2024-04-25T00:00:49.231199+0000","mapping_epoch":23,"log_start":"27'1527","ondisk_log_start":"27'1527","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7731,"log_dups_size":0,"ondisk_log_size":7731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:06.670518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1154,"num_read_kb":293116,"num_write":6516,"num_write_kb":1022776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"27'2529","reported_seq":2563,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.924912+0000","last_change":"2024-04-24T23:34:54.377593+0000","last_active":"2024-04-25T00:00:50.924912+0000","last_peered":"2024-04-25T00:00:50.924912+0000","last_clean":"2024-04-25T00:00:50.924912+0000","last_became_active":"2024-04-24T23:34:54.377270+0000","last_became_peered":"2024-04-24T23:34:54.377270+0000","last_unstale":"2024-04-25T00:00:50.924912+0000","last_undegraded":"2024-04-25T00:00:50.924912+0000","last_fullsized":"2024-04-25T00:00:50.924912+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2529,"log_dups_size":0,"ondisk_log_size":2529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:26.261786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"27'4549","reported_seq":4586,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:08.704934+0000","last_change":"2024-04-24T23:34:53.352981+0000","last_active":"2024-04-25T00:00:08.704934+0000","last_peered":"2024-04-25T00:00:08.704934+0000","last_clean":"2024-04-25T00:00:08.704934+0000","last_became_active":"2024-04-24T23:34:53.352669+0000","last_became_peered":"2024-04-24T23:34:53.352669+0000","last_unstale":"2024-04-25T00:00:08.704934+0000","last_undegraded":"2024-04-25T00:00:08.704934+0000","last_fullsized":"2024-04-25T00:00:08.704934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":4549,"log_dups_size":0,"ondisk_log_size":4549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:09:49.550050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4539,"num_write_kb":43853,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"27'9062","reported_seq":9842,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.115773+0000","last_change":"2024-04-24T23:35:43.529974+0000","last_active":"2024-04-25T00:00:50.115773+0000","last_peered":"2024-04-25T00:00:50.115773+0000","last_clean":"2024-04-25T00:00:50.115773+0000","last_became_active":"2024-04-24T23:35:43.529375+0000","last_became_peered":"2024-04-24T23:35:43.529375+0000","last_unstale":"2024-04-25T00:00:50.115773+0000","last_undegraded":"2024-04-25T00:00:50.115773+0000","last_fullsized":"2024-04-25T00:00:50.115773+0000","mapping_epoch":26,"log_start":"27'1912","ondisk_log_start":"27'1912","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7150,"log_dups_size":0,"ondisk_log_size":7150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:48.286783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":744,"num_read_kb":188976,"num_write":6372,"num_write_kb":873040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3,6],"acting":[4,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"27'2607","reported_seq":2641,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:51.008008+0000","last_change":"2024-04-24T23:34:54.376800+0000","last_active":"2024-04-25T00:00:51.008008+0000","last_peered":"2024-04-25T00:00:51.008008+0000","last_clean":"2024-04-25T00:00:51.008008+0000","last_became_active":"2024-04-24T23:34:54.376512+0000","last_became_peered":"2024-04-24T23:34:54.376512+0000","last_unstale":"2024-04-25T00:00:51.008008+0000","last_undegraded":"2024-04-25T00:00:51.008008+0000","last_fullsized":"2024-04-25T00:00:51.008008+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2607,"log_dups_size":0,"ondisk_log_size":2607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:49.048141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"27'3996","reported_seq":4033,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.147213+0000","last_change":"2024-04-24T23:34:53.355307+0000","last_active":"2024-04-25T00:00:47.147213+0000","last_peered":"2024-04-25T00:00:47.147213+0000","last_clean":"2024-04-25T00:00:47.147213+0000","last_became_active":"2024-04-24T23:34:53.354988+0000","last_became_peered":"2024-04-24T23:34:53.354988+0000","last_unstale":"2024-04-25T00:00:47.147213+0000","last_undegraded":"2024-04-25T00:00:47.147213+0000","last_fullsized":"2024-04-25T00:00:47.147213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":3996,"log_dups_size":0,"ondisk_log_size":3996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:39.457116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4001,"num_write_kb":39473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"27'9856","reported_seq":10229,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.129294+0000","last_change":"2024-04-24T23:34:58.399188+0000","last_active":"2024-04-25T00:00:50.129294+0000","last_peered":"2024-04-25T00:00:50.129294+0000","last_clean":"2024-04-25T00:00:50.129294+0000","last_became_active":"2024-04-24T23:34:58.398457+0000","last_became_peered":"2024-04-24T23:34:58.398457+0000","last_unstale":"2024-04-25T00:00:50.129294+0000","last_undegraded":"2024-04-25T00:00:50.129294+0000","last_fullsized":"2024-04-25T00:00:50.129294+0000","mapping_epoch":23,"log_start":"27'2619","ondisk_log_start":"27'2619","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7237,"log_dups_size":0,"ondisk_log_size":7237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":344,"num_read_kb":87376,"num_write":7128,"num_write_kb":1170154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"27'2472","reported_seq":2507,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:51.026585+0000","last_change":"2024-04-24T23:34:54.372150+0000","last_active":"2024-04-25T00:00:51.026585+0000","last_peered":"2024-04-25T00:00:51.026585+0000","last_clean":"2024-04-25T00:00:51.026585+0000","last_became_active":"2024-04-24T23:34:54.371703+0000","last_became_peered":"2024-04-24T23:34:54.371703+0000","last_unstale":"2024-04-25T00:00:51.026585+0000","last_undegraded":"2024-04-25T00:00:51.026585+0000","last_fullsized":"2024-04-25T00:00:51.026585+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2472,"log_dups_size":0,"ondisk_log_size":2472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:15.477061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"27'4712","reported_seq":4749,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:42.661191+0000","last_change":"2024-04-24T23:34:53.363787+0000","last_active":"2024-04-25T00:00:42.661191+0000","last_peered":"2024-04-25T00:00:42.661191+0000","last_clean":"2024-04-25T00:00:42.661191+0000","last_became_active":"2024-04-24T23:34:53.363515+0000","last_became_peered":"2024-04-24T23:34:53.363515+0000","last_unstale":"2024-04-25T00:00:42.661191+0000","last_undegraded":"2024-04-25T00:00:42.661191+0000","last_fullsized":"2024-04-25T00:00:42.661191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":4712,"log_dups_size":0,"ondisk_log_size":4712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:50.222015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4705,"num_write_kb":43976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"27'9394","reported_seq":9790,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.527845+0000","last_change":"2024-04-24T23:34:58.728420+0000","last_active":"2024-04-25T00:00:50.527845+0000","last_peered":"2024-04-25T00:00:50.527845+0000","last_clean":"2024-04-25T00:00:50.527845+0000","last_became_active":"2024-04-24T23:34:58.727425+0000","last_became_peered":"2024-04-24T23:34:58.727425+0000","last_unstale":"2024-04-25T00:00:50.527845+0000","last_undegraded":"2024-04-25T00:00:50.527845+0000","last_fullsized":"2024-04-25T00:00:50.527845+0000","mapping_epoch":23,"log_start":"27'3135","ondisk_log_start":"27'3135","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":6259,"log_dups_size":0,"ondisk_log_size":6259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":367,"num_read_kb":93218,"num_write":6677,"num_write_kb":924431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"27'2582","reported_seq":2616,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:51.031593+0000","last_change":"2024-04-24T23:34:54.379296+0000","last_active":"2024-04-25T00:00:51.031593+0000","last_peered":"2024-04-25T00:00:51.031593+0000","last_clean":"2024-04-25T00:00:51.031593+0000","last_became_active":"2024-04-24T23:34:54.379016+0000","last_became_peered":"2024-04-24T23:34:54.379016+0000","last_unstale":"2024-04-25T00:00:51.031593+0000","last_undegraded":"2024-04-25T00:00:51.031593+0000","last_fullsized":"2024-04-25T00:00:51.031593+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2582,"log_dups_size":0,"ondisk_log_size":2582,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:37.323551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"27'2767","reported_seq":2804,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.170814+0000","last_change":"2024-04-24T23:34:53.360210+0000","last_active":"2024-04-25T00:00:47.170814+0000","last_peered":"2024-04-25T00:00:47.170814+0000","last_clean":"2024-04-25T00:00:47.170814+0000","last_became_active":"2024-04-24T23:34:53.359581+0000","last_became_peered":"2024-04-24T23:34:53.359581+0000","last_unstale":"2024-04-25T00:00:47.170814+0000","last_undegraded":"2024-04-25T00:00:47.170814+0000","last_fullsized":"2024-04-25T00:00:47.170814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":2767,"log_dups_size":0,"ondisk_log_size":2767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:14.716741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2773,"num_write_kb":30370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"27'9089","reported_seq":9560,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.722223+0000","last_change":"2024-04-24T23:34:58.728491+0000","last_active":"2024-04-25T00:00:50.722223+0000","last_peered":"2024-04-25T00:00:50.722223+0000","last_clean":"2024-04-25T00:00:50.722223+0000","last_became_active":"2024-04-24T23:34:58.727686+0000","last_became_peered":"2024-04-24T23:34:58.727686+0000","last_unstale":"2024-04-25T00:00:50.722223+0000","last_undegraded":"2024-04-25T00:00:50.722223+0000","last_fullsized":"2024-04-25T00:00:50.722223+0000","mapping_epoch":23,"log_start":"27'2830","ondisk_log_start":"27'2830","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":6259,"log_dups_size":0,"ondisk_log_size":6259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":441,"num_read_kb":112014,"num_write":6536,"num_write_kb":907183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"27'2539","reported_seq":2575,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:51.022024+0000","last_change":"2024-04-24T23:34:54.372009+0000","last_active":"2024-04-25T00:00:51.022024+0000","last_peered":"2024-04-25T00:00:51.022024+0000","last_clean":"2024-04-25T00:00:51.022024+0000","last_became_active":"2024-04-24T23:34:54.371720+0000","last_became_peered":"2024-04-24T23:34:54.371720+0000","last_unstale":"2024-04-25T00:00:51.022024+0000","last_undegraded":"2024-04-25T00:00:51.022024+0000","last_fullsized":"2024-04-25T00:00:51.022024+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2539,"log_dups_size":0,"ondisk_log_size":2539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"27'6531","reported_seq":6568,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.222477+0000","last_change":"2024-04-24T23:34:53.353738+0000","last_active":"2024-04-25T00:00:47.222477+0000","last_peered":"2024-04-25T00:00:47.222477+0000","last_clean":"2024-04-25T00:00:47.222477+0000","last_became_active":"2024-04-24T23:34:53.353452+0000","last_became_peered":"2024-04-24T23:34:53.353452+0000","last_unstale":"2024-04-25T00:00:47.222477+0000","last_undegraded":"2024-04-25T00:00:47.222477+0000","last_fullsized":"2024-04-25T00:00:47.222477+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":6531,"log_dups_size":0,"ondisk_log_size":6531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971554,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6522,"num_write_kb":57515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"27'9615","reported_seq":10064,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:48.937364+0000","last_change":"2024-04-24T23:34:58.728503+0000","last_active":"2024-04-25T00:00:48.937364+0000","last_peered":"2024-04-25T00:00:48.937364+0000","last_clean":"2024-04-25T00:00:48.937364+0000","last_became_active":"2024-04-24T23:34:58.727921+0000","last_became_peered":"2024-04-24T23:34:58.727921+0000","last_unstale":"2024-04-25T00:00:48.937364+0000","last_undegraded":"2024-04-25T00:00:48.937364+0000","last_fullsized":"2024-04-25T00:00:48.937364+0000","mapping_epoch":23,"log_start":"27'1833","ondisk_log_start":"27'1833","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7782,"log_dups_size":0,"ondisk_log_size":7782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:19:40.928966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":421,"num_read_kb":106934,"num_write":7001,"num_write_kb":1058941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"27'2478","reported_seq":2511,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:51.041200+0000","last_change":"2024-04-24T23:34:54.368532+0000","last_active":"2024-04-25T00:00:51.041200+0000","last_peered":"2024-04-25T00:00:51.041200+0000","last_clean":"2024-04-25T00:00:51.041200+0000","last_became_active":"2024-04-24T23:34:54.367903+0000","last_became_peered":"2024-04-24T23:34:54.367903+0000","last_unstale":"2024-04-25T00:00:51.041200+0000","last_undegraded":"2024-04-25T00:00:51.041200+0000","last_fullsized":"2024-04-25T00:00:51.041200+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2478,"log_dups_size":0,"ondisk_log_size":2478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:49.372040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'9568","reported_seq":10206,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.204974+0000","last_change":"2024-04-24T23:34:53.362758+0000","last_active":"2024-04-25T00:00:47.204974+0000","last_peered":"2024-04-25T00:00:47.204974+0000","last_clean":"2024-04-25T00:00:47.204974+0000","last_became_active":"2024-04-24T23:34:53.362230+0000","last_became_peered":"2024-04-24T23:34:53.362230+0000","last_unstale":"2024-04-25T00:00:47.204974+0000","last_undegraded":"2024-04-25T00:00:47.204974+0000","last_fullsized":"2024-04-25T00:00:47.204974+0000","mapping_epoch":18,"log_start":"27'1600","ondisk_log_start":"27'1600","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":7968,"log_dups_size":0,"ondisk_log_size":7968,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:14.716741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389280,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":602,"num_read_kb":4494,"num_write":9568,"num_write_kb":76044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"27'9012","reported_seq":9419,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.335945+0000","last_change":"2024-04-24T23:34:58.400881+0000","last_active":"2024-04-25T00:00:50.335945+0000","last_peered":"2024-04-25T00:00:50.335945+0000","last_clean":"2024-04-25T00:00:50.335945+0000","last_became_active":"2024-04-24T23:34:58.400429+0000","last_became_peered":"2024-04-24T23:34:58.400429+0000","last_unstale":"2024-04-25T00:00:50.335945+0000","last_undegraded":"2024-04-25T00:00:50.335945+0000","last_fullsized":"2024-04-25T00:00:50.335945+0000","mapping_epoch":23,"log_start":"27'1625","ondisk_log_start":"27'1625","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7387,"log_dups_size":0,"ondisk_log_size":7387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":377,"num_read_kb":95758,"num_write":6342,"num_write_kb":955265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"27'2558","reported_seq":2593,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:51.032939+0000","last_change":"2024-04-24T23:34:54.373603+0000","last_active":"2024-04-25T00:00:51.032939+0000","last_peered":"2024-04-25T00:00:51.032939+0000","last_clean":"2024-04-25T00:00:51.032939+0000","last_became_active":"2024-04-24T23:34:54.373347+0000","last_became_peered":"2024-04-24T23:34:54.373347+0000","last_unstale":"2024-04-25T00:00:51.032939+0000","last_undegraded":"2024-04-25T00:00:51.032939+0000","last_fullsized":"2024-04-25T00:00:51.032939+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2558,"log_dups_size":0,"ondisk_log_size":2558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:13.263529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"27'5925","reported_seq":5962,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.819706+0000","last_change":"2024-04-24T23:34:53.354673+0000","last_active":"2024-04-25T00:00:50.819706+0000","last_peered":"2024-04-25T00:00:50.819706+0000","last_clean":"2024-04-25T00:00:50.819706+0000","last_became_active":"2024-04-24T23:34:53.354394+0000","last_became_peered":"2024-04-24T23:34:53.354394+0000","last_unstale":"2024-04-25T00:00:50.819706+0000","last_undegraded":"2024-04-25T00:00:50.819706+0000","last_fullsized":"2024-04-25T00:00:50.819706+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":5925,"log_dups_size":0,"ondisk_log_size":5925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5926,"num_write_kb":41236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"27'9157","reported_seq":9564,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.803421+0000","last_change":"2024-04-24T23:34:58.397739+0000","last_active":"2024-04-25T00:00:50.803421+0000","last_peered":"2024-04-25T00:00:50.803421+0000","last_clean":"2024-04-25T00:00:50.803421+0000","last_became_active":"2024-04-24T23:34:58.397353+0000","last_became_peered":"2024-04-24T23:34:58.397353+0000","last_unstale":"2024-04-25T00:00:50.803421+0000","last_undegraded":"2024-04-25T00:00:50.803421+0000","last_fullsized":"2024-04-25T00:00:50.803421+0000","mapping_epoch":23,"log_start":"27'2001","ondisk_log_start":"27'2001","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7156,"log_dups_size":0,"ondisk_log_size":7156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:55.672101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":378,"num_read_kb":96012,"num_write":6474,"num_write_kb":975733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"27'2584","reported_seq":2619,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:51.000993+0000","last_change":"2024-04-24T23:34:54.383348+0000","last_active":"2024-04-25T00:00:51.000993+0000","last_peered":"2024-04-25T00:00:51.000993+0000","last_clean":"2024-04-25T00:00:51.000993+0000","last_became_active":"2024-04-24T23:34:54.383082+0000","last_became_peered":"2024-04-24T23:34:54.383082+0000","last_unstale":"2024-04-25T00:00:51.000993+0000","last_undegraded":"2024-04-25T00:00:51.000993+0000","last_fullsized":"2024-04-25T00:00:51.000993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2584,"log_dups_size":0,"ondisk_log_size":2584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"27'7101","reported_seq":7137,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.223449+0000","last_change":"2024-04-24T23:34:53.364540+0000","last_active":"2024-04-25T00:00:47.223449+0000","last_peered":"2024-04-25T00:00:47.223449+0000","last_clean":"2024-04-25T00:00:47.223449+0000","last_became_active":"2024-04-24T23:34:53.364259+0000","last_became_peered":"2024-04-24T23:34:53.364259+0000","last_unstale":"2024-04-25T00:00:47.223449+0000","last_undegraded":"2024-04-25T00:00:47.223449+0000","last_fullsized":"2024-04-25T00:00:47.223449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":7101,"log_dups_size":0,"ondisk_log_size":7101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:41.422591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":7102,"num_write_kb":53865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"27'9038","reported_seq":9137,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.093414+0000","last_change":"2024-04-24T23:34:58.405865+0000","last_active":"2024-04-25T00:00:50.093414+0000","last_peered":"2024-04-25T00:00:50.093414+0000","last_clean":"2024-04-25T00:00:50.093414+0000","last_became_active":"2024-04-24T23:34:58.405032+0000","last_became_peered":"2024-04-24T23:34:58.405032+0000","last_unstale":"2024-04-25T00:00:50.093414+0000","last_undegraded":"2024-04-25T00:00:50.093414+0000","last_fullsized":"2024-04-25T00:00:50.093414+0000","mapping_epoch":23,"log_start":"27'2712","ondisk_log_start":"27'2712","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":6326,"log_dups_size":0,"ondisk_log_size":6326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":17526,"num_write":6404,"num_write_kb":965810,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"27'2558","reported_seq":2592,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.977619+0000","last_change":"2024-04-24T23:34:54.383631+0000","last_active":"2024-04-25T00:00:50.977619+0000","last_peered":"2024-04-25T00:00:50.977619+0000","last_clean":"2024-04-25T00:00:50.977619+0000","last_became_active":"2024-04-24T23:34:54.383354+0000","last_became_peered":"2024-04-24T23:34:54.383354+0000","last_unstale":"2024-04-25T00:00:50.977619+0000","last_undegraded":"2024-04-25T00:00:50.977619+0000","last_fullsized":"2024-04-25T00:00:50.977619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2558,"log_dups_size":0,"ondisk_log_size":2558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:50:17.621315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"27'6418","reported_seq":7300,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.177210+0000","last_change":"2024-04-24T23:34:53.362409+0000","last_active":"2024-04-25T00:00:47.177210+0000","last_peered":"2024-04-25T00:00:47.177210+0000","last_clean":"2024-04-25T00:00:47.177210+0000","last_became_active":"2024-04-24T23:34:53.362103+0000","last_became_peered":"2024-04-24T23:34:53.362103+0000","last_unstale":"2024-04-25T00:00:47.177210+0000","last_undegraded":"2024-04-25T00:00:47.177210+0000","last_fullsized":"2024-04-25T00:00:47.177210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":6418,"log_dups_size":0,"ondisk_log_size":6418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:57:14.381644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":847,"num_read_kb":4657,"num_write":6417,"num_write_kb":49337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"27'9416","reported_seq":9705,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.222154+0000","last_change":"2024-04-24T23:34:58.730088+0000","last_active":"2024-04-25T00:00:50.222154+0000","last_peered":"2024-04-25T00:00:50.222154+0000","last_clean":"2024-04-25T00:00:50.222154+0000","last_became_active":"2024-04-24T23:34:58.729217+0000","last_became_peered":"2024-04-24T23:34:58.729217+0000","last_unstale":"2024-04-25T00:00:50.222154+0000","last_undegraded":"2024-04-25T00:00:50.222154+0000","last_fullsized":"2024-04-25T00:00:50.222154+0000","mapping_epoch":23,"log_start":"27'3115","ondisk_log_start":"27'3115","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":6301,"log_dups_size":0,"ondisk_log_size":6301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":259,"num_read_kb":65786,"num_write":6628,"num_write_kb":1056336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"27'9147","reported_seq":9277,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:49.928931+0000","last_change":"2024-04-24T23:34:58.728892+0000","last_active":"2024-04-25T00:00:49.928931+0000","last_peered":"2024-04-25T00:00:49.928931+0000","last_clean":"2024-04-25T00:00:49.928931+0000","last_became_active":"2024-04-24T23:34:58.727914+0000","last_became_peered":"2024-04-24T23:34:58.727914+0000","last_unstale":"2024-04-25T00:00:49.928931+0000","last_undegraded":"2024-04-25T00:00:49.928931+0000","last_fullsized":"2024-04-25T00:00:49.928931+0000","mapping_epoch":23,"log_start":"27'1931","ondisk_log_start":"27'1931","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7216,"log_dups_size":0,"ondisk_log_size":7216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":99,"num_read_kb":25146,"num_write":6551,"num_write_kb":920441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"27'2499","reported_seq":2542,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.125218+0000","last_change":"2024-04-25T00:00:55.125262+0000","last_active":"2024-04-25T00:00:55.125218+0000","last_peered":"2024-04-25T00:00:55.125218+0000","last_clean":"2024-04-25T00:00:55.125218+0000","last_became_active":"2024-04-24T23:34:54.363724+0000","last_became_peered":"2024-04-24T23:34:54.363724+0000","last_unstale":"2024-04-25T00:00:55.125218+0000","last_undegraded":"2024-04-25T00:00:55.125218+0000","last_fullsized":"2024-04-25T00:00:55.125218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2499","last_scrub_stamp":"2024-04-25T00:00:55.125079+0000","last_deep_scrub":"27'2499","last_deep_scrub_stamp":"2024-04-25T00:00:55.125079+0000","last_clean_scrub_stamp":"2024-04-25T00:00:55.125079+0000","objects_scrubbed":0,"log_size":2499,"log_dups_size":0,"ondisk_log_size":2499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:41.142244+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"27'9687","reported_seq":9723,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.373826+0000","last_change":"2024-04-24T23:34:53.360041+0000","last_active":"2024-04-25T00:00:47.373826+0000","last_peered":"2024-04-25T00:00:47.373826+0000","last_clean":"2024-04-25T00:00:47.373826+0000","last_became_active":"2024-04-24T23:34:53.359707+0000","last_became_peered":"2024-04-24T23:34:53.359707+0000","last_unstale":"2024-04-25T00:00:47.373826+0000","last_undegraded":"2024-04-25T00:00:47.373826+0000","last_fullsized":"2024-04-25T00:00:47.373826+0000","mapping_epoch":18,"log_start":"27'1900","ondisk_log_start":"27'1900","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":7787,"log_dups_size":0,"ondisk_log_size":7787,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:50.222015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9677,"num_write_kb":67972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"27'9609","reported_seq":10700,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:49.203484+0000","last_change":"2024-04-24T23:35:43.834517+0000","last_active":"2024-04-25T00:00:49.203484+0000","last_peered":"2024-04-25T00:00:49.203484+0000","last_clean":"2024-04-25T00:00:49.203484+0000","last_became_active":"2024-04-24T23:35:43.834066+0000","last_became_peered":"2024-04-24T23:35:43.834066+0000","last_unstale":"2024-04-25T00:00:49.203484+0000","last_undegraded":"2024-04-25T00:00:49.203484+0000","last_fullsized":"2024-04-25T00:00:49.203484+0000","mapping_epoch":26,"log_start":"27'1708","ondisk_log_start":"27'1708","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":7901,"log_dups_size":0,"ondisk_log_size":7901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:48.286783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1054,"num_read_kb":267716,"num_write":6908,"num_write_kb":899799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2,0],"acting":[6,5,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"27'2603","reported_seq":2638,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:51.008680+0000","last_change":"2024-04-24T23:34:54.369729+0000","last_active":"2024-04-25T00:00:51.008680+0000","last_peered":"2024-04-25T00:00:51.008680+0000","last_clean":"2024-04-25T00:00:51.008680+0000","last_became_active":"2024-04-24T23:34:54.369155+0000","last_became_peered":"2024-04-24T23:34:54.369155+0000","last_unstale":"2024-04-25T00:00:51.008680+0000","last_undegraded":"2024-04-25T00:00:51.008680+0000","last_fullsized":"2024-04-25T00:00:51.008680+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2603,"log_dups_size":0,"ondisk_log_size":2603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"27'4844","reported_seq":4881,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.216042+0000","last_change":"2024-04-24T23:34:53.365328+0000","last_active":"2024-04-25T00:00:47.216042+0000","last_peered":"2024-04-25T00:00:47.216042+0000","last_clean":"2024-04-25T00:00:47.216042+0000","last_became_active":"2024-04-24T23:34:53.365029+0000","last_became_peered":"2024-04-24T23:34:53.365029+0000","last_unstale":"2024-04-25T00:00:47.216042+0000","last_undegraded":"2024-04-25T00:00:47.216042+0000","last_fullsized":"2024-04-25T00:00:47.216042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":4844,"log_dups_size":0,"ondisk_log_size":4844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4849,"num_write_kb":44132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"27'9228","reported_seq":9264,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.258606+0000","last_change":"2024-04-24T23:34:53.365123+0000","last_active":"2024-04-25T00:00:47.258606+0000","last_peered":"2024-04-25T00:00:47.258606+0000","last_clean":"2024-04-25T00:00:47.258606+0000","last_became_active":"2024-04-24T23:34:53.364821+0000","last_became_peered":"2024-04-24T23:34:53.364821+0000","last_unstale":"2024-04-25T00:00:47.258606+0000","last_undegraded":"2024-04-25T00:00:47.258606+0000","last_fullsized":"2024-04-25T00:00:47.258606+0000","mapping_epoch":18,"log_start":"27'1300","ondisk_log_start":"27'1300","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":7928,"log_dups_size":0,"ondisk_log_size":7928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:39.457116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9220,"num_write_kb":71599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"27'2536","reported_seq":2579,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.053278+0000","last_change":"2024-04-25T00:00:55.053300+0000","last_active":"2024-04-25T00:00:55.053278+0000","last_peered":"2024-04-25T00:00:55.053278+0000","last_clean":"2024-04-25T00:00:55.053278+0000","last_became_active":"2024-04-24T23:34:54.375689+0000","last_became_peered":"2024-04-24T23:34:54.375689+0000","last_unstale":"2024-04-25T00:00:55.053278+0000","last_undegraded":"2024-04-25T00:00:55.053278+0000","last_fullsized":"2024-04-25T00:00:55.053278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2536","last_scrub_stamp":"2024-04-25T00:00:55.053203+0000","last_deep_scrub":"27'2536","last_deep_scrub_stamp":"2024-04-25T00:00:55.053203+0000","last_clean_scrub_stamp":"2024-04-25T00:00:55.053203+0000","objects_scrubbed":0,"log_size":2536,"log_dups_size":0,"ondisk_log_size":2536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:49:14.011832+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"27'9729","reported_seq":9953,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.454702+0000","last_change":"2024-04-24T23:35:43.834538+0000","last_active":"2024-04-25T00:00:50.454702+0000","last_peered":"2024-04-25T00:00:50.454702+0000","last_clean":"2024-04-25T00:00:50.454702+0000","last_became_active":"2024-04-24T23:35:43.834061+0000","last_became_peered":"2024-04-24T23:35:43.834061+0000","last_unstale":"2024-04-25T00:00:50.454702+0000","last_undegraded":"2024-04-25T00:00:50.454702+0000","last_fullsized":"2024-04-25T00:00:50.454702+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":9729,"log_dups_size":0,"ondisk_log_size":9729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:41:52.244767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":189,"num_read_kb":48006,"num_write":6998,"num_write_kb":973944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3,5],"acting":[2,0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"27'7147","reported_seq":7184,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:47.203327+0000","last_change":"2024-04-24T23:34:53.363066+0000","last_active":"2024-04-25T00:00:47.203327+0000","last_peered":"2024-04-25T00:00:47.203327+0000","last_clean":"2024-04-25T00:00:47.203327+0000","last_became_active":"2024-04-24T23:34:53.362548+0000","last_became_peered":"2024-04-24T23:34:53.362548+0000","last_unstale":"2024-04-25T00:00:47.203327+0000","last_undegraded":"2024-04-25T00:00:47.203327+0000","last_fullsized":"2024-04-25T00:00:47.203327+0000","mapping_epoch":18,"log_start":"27'800","ondisk_log_start":"27'800","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:52.333566+0000","last_clean_scrub_stamp":"2024-04-24T23:34:52.333566+0000","objects_scrubbed":0,"log_size":6347,"log_dups_size":0,"ondisk_log_size":6347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7134,"num_write_kb":66117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"27'2573","reported_seq":2607,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.973745+0000","last_change":"2024-04-24T23:34:54.365704+0000","last_active":"2024-04-25T00:00:50.973745+0000","last_peered":"2024-04-25T00:00:50.973745+0000","last_clean":"2024-04-25T00:00:50.973745+0000","last_became_active":"2024-04-24T23:34:54.364814+0000","last_became_peered":"2024-04-24T23:34:54.364814+0000","last_unstale":"2024-04-25T00:00:50.973745+0000","last_undegraded":"2024-04-25T00:00:50.973745+0000","last_fullsized":"2024-04-25T00:00:50.973745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2573,"log_dups_size":0,"ondisk_log_size":2573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:22.094785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"27'9398","reported_seq":9605,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.632514+0000","last_change":"2024-04-24T23:34:58.407186+0000","last_active":"2024-04-25T00:00:50.632514+0000","last_peered":"2024-04-25T00:00:50.632514+0000","last_clean":"2024-04-25T00:00:50.632514+0000","last_became_active":"2024-04-24T23:34:58.406811+0000","last_became_peered":"2024-04-24T23:34:58.406811+0000","last_unstale":"2024-04-25T00:00:50.632514+0000","last_undegraded":"2024-04-25T00:00:50.632514+0000","last_fullsized":"2024-04-25T00:00:50.632514+0000","mapping_epoch":23,"log_start":"27'3112","ondisk_log_start":"27'3112","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:57.376109+0000","last_clean_scrub_stamp":"2024-04-24T23:34:57.376109+0000","objects_scrubbed":0,"log_size":6286,"log_dups_size":0,"ondisk_log_size":6286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":178,"num_read_kb":45212,"num_write":6641,"num_write_kb":1045709,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"27'2471","reported_seq":2507,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:51.031872+0000","last_change":"2024-04-24T23:34:54.375142+0000","last_active":"2024-04-25T00:00:51.031872+0000","last_peered":"2024-04-25T00:00:51.031872+0000","last_clean":"2024-04-25T00:00:51.031872+0000","last_became_active":"2024-04-24T23:34:54.374844+0000","last_became_peered":"2024-04-24T23:34:54.374844+0000","last_unstale":"2024-04-25T00:00:51.031872+0000","last_undegraded":"2024-04-25T00:00:51.031872+0000","last_fullsized":"2024-04-25T00:00:51.031872+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2471,"log_dups_size":0,"ondisk_log_size":2471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:29:07.593498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"27'2599","reported_seq":2635,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.943647+0000","last_change":"2024-04-24T23:34:54.383574+0000","last_active":"2024-04-25T00:00:50.943647+0000","last_peered":"2024-04-25T00:00:50.943647+0000","last_clean":"2024-04-25T00:00:50.943647+0000","last_became_active":"2024-04-24T23:34:54.383253+0000","last_became_peered":"2024-04-24T23:34:54.383253+0000","last_unstale":"2024-04-25T00:00:50.943647+0000","last_undegraded":"2024-04-25T00:00:50.943647+0000","last_fullsized":"2024-04-25T00:00:50.943647+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2599,"log_dups_size":0,"ondisk_log_size":2599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:43.211314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"27'2500","reported_seq":2522,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.991961+0000","last_change":"2024-04-24T23:35:43.526600+0000","last_active":"2024-04-25T00:00:50.991961+0000","last_peered":"2024-04-25T00:00:50.991961+0000","last_clean":"2024-04-25T00:00:50.991961+0000","last_became_active":"2024-04-24T23:35:43.525929+0000","last_became_peered":"2024-04-24T23:35:43.525929+0000","last_unstale":"2024-04-25T00:00:50.991961+0000","last_undegraded":"2024-04-25T00:00:50.991961+0000","last_fullsized":"2024-04-25T00:00:50.991961+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2500,"log_dups_size":0,"ondisk_log_size":2500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:49.372040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"27'2582","reported_seq":2619,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.986663+0000","last_change":"2024-04-24T23:34:54.366271+0000","last_active":"2024-04-25T00:00:50.986663+0000","last_peered":"2024-04-25T00:00:50.986663+0000","last_clean":"2024-04-25T00:00:50.986663+0000","last_became_active":"2024-04-24T23:34:54.366005+0000","last_became_peered":"2024-04-24T23:34:54.366005+0000","last_unstale":"2024-04-25T00:00:50.986663+0000","last_undegraded":"2024-04-25T00:00:50.986663+0000","last_fullsized":"2024-04-25T00:00:50.986663+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2582,"log_dups_size":0,"ondisk_log_size":2582,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"27'2585","reported_seq":2622,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:51.107699+0000","last_change":"2024-04-24T23:34:54.360529+0000","last_active":"2024-04-25T00:00:51.107699+0000","last_peered":"2024-04-25T00:00:51.107699+0000","last_clean":"2024-04-25T00:00:51.107699+0000","last_became_active":"2024-04-24T23:34:54.360266+0000","last_became_peered":"2024-04-24T23:34:54.360266+0000","last_unstale":"2024-04-25T00:00:51.107699+0000","last_undegraded":"2024-04-25T00:00:51.107699+0000","last_fullsized":"2024-04-25T00:00:51.107699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2585,"log_dups_size":0,"ondisk_log_size":2585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"27'2453","reported_seq":2489,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.968411+0000","last_change":"2024-04-24T23:34:54.365744+0000","last_active":"2024-04-25T00:00:50.968411+0000","last_peered":"2024-04-25T00:00:50.968411+0000","last_clean":"2024-04-25T00:00:50.968411+0000","last_became_active":"2024-04-24T23:34:54.365493+0000","last_became_peered":"2024-04-24T23:34:54.365493+0000","last_unstale":"2024-04-25T00:00:50.968411+0000","last_undegraded":"2024-04-25T00:00:50.968411+0000","last_fullsized":"2024-04-25T00:00:50.968411+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2453,"log_dups_size":0,"ondisk_log_size":2453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"27'2590","reported_seq":2625,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:51.032201+0000","last_change":"2024-04-24T23:34:54.379642+0000","last_active":"2024-04-25T00:00:51.032201+0000","last_peered":"2024-04-25T00:00:51.032201+0000","last_clean":"2024-04-25T00:00:51.032201+0000","last_became_active":"2024-04-24T23:34:54.378568+0000","last_became_peered":"2024-04-24T23:34:54.378568+0000","last_unstale":"2024-04-25T00:00:51.032201+0000","last_undegraded":"2024-04-25T00:00:51.032201+0000","last_fullsized":"2024-04-25T00:00:51.032201+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2590,"log_dups_size":0,"ondisk_log_size":2590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:42.199856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"27'2557","reported_seq":2593,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.984829+0000","last_change":"2024-04-24T23:34:54.363700+0000","last_active":"2024-04-25T00:00:50.984829+0000","last_peered":"2024-04-25T00:00:50.984829+0000","last_clean":"2024-04-25T00:00:50.984829+0000","last_became_active":"2024-04-24T23:34:54.363439+0000","last_became_peered":"2024-04-24T23:34:54.363439+0000","last_unstale":"2024-04-25T00:00:50.984829+0000","last_undegraded":"2024-04-25T00:00:50.984829+0000","last_fullsized":"2024-04-25T00:00:50.984829+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2557,"log_dups_size":0,"ondisk_log_size":2557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"27'2540","reported_seq":2575,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:51.006337+0000","last_change":"2024-04-24T23:34:54.375460+0000","last_active":"2024-04-25T00:00:51.006337+0000","last_peered":"2024-04-25T00:00:51.006337+0000","last_clean":"2024-04-25T00:00:51.006337+0000","last_became_active":"2024-04-24T23:34:54.375123+0000","last_became_peered":"2024-04-24T23:34:54.375123+0000","last_unstale":"2024-04-25T00:00:51.006337+0000","last_undegraded":"2024-04-25T00:00:51.006337+0000","last_fullsized":"2024-04-25T00:00:51.006337+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2540,"log_dups_size":0,"ondisk_log_size":2540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:18.207267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"27'2526","reported_seq":2562,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:51.025827+0000","last_change":"2024-04-24T23:34:54.369509+0000","last_active":"2024-04-25T00:00:51.025827+0000","last_peered":"2024-04-25T00:00:51.025827+0000","last_clean":"2024-04-25T00:00:51.025827+0000","last_became_active":"2024-04-24T23:34:54.369219+0000","last_became_peered":"2024-04-24T23:34:54.369219+0000","last_unstale":"2024-04-25T00:00:51.025827+0000","last_undegraded":"2024-04-25T00:00:51.025827+0000","last_fullsized":"2024-04-25T00:00:51.025827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2526,"log_dups_size":0,"ondisk_log_size":2526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"27'2564","reported_seq":2585,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.939852+0000","last_change":"2024-04-24T23:35:43.933911+0000","last_active":"2024-04-25T00:00:50.939852+0000","last_peered":"2024-04-25T00:00:50.939852+0000","last_clean":"2024-04-25T00:00:50.939852+0000","last_became_active":"2024-04-24T23:35:43.933686+0000","last_became_peered":"2024-04-24T23:35:43.933686+0000","last_unstale":"2024-04-25T00:00:50.939852+0000","last_undegraded":"2024-04-25T00:00:50.939852+0000","last_fullsized":"2024-04-25T00:00:50.939852+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2564,"log_dups_size":0,"ondisk_log_size":2564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:26.261786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2b","version":"27'2598","reported_seq":2634,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.985866+0000","last_change":"2024-04-24T23:34:54.371052+0000","last_active":"2024-04-25T00:00:50.985866+0000","last_peered":"2024-04-25T00:00:50.985866+0000","last_clean":"2024-04-25T00:00:50.985866+0000","last_became_active":"2024-04-24T23:34:54.370793+0000","last_became_peered":"2024-04-24T23:34:54.370793+0000","last_unstale":"2024-04-25T00:00:50.985866+0000","last_undegraded":"2024-04-25T00:00:50.985866+0000","last_fullsized":"2024-04-25T00:00:50.985866+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2598,"log_dups_size":0,"ondisk_log_size":2598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"27'2595","reported_seq":2630,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.986378+0000","last_change":"2024-04-24T23:34:54.377190+0000","last_active":"2024-04-25T00:00:50.986378+0000","last_peered":"2024-04-25T00:00:50.986378+0000","last_clean":"2024-04-25T00:00:50.986378+0000","last_became_active":"2024-04-24T23:34:54.376940+0000","last_became_peered":"2024-04-24T23:34:54.376940+0000","last_unstale":"2024-04-25T00:00:50.986378+0000","last_undegraded":"2024-04-25T00:00:50.986378+0000","last_fullsized":"2024-04-25T00:00:50.986378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2595,"log_dups_size":0,"ondisk_log_size":2595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:03:24.745895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"27'2588","reported_seq":2622,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:51.037788+0000","last_change":"2024-04-24T23:34:54.368770+0000","last_active":"2024-04-25T00:00:51.037788+0000","last_peered":"2024-04-25T00:00:51.037788+0000","last_clean":"2024-04-25T00:00:51.037788+0000","last_became_active":"2024-04-24T23:34:54.368228+0000","last_became_peered":"2024-04-24T23:34:54.368228+0000","last_unstale":"2024-04-25T00:00:51.037788+0000","last_undegraded":"2024-04-25T00:00:51.037788+0000","last_fullsized":"2024-04-25T00:00:51.037788+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2588,"log_dups_size":0,"ondisk_log_size":2588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:51.634876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"27'2532","reported_seq":2567,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.907932+0000","last_change":"2024-04-24T23:34:54.373038+0000","last_active":"2024-04-25T00:00:50.907932+0000","last_peered":"2024-04-25T00:00:50.907932+0000","last_clean":"2024-04-25T00:00:50.907932+0000","last_became_active":"2024-04-24T23:34:54.372766+0000","last_became_peered":"2024-04-24T23:34:54.372766+0000","last_unstale":"2024-04-25T00:00:50.907932+0000","last_undegraded":"2024-04-25T00:00:50.907932+0000","last_fullsized":"2024-04-25T00:00:50.907932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2532,"log_dups_size":0,"ondisk_log_size":2532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:13.263529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"27'2601","reported_seq":2636,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:51.034415+0000","last_change":"2024-04-24T23:34:54.383733+0000","last_active":"2024-04-25T00:00:51.034415+0000","last_peered":"2024-04-25T00:00:51.034415+0000","last_clean":"2024-04-25T00:00:51.034415+0000","last_became_active":"2024-04-24T23:34:54.383463+0000","last_became_peered":"2024-04-24T23:34:54.383463+0000","last_unstale":"2024-04-25T00:00:51.034415+0000","last_undegraded":"2024-04-25T00:00:51.034415+0000","last_fullsized":"2024-04-25T00:00:51.034415+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2601,"log_dups_size":0,"ondisk_log_size":2601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:51.634876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"27'2529","reported_seq":2564,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:51.001633+0000","last_change":"2024-04-24T23:34:54.380959+0000","last_active":"2024-04-25T00:00:51.001633+0000","last_peered":"2024-04-25T00:00:51.001633+0000","last_clean":"2024-04-25T00:00:51.001633+0000","last_became_active":"2024-04-24T23:34:54.380702+0000","last_became_peered":"2024-04-24T23:34:54.380702+0000","last_unstale":"2024-04-25T00:00:51.001633+0000","last_undegraded":"2024-04-25T00:00:51.001633+0000","last_fullsized":"2024-04-25T00:00:51.001633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2529,"log_dups_size":0,"ondisk_log_size":2529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:45.424046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"27'2650","reported_seq":2686,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:51.025885+0000","last_change":"2024-04-24T23:34:54.358461+0000","last_active":"2024-04-25T00:00:51.025885+0000","last_peered":"2024-04-25T00:00:51.025885+0000","last_clean":"2024-04-25T00:00:51.025885+0000","last_became_active":"2024-04-24T23:34:54.358203+0000","last_became_peered":"2024-04-24T23:34:54.358203+0000","last_unstale":"2024-04-25T00:00:51.025885+0000","last_undegraded":"2024-04-25T00:00:51.025885+0000","last_fullsized":"2024-04-25T00:00:51.025885+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2650,"log_dups_size":0,"ondisk_log_size":2650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"27'2497","reported_seq":2533,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:51.012321+0000","last_change":"2024-04-24T23:34:54.367860+0000","last_active":"2024-04-25T00:00:51.012321+0000","last_peered":"2024-04-25T00:00:51.012321+0000","last_clean":"2024-04-25T00:00:51.012321+0000","last_became_active":"2024-04-24T23:34:54.367595+0000","last_became_peered":"2024-04-24T23:34:54.367595+0000","last_unstale":"2024-04-25T00:00:51.012321+0000","last_undegraded":"2024-04-25T00:00:51.012321+0000","last_fullsized":"2024-04-25T00:00:51.012321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2497,"log_dups_size":0,"ondisk_log_size":2497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"27'2502","reported_seq":2523,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.995692+0000","last_change":"2024-04-24T23:35:43.529991+0000","last_active":"2024-04-25T00:00:50.995692+0000","last_peered":"2024-04-25T00:00:50.995692+0000","last_clean":"2024-04-25T00:00:50.995692+0000","last_became_active":"2024-04-24T23:35:43.529631+0000","last_became_peered":"2024-04-24T23:35:43.529631+0000","last_unstale":"2024-04-25T00:00:50.995692+0000","last_undegraded":"2024-04-25T00:00:50.995692+0000","last_fullsized":"2024-04-25T00:00:50.995692+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2502,"log_dups_size":0,"ondisk_log_size":2502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.34","version":"27'2450","reported_seq":2484,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.997251+0000","last_change":"2024-04-24T23:34:54.377573+0000","last_active":"2024-04-25T00:00:50.997251+0000","last_peered":"2024-04-25T00:00:50.997251+0000","last_clean":"2024-04-25T00:00:50.997251+0000","last_became_active":"2024-04-24T23:34:54.377277+0000","last_became_peered":"2024-04-24T23:34:54.377277+0000","last_unstale":"2024-04-25T00:00:50.997251+0000","last_undegraded":"2024-04-25T00:00:50.997251+0000","last_fullsized":"2024-04-25T00:00:50.997251+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2450,"log_dups_size":0,"ondisk_log_size":2450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:43.211314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"27'2496","reported_seq":2530,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.983969+0000","last_change":"2024-04-24T23:34:54.388134+0000","last_active":"2024-04-25T00:00:50.983969+0000","last_peered":"2024-04-25T00:00:50.983969+0000","last_clean":"2024-04-25T00:00:50.983969+0000","last_became_active":"2024-04-24T23:34:54.387781+0000","last_became_peered":"2024-04-24T23:34:54.387781+0000","last_unstale":"2024-04-25T00:00:50.983969+0000","last_undegraded":"2024-04-25T00:00:50.983969+0000","last_fullsized":"2024-04-25T00:00:50.983969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2496,"log_dups_size":0,"ondisk_log_size":2496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:56:17.763159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"27'2613","reported_seq":2647,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:51.008155+0000","last_change":"2024-04-24T23:34:54.388182+0000","last_active":"2024-04-25T00:00:51.008155+0000","last_peered":"2024-04-25T00:00:51.008155+0000","last_clean":"2024-04-25T00:00:51.008155+0000","last_became_active":"2024-04-24T23:34:54.387913+0000","last_became_peered":"2024-04-24T23:34:54.387913+0000","last_unstale":"2024-04-25T00:00:51.008155+0000","last_undegraded":"2024-04-25T00:00:51.008155+0000","last_fullsized":"2024-04-25T00:00:51.008155+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2613,"log_dups_size":0,"ondisk_log_size":2613,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:18.207267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"27'2583","reported_seq":2616,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:51.038456+0000","last_change":"2024-04-24T23:34:54.383303+0000","last_active":"2024-04-25T00:00:51.038456+0000","last_peered":"2024-04-25T00:00:51.038456+0000","last_clean":"2024-04-25T00:00:51.038456+0000","last_became_active":"2024-04-24T23:34:54.383024+0000","last_became_peered":"2024-04-24T23:34:54.383024+0000","last_unstale":"2024-04-25T00:00:51.038456+0000","last_undegraded":"2024-04-25T00:00:51.038456+0000","last_fullsized":"2024-04-25T00:00:51.038456+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2583,"log_dups_size":0,"ondisk_log_size":2583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:45.424046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"27'2526","reported_seq":2561,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:51.022053+0000","last_change":"2024-04-24T23:34:54.382803+0000","last_active":"2024-04-25T00:00:51.022053+0000","last_peered":"2024-04-25T00:00:51.022053+0000","last_clean":"2024-04-25T00:00:51.022053+0000","last_became_active":"2024-04-24T23:34:54.382482+0000","last_became_peered":"2024-04-24T23:34:54.382482+0000","last_unstale":"2024-04-25T00:00:51.022053+0000","last_undegraded":"2024-04-25T00:00:51.022053+0000","last_fullsized":"2024-04-25T00:00:51.022053+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2526,"log_dups_size":0,"ondisk_log_size":2526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"27'2536","reported_seq":2569,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.969085+0000","last_change":"2024-04-24T23:34:54.369167+0000","last_active":"2024-04-25T00:00:50.969085+0000","last_peered":"2024-04-25T00:00:50.969085+0000","last_clean":"2024-04-25T00:00:50.969085+0000","last_became_active":"2024-04-24T23:34:54.368663+0000","last_became_peered":"2024-04-24T23:34:54.368663+0000","last_unstale":"2024-04-25T00:00:50.969085+0000","last_undegraded":"2024-04-25T00:00:50.969085+0000","last_fullsized":"2024-04-25T00:00:50.969085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2536,"log_dups_size":0,"ondisk_log_size":2536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:18.207267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"27'2479","reported_seq":2513,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:51.008160+0000","last_change":"2024-04-24T23:34:54.380345+0000","last_active":"2024-04-25T00:00:51.008160+0000","last_peered":"2024-04-25T00:00:51.008160+0000","last_clean":"2024-04-25T00:00:51.008160+0000","last_became_active":"2024-04-24T23:34:54.380052+0000","last_became_peered":"2024-04-24T23:34:54.380052+0000","last_unstale":"2024-04-25T00:00:51.008160+0000","last_undegraded":"2024-04-25T00:00:51.008160+0000","last_fullsized":"2024-04-25T00:00:51.008160+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2479,"log_dups_size":0,"ondisk_log_size":2479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:42.199856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"27'2516","reported_seq":2549,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.970992+0000","last_change":"2024-04-24T23:34:54.365114+0000","last_active":"2024-04-25T00:00:50.970992+0000","last_peered":"2024-04-25T00:00:50.970992+0000","last_clean":"2024-04-25T00:00:50.970992+0000","last_became_active":"2024-04-24T23:34:54.364605+0000","last_became_peered":"2024-04-24T23:34:54.364605+0000","last_unstale":"2024-04-25T00:00:50.970992+0000","last_undegraded":"2024-04-25T00:00:50.970992+0000","last_fullsized":"2024-04-25T00:00:50.970992+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2516,"log_dups_size":0,"ondisk_log_size":2516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:07:28.785202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"27'2568","reported_seq":2590,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.968500+0000","last_change":"2024-04-24T23:35:43.525630+0000","last_active":"2024-04-25T00:00:50.968500+0000","last_peered":"2024-04-25T00:00:50.968500+0000","last_clean":"2024-04-25T00:00:50.968500+0000","last_became_active":"2024-04-24T23:35:43.525191+0000","last_became_peered":"2024-04-24T23:35:43.525191+0000","last_unstale":"2024-04-25T00:00:50.968500+0000","last_undegraded":"2024-04-25T00:00:50.968500+0000","last_fullsized":"2024-04-25T00:00:50.968500+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2568,"log_dups_size":0,"ondisk_log_size":2568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:07:28.785202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"27'2637","reported_seq":2674,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.989028+0000","last_change":"2024-04-24T23:34:54.366079+0000","last_active":"2024-04-25T00:00:50.989028+0000","last_peered":"2024-04-25T00:00:50.989028+0000","last_clean":"2024-04-25T00:00:50.989028+0000","last_became_active":"2024-04-24T23:34:54.365814+0000","last_became_peered":"2024-04-24T23:34:54.365814+0000","last_unstale":"2024-04-25T00:00:50.989028+0000","last_undegraded":"2024-04-25T00:00:50.989028+0000","last_fullsized":"2024-04-25T00:00:50.989028+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2637,"log_dups_size":0,"ondisk_log_size":2637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"27'2476","reported_seq":2513,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.981337+0000","last_change":"2024-04-24T23:34:54.368976+0000","last_active":"2024-04-25T00:00:50.981337+0000","last_peered":"2024-04-25T00:00:50.981337+0000","last_clean":"2024-04-25T00:00:50.981337+0000","last_became_active":"2024-04-24T23:34:54.368663+0000","last_became_peered":"2024-04-24T23:34:54.368663+0000","last_unstale":"2024-04-25T00:00:50.981337+0000","last_undegraded":"2024-04-25T00:00:50.981337+0000","last_fullsized":"2024-04-25T00:00:50.981337+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2476,"log_dups_size":0,"ondisk_log_size":2476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"27'2458","reported_seq":2493,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:51.038647+0000","last_change":"2024-04-24T23:34:54.373810+0000","last_active":"2024-04-25T00:00:51.038647+0000","last_peered":"2024-04-25T00:00:51.038647+0000","last_clean":"2024-04-25T00:00:51.038647+0000","last_became_active":"2024-04-24T23:34:54.373531+0000","last_became_peered":"2024-04-24T23:34:54.373531+0000","last_unstale":"2024-04-25T00:00:51.038647+0000","last_undegraded":"2024-04-25T00:00:51.038647+0000","last_fullsized":"2024-04-25T00:00:51.038647+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2458,"log_dups_size":0,"ondisk_log_size":2458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:50:17.621315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12310,"num_read_kb":3126740,"num_write":212524,"num_write_kb":30664179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":230531,"ondisk_log_size":230531,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1120,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":162975,"ondisk_log_size":162975,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":236421422,"num_objects":79,"num_object_clones":0,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":0,"num_read":3152,"num_read_kb":19253,"num_write":210114,"num_write_kb":1682543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":287891456,"data_stored":472842844,"data_compressed":55745062,"data_compressed_allocated":81436672,"data_compressed_original":266452992,"omap_allocated":0,"internal_metadata":0},"log_size":195662,"ondisk_log_size":195662,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65248,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542459,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":961884,"kb_used_data":28436,"kb_used_omap":1,"kb_used_meta":933438,"kb_avail":93409956,"statfs":{"total":96636764160,"available":95651794944,"internally_reserved":0,"allocated":29118464,"data_stored":47674208,"data_compressed":5564865,"data_compressed_allocated":8151040,"data_compressed_original":26886144,"omap_allocated":1590,"internal_metadata":955840970},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,0,22,3,60],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542457,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1126144,"kb_used_data":40056,"kb_used_omap":1,"kb_used_meta":1086078,"kb_avail":93245696,"statfs":{"total":96636764160,"available":95483592704,"internally_reserved":0,"allocated":41017344,"data_stored":68055450,"data_compressed":8055094,"data_compressed_allocated":11661312,"data_compressed_original":38879232,"omap_allocated":1594,"internal_metadata":1112144326},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542458,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":980308,"kb_used_data":24716,"kb_used_omap":1,"kb_used_meta":955582,"kb_avail":93391532,"statfs":{"total":96636764160,"available":95632928768,"internally_reserved":0,"allocated":25309184,"data_stored":42060410,"data_compressed":5058379,"data_compressed_allocated":7299072,"data_compressed_original":24223744,"omap_allocated":1595,"internal_metadata":978516421},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542457,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":920748,"kb_used_data":41380,"kb_used_omap":1,"kb_used_meta":879358,"kb_avail":93451092,"statfs":{"total":96636764160,"available":95693918208,"internally_reserved":0,"allocated":42373120,"data_stored":70089374,"data_compressed":7916354,"data_compressed_allocated":11673600,"data_compressed_original":39571456,"omap_allocated":1595,"internal_metadata":900463045},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542459,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":660908,"kb_used_data":34148,"kb_used_omap":1,"kb_used_meta":626750,"kb_avail":93710932,"statfs":{"total":96636764160,"available":95959994368,"internally_reserved":0,"allocated":34967552,"data_stored":54643232,"data_compressed":6190216,"data_compressed_allocated":9224192,"data_compressed_original":29069312,"omap_allocated":1595,"internal_metadata":641792453},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542459,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1239756,"kb_used_data":39748,"kb_used_omap":1,"kb_used_meta":1199998,"kb_avail":93132084,"statfs":{"total":96636764160,"available":95367254016,"internally_reserved":0,"allocated":40701952,"data_stored":63742294,"data_compressed":7360432,"data_compressed_allocated":10985472,"data_compressed_original":34205696,"omap_allocated":1595,"internal_metadata":1228798405},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542459,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":827048,"kb_used_data":33824,"kb_used_omap":1,"kb_used_meta":793214,"kb_avail":93544792,"statfs":{"total":96636764160,"available":95789867008,"internally_reserved":0,"allocated":34635776,"data_stored":60138836,"data_compressed":7346778,"data_compressed_allocated":10571776,"data_compressed_original":36261888,"omap_allocated":1595,"internal_metadata":812251589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575164,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1145812,"kb_used_data":41612,"kb_used_omap":1,"kb_used_meta":1104190,"kb_avail":93226028,"statfs":{"total":96636764160,"available":95463452672,"internally_reserved":0,"allocated":42610688,"data_stored":71421120,"data_compressed":8449066,"data_compressed_allocated":12222464,"data_compressed_original":41209856,"omap_allocated":1593,"internal_metadata":1130691015},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32624,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32624,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":34283520,"data_stored":59431188,"data_compressed":7314154,"data_compressed_allocated":10526720,"data_compressed_original":35688448,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40415232,"data_stored":63625014,"data_compressed":7360432,"data_compressed_allocated":10985472,"data_compressed_original":34205696,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":34680832,"data_stored":54525952,"data_compressed":6190216,"data_compressed_allocated":9224192,"data_compressed_original":29069312,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":42323968,"data_stored":71303840,"data_compressed":8449066,"data_compressed_allocated":12222464,"data_compressed_original":41209856,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":41668608,"data_stored":67108990,"data_compressed":7785480,"data_compressed_allocated":11411456,"data_compressed_original":36864000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":25022464,"data_stored":41943130,"data_compressed":5058379,"data_compressed_allocated":7299072,"data_compressed_original":24223744,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40730624,"data_stored":67938170,"data_compressed":8055094,"data_compressed_allocated":11661312,"data_compressed_original":38879232,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28766208,"data_stored":46966560,"data_compressed":5532241,"data_compressed_allocated":8105984,"data_compressed_original":26312704,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:00:57.678 INFO:tasks.ceph:pgid 2.1f last_scrub_stamp 2024-04-24T23:34:52.333566+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=52, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.678 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.679 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-04-24T23:34:57.376109+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=57, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.679 INFO:tasks.ceph:pgid 2.1e last_scrub_stamp 2024-04-24T23:34:52.333566+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=52, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.679 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.679 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2024-04-24T23:34:57.376109+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=57, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.679 INFO:tasks.ceph:pgid 2.1d last_scrub_stamp 2024-04-24T23:34:52.333566+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=52, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.679 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.679 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2024-04-24T23:34:57.376109+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=57, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.680 INFO:tasks.ceph:pgid 2.1c last_scrub_stamp 2024-04-24T23:34:52.333566+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=52, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.680 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.680 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2024-04-24T23:34:57.376109+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=57, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.680 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.680 INFO:tasks.ceph:pgid 2.b last_scrub_stamp 2024-04-24T23:34:52.333566+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=52, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.680 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2024-04-24T23:34:57.376109+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=57, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.681 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.681 INFO:tasks.ceph:pgid 2.a last_scrub_stamp 2024-04-24T23:34:52.333566+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=52, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.681 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2024-04-24T23:34:57.376109+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=57, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.681 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.681 INFO:tasks.ceph:pgid 2.9 last_scrub_stamp 2024-04-24T23:34:52.333566+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=52, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.681 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2024-04-24T23:34:57.376109+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=57, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.681 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.681 INFO:tasks.ceph:pgid 2.8 last_scrub_stamp 2024-04-24T23:34:52.333566+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=52, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.681 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2024-04-24T23:34:57.376109+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=57, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.682 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.682 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-04-24T23:34:52.333566+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=52, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.682 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2024-04-24T23:34:57.376109+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=57, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.682 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.682 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-04-24T23:34:52.333566+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=52, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.682 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2024-04-24T23:34:57.376109+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=57, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.682 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.682 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-04-24T23:34:52.333566+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=52, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.682 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2024-04-24T23:34:57.376109+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=57, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.683 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.683 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-04-24T23:34:52.333566+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=52, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.683 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2024-04-24T23:34:57.376109+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=57, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.683 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.683 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-04-24T23:34:52.333566+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=52, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.684 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2024-04-24T23:34:57.376109+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=57, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.684 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.684 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-04-24T23:34:52.333566+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=52, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.684 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2024-04-24T23:34:57.376109+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=57, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.684 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.684 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-04-24T23:34:52.333566+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=52, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.685 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2024-04-24T23:34:57.376109+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=57, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.685 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.685 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-04-24T23:34:52.333566+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=52, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.685 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-24T23:34:48.587078+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=48, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.685 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2024-04-24T23:34:57.376109+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=57, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.685 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.685 INFO:tasks.ceph:pgid 2.c last_scrub_stamp 2024-04-24T23:34:52.333566+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=52, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.685 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2024-04-24T23:34:57.376109+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=57, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.686 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.686 INFO:tasks.ceph:pgid 2.d last_scrub_stamp 2024-04-24T23:34:52.333566+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=52, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.686 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2024-04-24T23:34:57.376109+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=57, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.686 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.686 INFO:tasks.ceph:pgid 2.e last_scrub_stamp 2024-04-24T23:34:52.333566+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=52, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.686 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2024-04-24T23:34:57.376109+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=57, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.686 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.687 INFO:tasks.ceph:pgid 2.f last_scrub_stamp 2024-04-24T23:34:52.333566+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=52, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.687 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2024-04-24T23:34:57.376109+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=57, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.687 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.687 INFO:tasks.ceph:pgid 2.10 last_scrub_stamp 2024-04-24T23:34:52.333566+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=52, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.687 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-04-24T23:34:57.376109+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=57, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.688 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.688 INFO:tasks.ceph:pgid 2.11 last_scrub_stamp 2024-04-24T23:34:52.333566+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=52, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.688 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2024-04-24T23:34:57.376109+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=57, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.688 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.688 INFO:tasks.ceph:pgid 2.12 last_scrub_stamp 2024-04-24T23:34:52.333566+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=52, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.688 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2024-04-24T23:34:57.376109+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=57, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.688 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.688 INFO:tasks.ceph:pgid 2.13 last_scrub_stamp 2024-04-24T23:34:52.333566+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=52, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.688 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2024-04-24T23:34:57.376109+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=57, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.688 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.689 INFO:tasks.ceph:pgid 2.14 last_scrub_stamp 2024-04-24T23:34:52.333566+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=52, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.689 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2024-04-24T23:34:57.376109+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=57, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.689 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.689 INFO:tasks.ceph:pgid 2.15 last_scrub_stamp 2024-04-24T23:34:52.333566+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=52, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.689 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2024-04-24T23:34:57.376109+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=57, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.689 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.689 INFO:tasks.ceph:pgid 2.16 last_scrub_stamp 2024-04-24T23:34:52.333566+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=52, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.689 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2024-04-24T23:34:57.376109+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=57, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.689 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.690 INFO:tasks.ceph:pgid 2.17 last_scrub_stamp 2024-04-24T23:34:52.333566+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=52, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.690 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2024-04-24T23:34:57.376109+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=57, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.690 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2024-04-24T23:34:57.376109+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=57, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.690 INFO:tasks.ceph:pgid 2.18 last_scrub_stamp 2024-04-24T23:34:52.333566+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=52, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.690 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2024-04-24T23:34:57.376109+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=57, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.690 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.691 INFO:tasks.ceph:pgid 2.19 last_scrub_stamp 2024-04-24T23:34:52.333566+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=52, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.691 INFO:tasks.ceph:pgid 2.1a last_scrub_stamp 2024-04-24T23:34:52.333566+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=52, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.691 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2024-04-24T23:34:57.376109+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=57, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.691 INFO:tasks.ceph:pgid 2.1b last_scrub_stamp 2024-04-24T23:34:52.333566+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=52, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.691 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.691 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2024-04-24T23:34:57.376109+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=57, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.691 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.691 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.692 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.692 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.692 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.692 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.692 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.692 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.692 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.692 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.692 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.692 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.693 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.693 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.693 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.693 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.693 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.693 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.693 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.693 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.693 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.694 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.694 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.694 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.694 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.694 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.694 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.694 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.694 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.695 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.695 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.695 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:00:57.695 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-25T00:01:17.697 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:01:17.801 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.798+0000 7f66d6bfb640 1 Processor -- start 2024-04-25T00:01:17.801 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.798+0000 7f66d6bfb640 1 -- start start 2024-04-25T00:01:17.801 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.798+0000 7f66d6bfb640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f66d0073130 0x7f66d00c2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:17.802 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.798+0000 7f66d6bfb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f66d00c4b20 0x7f66d006ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:17.802 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.798+0000 7f66d6bfb640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66d014b720 0x7f66d014bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:17.802 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.798+0000 7f66d6bfb640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f66d006c400 con 0x7f66d00c4b20 2024-04-25T00:01:17.802 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.798+0000 7f66d6bfb640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f66d0073510 con 0x7f66d0073130 2024-04-25T00:01:17.802 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.798+0000 7f66d6bfb640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f66d006c6a0 con 0x7f66d014b720 2024-04-25T00:01:17.802 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.798+0000 7f66cffff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f66d00c4b20 0x7f66d006ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:17.802 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.798+0000 7f66cffff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f66d00c4b20 0x7f66d006ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:53736/0 (socket says 172.21.15.157:53736) 2024-04-25T00:01:17.802 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.798+0000 7f66cffff640 1 -- 172.21.15.157:0/1242495731 learned_addr learned my addr 172.21.15.157:0/1242495731 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:01:17.802 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.799+0000 7f66d4997640 1 --2- 172.21.15.157:0/1242495731 >> v2:172.21.15.169:3300/0 conn(0x7f66d0073130 0x7f66d00c2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:17.802 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.799+0000 7f66d5198640 1 --2- 172.21.15.157:0/1242495731 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66d014b720 0x7f66d014bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:17.802 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.799+0000 7f66cffff640 1 -- 172.21.15.157:0/1242495731 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66d014b720 msgr2=0x7f66d014bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:17.802 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.799+0000 7f66cffff640 1 --2- 172.21.15.157:0/1242495731 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66d014b720 0x7f66d014bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.803 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.799+0000 7f66cffff640 1 -- 172.21.15.157:0/1242495731 >> v2:172.21.15.169:3300/0 conn(0x7f66d0073130 msgr2=0x7f66d00c2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:17.803 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.799+0000 7f66cffff640 1 --2- 172.21.15.157:0/1242495731 >> v2:172.21.15.169:3300/0 conn(0x7f66d0073130 0x7f66d00c2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.803 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.799+0000 7f66cffff640 1 -- 172.21.15.157:0/1242495731 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66d00c1eb0 con 0x7f66d00c4b20 2024-04-25T00:01:17.803 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.799+0000 7f66cffff640 1 --2- 172.21.15.157:0/1242495731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66d00c4b20 0x7f66d006ccb0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f66d00a93c0 tx=0x7f66bc02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d617bff90a1d5512 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:17.803 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.800+0000 7f66cf7fe640 1 -- 172.21.15.157:0/1242495731 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66bc037070 con 0x7f66d00c4b20 2024-04-25T00:01:17.803 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.800+0000 7f66cf7fe640 1 -- 172.21.15.157:0/1242495731 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66bc032440 con 0x7f66d00c4b20 2024-04-25T00:01:17.803 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.800+0000 7f66cf7fe640 1 -- 172.21.15.157:0/1242495731 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66bc032810 con 0x7f66d00c4b20 2024-04-25T00:01:17.803 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.800+0000 7f66d6bfb640 1 -- 172.21.15.157:0/1242495731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66d00c4b20 msgr2=0x7f66d006ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:17.803 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.800+0000 7f66d6bfb640 1 --2- 172.21.15.157:0/1242495731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66d00c4b20 0x7f66d006ccb0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f66d00a93c0 tx=0x7f66bc02a140 comp rx=0 tx=0).stop 2024-04-25T00:01:17.803 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.800+0000 7f66d6bfb640 1 -- 172.21.15.157:0/1242495731 shutdown_connections 2024-04-25T00:01:17.803 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.800+0000 7f66d6bfb640 1 --2- 172.21.15.157:0/1242495731 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66d014b720 0x7f66d014bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.804 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.800+0000 7f66d6bfb640 1 --2- 172.21.15.157:0/1242495731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66d00c4b20 0x7f66d006ccb0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.804 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.800+0000 7f66d6bfb640 1 --2- 172.21.15.157:0/1242495731 >> v2:172.21.15.169:3300/0 conn(0x7f66d0073130 0x7f66d00c2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.804 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.800+0000 7f66d6bfb640 1 -- 172.21.15.157:0/1242495731 >> 172.21.15.157:0/1242495731 conn(0x7f66d00a8720 msgr2=0x7f66d00a8b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:17.804 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.800+0000 7f66d6bfb640 1 -- 172.21.15.157:0/1242495731 shutdown_connections 2024-04-25T00:01:17.804 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.800+0000 7f66d6bfb640 1 -- 172.21.15.157:0/1242495731 wait complete. 2024-04-25T00:01:17.807 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.801+0000 7f66d6bfb640 1 Processor -- start 2024-04-25T00:01:17.807 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.801+0000 7f66d6bfb640 1 -- start start 2024-04-25T00:01:17.807 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.802+0000 7f66d6bfb640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66d0073130 0x7f66d00bfda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:17.807 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.802+0000 7f66d6bfb640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66d00c4b20 0x7f66d014a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:17.807 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.802+0000 7f66d6bfb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66d014b720 0x7f66d01514c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:17.807 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.802+0000 7f66d6bfb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f66d006d1f0 con 0x7f66d014b720 2024-04-25T00:01:17.807 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.802+0000 7f66d6bfb640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f66d00c2470 con 0x7f66d00c4b20 2024-04-25T00:01:17.807 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.802+0000 7f66d6bfb640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f66d006d370 con 0x7f66d0073130 2024-04-25T00:01:17.807 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.802+0000 7f66d5198640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66d014b720 0x7f66d01514c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:17.808 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.802+0000 7f66d5198640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66d014b720 0x7f66d01514c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:53738/0 (socket says 172.21.15.157:53738) 2024-04-25T00:01:17.808 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.802+0000 7f66cffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66d00c4b20 0x7f66d014a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:17.808 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.802+0000 7f66d5198640 1 -- 172.21.15.157:0/3992110008 learned_addr learned my addr 172.21.15.157:0/3992110008 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:01:17.808 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.802+0000 7f66d4997640 1 --2- 172.21.15.157:0/3992110008 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66d0073130 0x7f66d00bfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:17.808 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.803+0000 7f66d5198640 1 -- 172.21.15.157:0/3992110008 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66d0073130 msgr2=0x7f66d00bfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:17.808 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.803+0000 7f66d5198640 1 --2- 172.21.15.157:0/3992110008 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66d0073130 0x7f66d00bfda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.808 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.803+0000 7f66d5198640 1 -- 172.21.15.157:0/3992110008 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66d00c4b20 msgr2=0x7f66d014a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:17.808 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.803+0000 7f66d5198640 1 --2- 172.21.15.157:0/3992110008 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66d00c4b20 0x7f66d014a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.808 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.803+0000 7f66d5198640 1 -- 172.21.15.157:0/3992110008 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66d01ef740 con 0x7f66d014b720 2024-04-25T00:01:17.808 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.803+0000 7f66d4997640 1 --2- 172.21.15.157:0/3992110008 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66d0073130 0x7f66d00bfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:01:17.808 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.803+0000 7f66cffff640 1 --2- 172.21.15.157:0/3992110008 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66d00c4b20 0x7f66d014a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:01:17.808 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.803+0000 7f66d5198640 1 --2- 172.21.15.157:0/3992110008 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66d014b720 0x7f66d01514c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f66c800efc0 tx=0x7f66c800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:17.808 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.803+0000 7f66cd7fa640 1 -- 172.21.15.157:0/3992110008 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c8018020 con 0x7f66d014b720 2024-04-25T00:01:17.809 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.803+0000 7f66cd7fa640 1 -- 172.21.15.157:0/3992110008 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66c8014480 con 0x7f66d014b720 2024-04-25T00:01:17.809 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.803+0000 7f66cd7fa640 1 -- 172.21.15.157:0/3992110008 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c8014870 con 0x7f66d014b720 2024-04-25T00:01:17.809 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.803+0000 7f66d6bfb640 1 -- 172.21.15.157:0/3992110008 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66d01eb710 con 0x7f66d014b720 2024-04-25T00:01:17.809 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.803+0000 7f66d6bfb640 1 -- 172.21.15.157:0/3992110008 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66d01ebcd0 con 0x7f66d014b720 2024-04-25T00:01:17.809 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.804+0000 7f66d6bfb640 1 -- 172.21.15.157:0/3992110008 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66d00c2990 con 0x7f66d014b720 2024-04-25T00:01:17.809 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.805+0000 7f66cd7fa640 1 -- 172.21.15.157:0/3992110008 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f66c8005d10 con 0x7f66d014b720 2024-04-25T00:01:17.809 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.805+0000 7f66cd7fa640 1 --2- 172.21.15.157:0/3992110008 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f66b406be30 0x7f66b406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:17.809 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.805+0000 7f66cd7fa640 1 -- 172.21.15.157:0/3992110008 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f66c808be90 con 0x7f66d014b720 2024-04-25T00:01:17.809 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.805+0000 7f66d4997640 1 --2- 172.21.15.157:0/3992110008 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f66b406be30 0x7f66b406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:17.809 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.806+0000 7f66d4997640 1 --2- 172.21.15.157:0/3992110008 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f66b406be30 0x7f66b406e2f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f66c4009e90 tx=0x7f66c4009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:17.809 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.809+0000 7f66cd7fa640 1 -- 172.21.15.157:0/3992110008 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f66d00c2990 con 0x7f66d014b720 2024-04-25T00:01:17.983 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.982+0000 7f66d6bfb640 1 -- 172.21.15.157:0/3992110008 --> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f66d01ef4f0 con 0x7f66b406be30 2024-04-25T00:01:17.991 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.990+0000 7f66cd7fa640 1 -- 172.21.15.157:0/3992110008 <== mgr.4105 v2:172.21.15.169:6832/662378289 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+337071 (secure 0 0 0) 0x7f66d01ef4f0 con 0x7f66b406be30 2024-04-25T00:01:17.995 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-25T00:01:17.997 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2024-04-25T00:01:17.998 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.994+0000 7f66d6bfb640 1 -- 172.21.15.157:0/3992110008 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f66b406be30 msgr2=0x7f66b406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:17.999 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.994+0000 7f66d6bfb640 1 --2- 172.21.15.157:0/3992110008 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f66b406be30 0x7f66b406e2f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f66c4009e90 tx=0x7f66c4009190 comp rx=0 tx=0).stop 2024-04-25T00:01:17.999 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.994+0000 7f66d6bfb640 1 -- 172.21.15.157:0/3992110008 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66d014b720 msgr2=0x7f66d01514c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:17.999 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.994+0000 7f66d6bfb640 1 --2- 172.21.15.157:0/3992110008 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66d014b720 0x7f66d01514c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f66c800efc0 tx=0x7f66c800c490 comp rx=0 tx=0).stop 2024-04-25T00:01:17.999 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.994+0000 7f66d6bfb640 1 -- 172.21.15.157:0/3992110008 shutdown_connections 2024-04-25T00:01:17.999 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.994+0000 7f66d6bfb640 1 --2- 172.21.15.157:0/3992110008 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f66b406be30 0x7f66b406e2f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.999 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.994+0000 7f66d6bfb640 1 --2- 172.21.15.157:0/3992110008 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66d014b720 0x7f66d01514c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.999 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.994+0000 7f66d6bfb640 1 --2- 172.21.15.157:0/3992110008 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66d00c4b20 0x7f66d014a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.999 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.994+0000 7f66d6bfb640 1 --2- 172.21.15.157:0/3992110008 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66d0073130 0x7f66d00bfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.999 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.994+0000 7f66d6bfb640 1 -- 172.21.15.157:0/3992110008 >> 172.21.15.157:0/3992110008 conn(0x7f66d00a8720 msgr2=0x7f66d00a8b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:17.999 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.994+0000 7f66d6bfb640 1 -- 172.21.15.157:0/3992110008 shutdown_connections 2024-04-25T00:01:17.999 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:17.995+0000 7f66d6bfb640 1 -- 172.21.15.157:0/3992110008 wait complete. 2024-04-25T00:01:18.006 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":815,"stamp":"2024-04-25T00:01:16.178165+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":237011790,"num_objects":81,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":0,"num_read":15578,"num_read_kb":3146093,"num_write":423917,"num_write_kb":32349138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5807,"num_omap_keys":12,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":589356,"ondisk_log_size":589356,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":7866360,"kb_used_data":283512,"kb_used_omap":12,"kb_used_meta":7582771,"kb_avail":747108360,"statfs":{"total":773094113280,"available":765038960640,"internally_reserved":0,"allocated":290316288,"data_stored":474961820,"data_compressed":55810310,"data_compressed_allocated":81526784,"data_compressed_original":267599872,"omap_allocated":12752,"internal_metadata":7764758064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":246,"num_omap_keys":1,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002457"},"pg_stats":[{"pgid":"2.1f","version":"27'2680","reported_seq":2727,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:57.060012+0000","last_change":"2024-04-25T00:00:57.060042+0000","last_active":"2024-04-25T00:00:57.060012+0000","last_peered":"2024-04-25T00:00:57.060012+0000","last_clean":"2024-04-25T00:00:57.060012+0000","last_became_active":"2024-04-24T23:34:53.362104+0000","last_became_peered":"2024-04-24T23:34:53.362104+0000","last_unstale":"2024-04-25T00:00:57.060012+0000","last_undegraded":"2024-04-25T00:00:57.060012+0000","last_fullsized":"2024-04-25T00:00:57.060012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2680","last_scrub_stamp":"2024-04-25T00:00:57.059949+0000","last_deep_scrub":"27'2680","last_deep_scrub_stamp":"2024-04-25T00:00:57.059949+0000","last_clean_scrub_stamp":"2024-04-25T00:00:57.059949+0000","objects_scrubbed":2,"log_size":2680,"log_dups_size":0,"ondisk_log_size":2680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:38:10.304560+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":2687,"num_write_kb":18218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":561,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"27'2551","reported_seq":2597,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:56.270864+0000","last_change":"2024-04-25T00:00:56.270864+0000","last_active":"2024-04-25T00:00:56.270864+0000","last_peered":"2024-04-25T00:00:56.270864+0000","last_clean":"2024-04-25T00:00:56.270864+0000","last_became_active":"2024-04-24T23:34:54.373289+0000","last_became_peered":"2024-04-24T23:34:54.373289+0000","last_unstale":"2024-04-25T00:00:56.270864+0000","last_undegraded":"2024-04-25T00:00:56.270864+0000","last_fullsized":"2024-04-25T00:00:56.270864+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2551","last_scrub_stamp":"2024-04-25T00:00:56.270712+0000","last_deep_scrub":"27'2551","last_deep_scrub_stamp":"2024-04-25T00:00:56.270712+0000","last_clean_scrub_stamp":"2024-04-25T00:00:56.270712+0000","objects_scrubbed":0,"log_size":2551,"log_dups_size":0,"ondisk_log_size":2551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:16.906001+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"27'9462","reported_seq":9818,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:15.374476+0000","last_change":"2024-04-25T00:01:15.374522+0000","last_active":"2024-04-25T00:01:15.374476+0000","last_peered":"2024-04-25T00:01:15.374476+0000","last_clean":"2024-04-25T00:01:15.374476+0000","last_became_active":"2024-04-24T23:34:58.729816+0000","last_became_peered":"2024-04-24T23:34:58.729816+0000","last_unstale":"2024-04-25T00:01:15.374476+0000","last_undegraded":"2024-04-25T00:01:15.374476+0000","last_fullsized":"2024-04-25T00:01:15.374476+0000","mapping_epoch":23,"log_start":"27'3200","ondisk_log_start":"27'3200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9462","last_scrub_stamp":"2024-04-25T00:01:15.374342+0000","last_deep_scrub":"27'9462","last_deep_scrub_stamp":"2024-04-25T00:01:15.374342+0000","last_clean_scrub_stamp":"2024-04-25T00:01:15.374342+0000","objects_scrubbed":0,"log_size":6262,"log_dups_size":0,"ondisk_log_size":6262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:53:20.396466+0000","scrub_duration":41,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":319,"num_read_kb":81026,"num_write":6761,"num_write_kb":1031213,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"27'6592","reported_seq":6640,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:00.297107+0000","last_change":"2024-04-25T00:01:00.297107+0000","last_active":"2024-04-25T00:01:00.297107+0000","last_peered":"2024-04-25T00:01:00.297107+0000","last_clean":"2024-04-25T00:01:00.297107+0000","last_became_active":"2024-04-24T23:34:53.361366+0000","last_became_peered":"2024-04-24T23:34:53.361366+0000","last_unstale":"2024-04-25T00:01:00.297107+0000","last_undegraded":"2024-04-25T00:01:00.297107+0000","last_fullsized":"2024-04-25T00:01:00.297107+0000","mapping_epoch":18,"log_start":"27'300","ondisk_log_start":"27'300","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6592","last_scrub_stamp":"2024-04-25T00:01:00.296977+0000","last_deep_scrub":"27'6592","last_deep_scrub_stamp":"2024-04-25T00:01:00.296977+0000","last_clean_scrub_stamp":"2024-04-25T00:01:00.296977+0000","objects_scrubbed":1,"log_size":6292,"log_dups_size":0,"ondisk_log_size":6292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:01.320608+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6593,"num_write_kb":53610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"27'2529","reported_seq":2574,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:05.990749+0000","last_change":"2024-04-25T00:01:05.990807+0000","last_active":"2024-04-25T00:01:05.990749+0000","last_peered":"2024-04-25T00:01:05.990749+0000","last_clean":"2024-04-25T00:01:05.990749+0000","last_became_active":"2024-04-24T23:34:54.383160+0000","last_became_peered":"2024-04-24T23:34:54.383160+0000","last_unstale":"2024-04-25T00:01:05.990749+0000","last_undegraded":"2024-04-25T00:01:05.990749+0000","last_fullsized":"2024-04-25T00:01:05.990749+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2529","last_scrub_stamp":"2024-04-25T00:01:05.990668+0000","last_deep_scrub":"27'2529","last_deep_scrub_stamp":"2024-04-25T00:01:05.990668+0000","last_clean_scrub_stamp":"2024-04-25T00:01:05.990668+0000","objects_scrubbed":0,"log_size":2529,"log_dups_size":0,"ondisk_log_size":2529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:26:32.280888+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"27'9228","reported_seq":9434,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:08.713941+0000","last_change":"2024-04-25T00:01:08.713983+0000","last_active":"2024-04-25T00:01:08.713941+0000","last_peered":"2024-04-25T00:01:08.713941+0000","last_clean":"2024-04-25T00:01:08.713941+0000","last_became_active":"2024-04-24T23:34:58.403984+0000","last_became_peered":"2024-04-24T23:34:58.403984+0000","last_unstale":"2024-04-25T00:01:08.713941+0000","last_undegraded":"2024-04-25T00:01:08.713941+0000","last_fullsized":"2024-04-25T00:01:08.713941+0000","mapping_epoch":23,"log_start":"27'2060","ondisk_log_start":"27'2060","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9228","last_scrub_stamp":"2024-04-25T00:01:08.713806+0000","last_deep_scrub":"27'9228","last_deep_scrub_stamp":"2024-04-25T00:01:08.713806+0000","last_clean_scrub_stamp":"2024-04-25T00:01:08.713806+0000","objects_scrubbed":0,"log_size":7168,"log_dups_size":0,"ondisk_log_size":7168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:28.198746+0000","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":169,"num_read_kb":42926,"num_write":6485,"num_write_kb":1085292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"27'10055","reported_seq":10219,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:05.932579+0000","last_change":"2024-04-25T00:01:05.932614+0000","last_active":"2024-04-25T00:01:05.932579+0000","last_peered":"2024-04-25T00:01:05.932579+0000","last_clean":"2024-04-25T00:01:05.932579+0000","last_became_active":"2024-04-24T23:34:53.366430+0000","last_became_peered":"2024-04-24T23:34:53.366430+0000","last_unstale":"2024-04-25T00:01:05.932579+0000","last_undegraded":"2024-04-25T00:01:05.932579+0000","last_fullsized":"2024-04-25T00:01:05.932579+0000","mapping_epoch":18,"log_start":"27'3200","ondisk_log_start":"27'3200","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'10055","last_scrub_stamp":"2024-04-25T00:01:05.932513+0000","last_deep_scrub":"27'10055","last_deep_scrub_stamp":"2024-04-25T00:01:05.932513+0000","last_clean_scrub_stamp":"2024-04-25T00:01:05.932513+0000","objects_scrubbed":3,"log_size":6855,"log_dups_size":0,"ondisk_log_size":6855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:42:21.890767+0000","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5022938,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":120,"num_read_kb":893,"num_write":10045,"num_write_kb":77054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"27'2638","reported_seq":2681,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:03.880821+0000","last_change":"2024-04-25T00:01:03.880870+0000","last_active":"2024-04-25T00:01:03.880821+0000","last_peered":"2024-04-25T00:01:03.880821+0000","last_clean":"2024-04-25T00:01:03.880821+0000","last_became_active":"2024-04-24T23:34:54.372844+0000","last_became_peered":"2024-04-24T23:34:54.372844+0000","last_unstale":"2024-04-25T00:01:03.880821+0000","last_undegraded":"2024-04-25T00:01:03.880821+0000","last_fullsized":"2024-04-25T00:01:03.880821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2638","last_scrub_stamp":"2024-04-25T00:01:03.880722+0000","last_deep_scrub":"27'2638","last_deep_scrub_stamp":"2024-04-25T00:01:03.880722+0000","last_clean_scrub_stamp":"2024-04-25T00:01:03.880722+0000","objects_scrubbed":0,"log_size":2638,"log_dups_size":0,"ondisk_log_size":2638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:29:04.046881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"27'9403","reported_seq":10028,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:04.693593+0000","last_change":"2024-04-25T00:01:04.693617+0000","last_active":"2024-04-25T00:01:04.693593+0000","last_peered":"2024-04-25T00:01:04.693593+0000","last_clean":"2024-04-25T00:01:04.693593+0000","last_became_active":"2024-04-24T23:34:58.397817+0000","last_became_peered":"2024-04-24T23:34:58.397817+0000","last_unstale":"2024-04-25T00:01:04.693593+0000","last_undegraded":"2024-04-25T00:01:04.693593+0000","last_fullsized":"2024-04-25T00:01:04.693593+0000","mapping_epoch":23,"log_start":"27'2212","ondisk_log_start":"27'2212","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9403","last_scrub_stamp":"2024-04-25T00:01:04.693512+0000","last_deep_scrub":"27'9403","last_deep_scrub_stamp":"2024-04-25T00:01:04.693512+0000","last_clean_scrub_stamp":"2024-04-25T00:01:04.693512+0000","objects_scrubbed":0,"log_size":7191,"log_dups_size":0,"ondisk_log_size":7191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:18:20.800593+0000","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":588,"num_read_kb":149352,"num_write":6710,"num_write_kb":891395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"27'5159","reported_seq":5203,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:02.952659+0000","last_change":"2024-04-25T00:01:02.952701+0000","last_active":"2024-04-25T00:01:02.952659+0000","last_peered":"2024-04-25T00:01:02.952659+0000","last_clean":"2024-04-25T00:01:02.952659+0000","last_became_active":"2024-04-24T23:34:53.362337+0000","last_became_peered":"2024-04-24T23:34:53.362337+0000","last_unstale":"2024-04-25T00:01:02.952659+0000","last_undegraded":"2024-04-25T00:01:02.952659+0000","last_fullsized":"2024-04-25T00:01:02.952659+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5159","last_scrub_stamp":"2024-04-25T00:01:02.952565+0000","last_deep_scrub":"27'5159","last_deep_scrub_stamp":"2024-04-25T00:01:02.952565+0000","last_clean_scrub_stamp":"2024-04-25T00:01:02.952565+0000","objects_scrubbed":1,"log_size":5159,"log_dups_size":0,"ondisk_log_size":5159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:57:16.859658+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":5165,"num_write_kb":44324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"27'2566","reported_seq":2609,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:05.884139+0000","last_change":"2024-04-25T00:01:05.884161+0000","last_active":"2024-04-25T00:01:05.884139+0000","last_peered":"2024-04-25T00:01:05.884139+0000","last_clean":"2024-04-25T00:01:05.884139+0000","last_became_active":"2024-04-24T23:34:54.376019+0000","last_became_peered":"2024-04-24T23:34:54.376019+0000","last_unstale":"2024-04-25T00:01:05.884139+0000","last_undegraded":"2024-04-25T00:01:05.884139+0000","last_fullsized":"2024-04-25T00:01:05.884139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2566","last_scrub_stamp":"2024-04-25T00:01:05.884075+0000","last_deep_scrub":"27'2566","last_deep_scrub_stamp":"2024-04-25T00:01:05.884075+0000","last_clean_scrub_stamp":"2024-04-25T00:01:05.884075+0000","objects_scrubbed":0,"log_size":2566,"log_dups_size":0,"ondisk_log_size":2566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:26:32.174295+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"27'8765","reported_seq":9274,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:02.681788+0000","last_change":"2024-04-25T00:01:02.681812+0000","last_active":"2024-04-25T00:01:02.681788+0000","last_peered":"2024-04-25T00:01:02.681788+0000","last_clean":"2024-04-25T00:01:02.681788+0000","last_became_active":"2024-04-24T23:34:58.396658+0000","last_became_peered":"2024-04-24T23:34:58.396658+0000","last_unstale":"2024-04-25T00:01:02.681788+0000","last_undegraded":"2024-04-25T00:01:02.681788+0000","last_fullsized":"2024-04-25T00:01:02.681788+0000","mapping_epoch":23,"log_start":"27'1609","ondisk_log_start":"27'1609","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8765","last_scrub_stamp":"2024-04-25T00:01:02.681708+0000","last_deep_scrub":"27'8765","last_deep_scrub_stamp":"2024-04-25T00:01:02.681708+0000","last_clean_scrub_stamp":"2024-04-25T00:01:02.681708+0000","objects_scrubbed":0,"log_size":7156,"log_dups_size":0,"ondisk_log_size":7156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:08:10.518212+0000","scrub_duration":45,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":472,"num_read_kb":119888,"num_write":5982,"num_write_kb":849049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"27'2561","reported_seq":2605,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:05.138669+0000","last_change":"2024-04-25T00:01:05.138692+0000","last_active":"2024-04-25T00:01:05.138669+0000","last_peered":"2024-04-25T00:01:05.138669+0000","last_clean":"2024-04-25T00:01:05.138669+0000","last_became_active":"2024-04-24T23:34:54.377576+0000","last_became_peered":"2024-04-24T23:34:54.377576+0000","last_unstale":"2024-04-25T00:01:05.138669+0000","last_undegraded":"2024-04-25T00:01:05.138669+0000","last_fullsized":"2024-04-25T00:01:05.138669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2561","last_scrub_stamp":"2024-04-25T00:01:05.138606+0000","last_deep_scrub":"27'2561","last_deep_scrub_stamp":"2024-04-25T00:01:05.138606+0000","last_clean_scrub_stamp":"2024-04-25T00:01:05.138606+0000","objects_scrubbed":0,"log_size":2561,"log_dups_size":0,"ondisk_log_size":2561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:18:21.245687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"27'6838","reported_seq":6883,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:01.026815+0000","last_change":"2024-04-25T00:01:01.026856+0000","last_active":"2024-04-25T00:01:01.026815+0000","last_peered":"2024-04-25T00:01:01.026815+0000","last_clean":"2024-04-25T00:01:01.026815+0000","last_became_active":"2024-04-24T23:34:53.360635+0000","last_became_peered":"2024-04-24T23:34:53.360635+0000","last_unstale":"2024-04-25T00:01:01.026815+0000","last_undegraded":"2024-04-25T00:01:01.026815+0000","last_fullsized":"2024-04-25T00:01:01.026815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6838","last_scrub_stamp":"2024-04-25T00:01:01.026748+0000","last_deep_scrub":"27'6838","last_deep_scrub_stamp":"2024-04-25T00:01:01.026748+0000","last_clean_scrub_stamp":"2024-04-25T00:01:01.026748+0000","objects_scrubbed":4,"log_size":6838,"log_dups_size":0,"ondisk_log_size":6838,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:31:16.018413+0000","scrub_duration":64,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6840,"num_write_kb":53782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":246,"num_omap_keys":1,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"27'10213","reported_seq":10504,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:09.697769+0000","last_change":"2024-04-25T00:01:09.697805+0000","last_active":"2024-04-25T00:01:09.697769+0000","last_peered":"2024-04-25T00:01:09.697769+0000","last_clean":"2024-04-25T00:01:09.697769+0000","last_became_active":"2024-04-24T23:34:58.406844+0000","last_became_peered":"2024-04-24T23:34:58.406844+0000","last_unstale":"2024-04-25T00:01:09.697769+0000","last_undegraded":"2024-04-25T00:01:09.697769+0000","last_fullsized":"2024-04-25T00:01:09.697769+0000","mapping_epoch":23,"log_start":"27'3041","ondisk_log_start":"27'3041","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'10213","last_scrub_stamp":"2024-04-25T00:01:09.697684+0000","last_deep_scrub":"27'10213","last_deep_scrub_stamp":"2024-04-25T00:01:09.697684+0000","last_clean_scrub_stamp":"2024-04-25T00:01:09.697684+0000","objects_scrubbed":0,"log_size":7172,"log_dups_size":0,"ondisk_log_size":7172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:00.362322+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":253,"num_read_kb":64262,"num_write":7465,"num_write_kb":1030177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"27'2579","reported_seq":2623,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:06.203818+0000","last_change":"2024-04-25T00:01:06.203841+0000","last_active":"2024-04-25T00:01:06.203818+0000","last_peered":"2024-04-25T00:01:06.203818+0000","last_clean":"2024-04-25T00:01:06.203818+0000","last_became_active":"2024-04-24T23:34:54.366426+0000","last_became_peered":"2024-04-24T23:34:54.366426+0000","last_unstale":"2024-04-25T00:01:06.203818+0000","last_undegraded":"2024-04-25T00:01:06.203818+0000","last_fullsized":"2024-04-25T00:01:06.203818+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2579","last_scrub_stamp":"2024-04-25T00:01:06.203744+0000","last_deep_scrub":"27'2579","last_deep_scrub_stamp":"2024-04-25T00:01:06.203744+0000","last_clean_scrub_stamp":"2024-04-25T00:01:06.203744+0000","objects_scrubbed":0,"log_size":2579,"log_dups_size":0,"ondisk_log_size":2579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:54.928622+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"27'7915","reported_seq":7962,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:59.098554+0000","last_change":"2024-04-25T00:00:59.098554+0000","last_active":"2024-04-25T00:00:59.098554+0000","last_peered":"2024-04-25T00:00:59.098554+0000","last_clean":"2024-04-25T00:00:59.098554+0000","last_became_active":"2024-04-24T23:34:53.358841+0000","last_became_peered":"2024-04-24T23:34:53.358841+0000","last_unstale":"2024-04-25T00:00:59.098554+0000","last_undegraded":"2024-04-25T00:00:59.098554+0000","last_fullsized":"2024-04-25T00:00:59.098554+0000","mapping_epoch":18,"log_start":"27'500","ondisk_log_start":"27'500","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7915","last_scrub_stamp":"2024-04-25T00:00:59.098404+0000","last_deep_scrub":"27'7915","last_deep_scrub_stamp":"2024-04-25T00:00:59.098404+0000","last_clean_scrub_stamp":"2024-04-25T00:00:59.098404+0000","objects_scrubbed":2,"log_size":7415,"log_dups_size":0,"ondisk_log_size":7415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:59:04.579615+0000","scrub_duration":41,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7912,"num_write_kb":58475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"27'9714","reported_seq":10155,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:06.714444+0000","last_change":"2024-04-25T00:01:06.714487+0000","last_active":"2024-04-25T00:01:06.714444+0000","last_peered":"2024-04-25T00:01:06.714444+0000","last_clean":"2024-04-25T00:01:06.714444+0000","last_became_active":"2024-04-24T23:34:58.399375+0000","last_became_peered":"2024-04-24T23:34:58.399375+0000","last_unstale":"2024-04-25T00:01:06.714444+0000","last_undegraded":"2024-04-25T00:01:06.714444+0000","last_fullsized":"2024-04-25T00:01:06.714444+0000","mapping_epoch":23,"log_start":"27'2556","ondisk_log_start":"27'2556","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9714","last_scrub_stamp":"2024-04-25T00:01:06.714325+0000","last_deep_scrub":"27'9714","last_deep_scrub_stamp":"2024-04-25T00:01:06.714325+0000","last_clean_scrub_stamp":"2024-04-25T00:01:06.714325+0000","objects_scrubbed":0,"log_size":7158,"log_dups_size":0,"ondisk_log_size":7158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:07:54.046986+0000","scrub_duration":29,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":403,"num_read_kb":102362,"num_write":6994,"num_write_kb":969161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"27'2649","reported_seq":2694,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:05.228281+0000","last_change":"2024-04-25T00:01:05.228304+0000","last_active":"2024-04-25T00:01:05.228281+0000","last_peered":"2024-04-25T00:01:05.228281+0000","last_clean":"2024-04-25T00:01:05.228281+0000","last_became_active":"2024-04-24T23:34:54.364744+0000","last_became_peered":"2024-04-24T23:34:54.364744+0000","last_unstale":"2024-04-25T00:01:05.228281+0000","last_undegraded":"2024-04-25T00:01:05.228281+0000","last_fullsized":"2024-04-25T00:01:05.228281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2649","last_scrub_stamp":"2024-04-25T00:01:05.228201+0000","last_deep_scrub":"27'2649","last_deep_scrub_stamp":"2024-04-25T00:01:05.228201+0000","last_clean_scrub_stamp":"2024-04-25T00:01:05.228201+0000","objects_scrubbed":0,"log_size":2649,"log_dups_size":0,"ondisk_log_size":2649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:08:45.592988+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"27'4022","reported_seq":4069,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:01.102491+0000","last_change":"2024-04-25T00:01:01.102519+0000","last_active":"2024-04-25T00:01:01.102491+0000","last_peered":"2024-04-25T00:01:01.102491+0000","last_clean":"2024-04-25T00:01:01.102491+0000","last_became_active":"2024-04-24T23:34:53.360570+0000","last_became_peered":"2024-04-24T23:34:53.360570+0000","last_unstale":"2024-04-25T00:01:01.102491+0000","last_undegraded":"2024-04-25T00:01:01.102491+0000","last_fullsized":"2024-04-25T00:01:01.102491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4022","last_scrub_stamp":"2024-04-25T00:01:01.102425+0000","last_deep_scrub":"27'4022","last_deep_scrub_stamp":"2024-04-25T00:01:01.102425+0000","last_clean_scrub_stamp":"2024-04-25T00:01:01.102425+0000","objects_scrubbed":0,"log_size":4022,"log_dups_size":0,"ondisk_log_size":4022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:45:00.864134+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4022,"num_write_kb":27320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"27'9656","reported_seq":10086,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:11.200114+0000","last_change":"2024-04-25T00:01:11.200158+0000","last_active":"2024-04-25T00:01:11.200114+0000","last_peered":"2024-04-25T00:01:11.200114+0000","last_clean":"2024-04-25T00:01:11.200114+0000","last_became_active":"2024-04-24T23:34:58.731485+0000","last_became_peered":"2024-04-24T23:34:58.731485+0000","last_unstale":"2024-04-25T00:01:11.200114+0000","last_undegraded":"2024-04-25T00:01:11.200114+0000","last_fullsized":"2024-04-25T00:01:11.200114+0000","mapping_epoch":23,"log_start":"27'2316","ondisk_log_start":"27'2316","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9656","last_scrub_stamp":"2024-04-25T00:01:11.199988+0000","last_deep_scrub":"27'9656","last_deep_scrub_stamp":"2024-04-25T00:01:11.199988+0000","last_clean_scrub_stamp":"2024-04-25T00:01:11.199988+0000","objects_scrubbed":0,"log_size":7340,"log_dups_size":0,"ondisk_log_size":7340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:04:35.457528+0000","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":392,"num_read_kb":99568,"num_write":7039,"num_write_kb":904216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"27'2554","reported_seq":2599,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:57.711687+0000","last_change":"2024-04-25T00:00:57.711687+0000","last_active":"2024-04-25T00:00:57.711687+0000","last_peered":"2024-04-25T00:00:57.711687+0000","last_clean":"2024-04-25T00:00:57.711687+0000","last_became_active":"2024-04-24T23:34:54.362141+0000","last_became_peered":"2024-04-24T23:34:54.362141+0000","last_unstale":"2024-04-25T00:00:57.711687+0000","last_undegraded":"2024-04-25T00:00:57.711687+0000","last_fullsized":"2024-04-25T00:00:57.711687+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2554","last_scrub_stamp":"2024-04-25T00:00:57.711603+0000","last_deep_scrub":"27'2554","last_deep_scrub_stamp":"2024-04-25T00:00:57.711603+0000","last_clean_scrub_stamp":"2024-04-25T00:00:57.711603+0000","objects_scrubbed":0,"log_size":2554,"log_dups_size":0,"ondisk_log_size":2554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:02:44.568864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"27'2571","reported_seq":2617,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:04.967048+0000","last_change":"2024-04-25T00:01:04.967073+0000","last_active":"2024-04-25T00:01:04.967048+0000","last_peered":"2024-04-25T00:01:04.967048+0000","last_clean":"2024-04-25T00:01:04.967048+0000","last_became_active":"2024-04-24T23:34:53.366110+0000","last_became_peered":"2024-04-24T23:34:53.366110+0000","last_unstale":"2024-04-25T00:01:04.967048+0000","last_undegraded":"2024-04-25T00:01:04.967048+0000","last_fullsized":"2024-04-25T00:01:04.967048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2571","last_scrub_stamp":"2024-04-25T00:01:04.966976+0000","last_deep_scrub":"27'2571","last_deep_scrub_stamp":"2024-04-25T00:01:04.966976+0000","last_clean_scrub_stamp":"2024-04-25T00:01:04.966976+0000","objects_scrubbed":1,"log_size":2571,"log_dups_size":0,"ondisk_log_size":2571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:49:24.660148+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2576,"num_write_kb":22186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"27'9201","reported_seq":9390,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:07.712985+0000","last_change":"2024-04-25T00:01:07.713013+0000","last_active":"2024-04-25T00:01:07.712985+0000","last_peered":"2024-04-25T00:01:07.712985+0000","last_clean":"2024-04-25T00:01:07.712985+0000","last_became_active":"2024-04-24T23:34:58.402365+0000","last_became_peered":"2024-04-24T23:34:58.402365+0000","last_unstale":"2024-04-25T00:01:07.712985+0000","last_undegraded":"2024-04-25T00:01:07.712985+0000","last_fullsized":"2024-04-25T00:01:07.712985+0000","mapping_epoch":23,"log_start":"27'2049","ondisk_log_start":"27'2049","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9201","last_scrub_stamp":"2024-04-25T00:01:07.712900+0000","last_deep_scrub":"27'9201","last_deep_scrub_stamp":"2024-04-25T00:01:07.712900+0000","last_clean_scrub_stamp":"2024-04-25T00:01:07.712900+0000","objects_scrubbed":0,"log_size":7152,"log_dups_size":0,"ondisk_log_size":7152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:29:25.087812+0000","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":151,"num_read_kb":38354,"num_write":6553,"num_write_kb":977677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"27'2524","reported_seq":2569,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:01.012496+0000","last_change":"2024-04-25T00:01:01.012517+0000","last_active":"2024-04-25T00:01:01.012496+0000","last_peered":"2024-04-25T00:01:01.012496+0000","last_clean":"2024-04-25T00:01:01.012496+0000","last_became_active":"2024-04-24T23:34:54.369835+0000","last_became_peered":"2024-04-24T23:34:54.369835+0000","last_unstale":"2024-04-25T00:01:01.012496+0000","last_undegraded":"2024-04-25T00:01:01.012496+0000","last_fullsized":"2024-04-25T00:01:01.012496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2524","last_scrub_stamp":"2024-04-25T00:01:01.012421+0000","last_deep_scrub":"27'2524","last_deep_scrub_stamp":"2024-04-25T00:01:01.012421+0000","last_clean_scrub_stamp":"2024-04-25T00:01:01.012421+0000","objects_scrubbed":0,"log_size":2524,"log_dups_size":0,"ondisk_log_size":2524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:37:04.673623+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"27'4488","reported_seq":4535,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:01.248458+0000","last_change":"2024-04-25T00:01:01.248513+0000","last_active":"2024-04-25T00:01:01.248458+0000","last_peered":"2024-04-25T00:01:01.248458+0000","last_clean":"2024-04-25T00:01:01.248458+0000","last_became_active":"2024-04-24T23:34:53.365783+0000","last_became_peered":"2024-04-24T23:34:53.365783+0000","last_unstale":"2024-04-25T00:01:01.248458+0000","last_undegraded":"2024-04-25T00:01:01.248458+0000","last_fullsized":"2024-04-25T00:01:01.248458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4488","last_scrub_stamp":"2024-04-25T00:01:01.248363+0000","last_deep_scrub":"27'4488","last_deep_scrub_stamp":"2024-04-25T00:01:01.248363+0000","last_clean_scrub_stamp":"2024-04-25T00:01:01.248363+0000","objects_scrubbed":4,"log_size":4488,"log_dups_size":0,"ondisk_log_size":4488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:59:36.082415+0000","scrub_duration":48,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":4494,"num_write_kb":39872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"27'9221","reported_seq":9450,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:10.696821+0000","last_change":"2024-04-25T00:01:10.696849+0000","last_active":"2024-04-25T00:01:10.696821+0000","last_peered":"2024-04-25T00:01:10.696821+0000","last_clean":"2024-04-25T00:01:10.696821+0000","last_became_active":"2024-04-24T23:34:58.727653+0000","last_became_peered":"2024-04-24T23:34:58.727653+0000","last_unstale":"2024-04-25T00:01:10.696821+0000","last_undegraded":"2024-04-25T00:01:10.696821+0000","last_fullsized":"2024-04-25T00:01:10.696821+0000","mapping_epoch":23,"log_start":"27'2006","ondisk_log_start":"27'2006","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9221","last_scrub_stamp":"2024-04-25T00:01:10.696738+0000","last_deep_scrub":"27'9221","last_deep_scrub_stamp":"2024-04-25T00:01:10.696738+0000","last_clean_scrub_stamp":"2024-04-25T00:01:10.696738+0000","objects_scrubbed":0,"log_size":7215,"log_dups_size":0,"ondisk_log_size":7215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:47.378522+0000","scrub_duration":29,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":190,"num_read_kb":48260,"num_write":6573,"num_write_kb":840416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"27'2508","reported_seq":2553,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:07.250632+0000","last_change":"2024-04-25T00:01:07.250655+0000","last_active":"2024-04-25T00:01:07.250632+0000","last_peered":"2024-04-25T00:01:07.250632+0000","last_clean":"2024-04-25T00:01:07.250632+0000","last_became_active":"2024-04-24T23:34:54.367991+0000","last_became_peered":"2024-04-24T23:34:54.367991+0000","last_unstale":"2024-04-25T00:01:07.250632+0000","last_undegraded":"2024-04-25T00:01:07.250632+0000","last_fullsized":"2024-04-25T00:01:07.250632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2508","last_scrub_stamp":"2024-04-25T00:01:07.250562+0000","last_deep_scrub":"27'2508","last_deep_scrub_stamp":"2024-04-25T00:01:07.250562+0000","last_clean_scrub_stamp":"2024-04-25T00:01:07.250562+0000","objects_scrubbed":0,"log_size":2508,"log_dups_size":0,"ondisk_log_size":2508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:45:34.404171+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"27'7476","reported_seq":8148,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:02.196749+0000","last_change":"2024-04-25T00:01:02.196778+0000","last_active":"2024-04-25T00:01:02.196749+0000","last_peered":"2024-04-25T00:01:02.196749+0000","last_clean":"2024-04-25T00:01:02.196749+0000","last_became_active":"2024-04-24T23:34:53.363842+0000","last_became_peered":"2024-04-24T23:34:53.363842+0000","last_unstale":"2024-04-25T00:01:02.196749+0000","last_undegraded":"2024-04-25T00:01:02.196749+0000","last_fullsized":"2024-04-25T00:01:02.196749+0000","mapping_epoch":18,"log_start":"27'100","ondisk_log_start":"27'100","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7476","last_scrub_stamp":"2024-04-25T00:01:02.196688+0000","last_deep_scrub":"27'7476","last_deep_scrub_stamp":"2024-04-25T00:01:02.196688+0000","last_clean_scrub_stamp":"2024-04-25T00:01:02.196688+0000","objects_scrubbed":3,"log_size":7376,"log_dups_size":0,"ondisk_log_size":7376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:49:21.889860+0000","scrub_duration":111,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":626,"num_read_kb":4509,"num_write":10321,"num_write_kb":61532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"27'8920","reported_seq":9238,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:13.312090+0000","last_change":"2024-04-25T00:01:13.312133+0000","last_active":"2024-04-25T00:01:13.312090+0000","last_peered":"2024-04-25T00:01:13.312090+0000","last_clean":"2024-04-25T00:01:13.312090+0000","last_became_active":"2024-04-24T23:34:58.729082+0000","last_became_peered":"2024-04-24T23:34:58.729082+0000","last_unstale":"2024-04-25T00:01:13.312090+0000","last_undegraded":"2024-04-25T00:01:13.312090+0000","last_fullsized":"2024-04-25T00:01:13.312090+0000","mapping_epoch":23,"log_start":"27'2609","ondisk_log_start":"27'2609","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8920","last_scrub_stamp":"2024-04-25T00:01:13.311992+0000","last_deep_scrub":"27'8920","last_deep_scrub_stamp":"2024-04-25T00:01:13.311992+0000","last_clean_scrub_stamp":"2024-04-25T00:01:13.311992+0000","objects_scrubbed":0,"log_size":6311,"log_dups_size":0,"ondisk_log_size":6311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:17:28.784349+0000","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":278,"num_read_kb":70612,"num_write":6238,"num_write_kb":899702,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"27'2598","reported_seq":2643,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:04.112185+0000","last_change":"2024-04-25T00:01:04.112210+0000","last_active":"2024-04-25T00:01:04.112185+0000","last_peered":"2024-04-25T00:01:04.112185+0000","last_clean":"2024-04-25T00:01:04.112185+0000","last_became_active":"2024-04-24T23:34:54.356560+0000","last_became_peered":"2024-04-24T23:34:54.356560+0000","last_unstale":"2024-04-25T00:01:04.112185+0000","last_undegraded":"2024-04-25T00:01:04.112185+0000","last_fullsized":"2024-04-25T00:01:04.112185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2598","last_scrub_stamp":"2024-04-25T00:01:04.112109+0000","last_deep_scrub":"27'2598","last_deep_scrub_stamp":"2024-04-25T00:01:04.112109+0000","last_clean_scrub_stamp":"2024-04-25T00:01:04.112109+0000","objects_scrubbed":0,"log_size":2598,"log_dups_size":0,"ondisk_log_size":2598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:07:51.444770+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"27'6263","reported_seq":6309,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:03.994566+0000","last_change":"2024-04-25T00:01:03.994625+0000","last_active":"2024-04-25T00:01:03.994566+0000","last_peered":"2024-04-25T00:01:03.994566+0000","last_clean":"2024-04-25T00:01:03.994566+0000","last_became_active":"2024-04-24T23:34:53.365636+0000","last_became_peered":"2024-04-24T23:34:53.365636+0000","last_unstale":"2024-04-25T00:01:03.994566+0000","last_undegraded":"2024-04-25T00:01:03.994566+0000","last_fullsized":"2024-04-25T00:01:03.994566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6263","last_scrub_stamp":"2024-04-25T00:01:03.994496+0000","last_deep_scrub":"27'6263","last_deep_scrub_stamp":"2024-04-25T00:01:03.994496+0000","last_clean_scrub_stamp":"2024-04-25T00:01:03.994496+0000","objects_scrubbed":2,"log_size":6263,"log_dups_size":0,"ondisk_log_size":6263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:45:03.756205+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":6274,"num_write_kb":49208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"27'9152","reported_seq":9695,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:09.005063+0000","last_change":"2024-04-25T00:01:09.005107+0000","last_active":"2024-04-25T00:01:09.005063+0000","last_peered":"2024-04-25T00:01:09.005063+0000","last_clean":"2024-04-25T00:01:09.005063+0000","last_became_active":"2024-04-24T23:34:58.727885+0000","last_became_peered":"2024-04-24T23:34:58.727885+0000","last_unstale":"2024-04-25T00:01:09.005063+0000","last_undegraded":"2024-04-25T00:01:09.005063+0000","last_fullsized":"2024-04-25T00:01:09.005063+0000","mapping_epoch":23,"log_start":"27'1968","ondisk_log_start":"27'1968","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9152","last_scrub_stamp":"2024-04-25T00:01:09.004937+0000","last_deep_scrub":"27'9152","last_deep_scrub_stamp":"2024-04-25T00:01:09.004937+0000","last_clean_scrub_stamp":"2024-04-25T00:01:09.004937+0000","objects_scrubbed":0,"log_size":7184,"log_dups_size":0,"ondisk_log_size":7184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:45.686721+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":504,"num_read_kb":128016,"num_write":6457,"num_write_kb":842362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"27'2505","reported_seq":2550,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:01.936748+0000","last_change":"2024-04-25T00:01:01.936790+0000","last_active":"2024-04-25T00:01:01.936748+0000","last_peered":"2024-04-25T00:01:01.936748+0000","last_clean":"2024-04-25T00:01:01.936748+0000","last_became_active":"2024-04-24T23:34:54.371947+0000","last_became_peered":"2024-04-24T23:34:54.371947+0000","last_unstale":"2024-04-25T00:01:01.936748+0000","last_undegraded":"2024-04-25T00:01:01.936748+0000","last_fullsized":"2024-04-25T00:01:01.936748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2505","last_scrub_stamp":"2024-04-25T00:01:01.936631+0000","last_deep_scrub":"27'2505","last_deep_scrub_stamp":"2024-04-25T00:01:01.936631+0000","last_clean_scrub_stamp":"2024-04-25T00:01:01.936631+0000","objects_scrubbed":0,"log_size":2505,"log_dups_size":0,"ondisk_log_size":2505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:49:21.629803+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"27'7230","reported_seq":7277,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:00.113053+0000","last_change":"2024-04-25T00:01:00.113141+0000","last_active":"2024-04-25T00:01:00.113053+0000","last_peered":"2024-04-25T00:01:00.113053+0000","last_clean":"2024-04-25T00:01:00.113053+0000","last_became_active":"2024-04-24T23:34:53.360082+0000","last_became_peered":"2024-04-24T23:34:53.360082+0000","last_unstale":"2024-04-25T00:01:00.113053+0000","last_undegraded":"2024-04-25T00:01:00.113053+0000","last_fullsized":"2024-04-25T00:01:00.113053+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7230","last_scrub_stamp":"2024-04-25T00:01:00.112956+0000","last_deep_scrub":"27'7230","last_deep_scrub_stamp":"2024-04-25T00:01:00.112956+0000","last_clean_scrub_stamp":"2024-04-25T00:01:00.112956+0000","objects_scrubbed":5,"log_size":7230,"log_dups_size":0,"ondisk_log_size":7230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:08:07.949460+0000","scrub_duration":63,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17487510,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":7224,"num_write_kb":70858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"27'9197","reported_seq":9391,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:10.157328+0000","last_change":"2024-04-25T00:01:10.157355+0000","last_active":"2024-04-25T00:01:10.157328+0000","last_peered":"2024-04-25T00:01:10.157328+0000","last_clean":"2024-04-25T00:01:10.157328+0000","last_became_active":"2024-04-24T23:34:58.403735+0000","last_became_peered":"2024-04-24T23:34:58.403735+0000","last_unstale":"2024-04-25T00:01:10.157328+0000","last_undegraded":"2024-04-25T00:01:10.157328+0000","last_fullsized":"2024-04-25T00:01:10.157328+0000","mapping_epoch":23,"log_start":"27'1819","ondisk_log_start":"27'1819","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9197","last_scrub_stamp":"2024-04-25T00:01:10.157243+0000","last_deep_scrub":"27'9197","last_deep_scrub_stamp":"2024-04-25T00:01:10.157243+0000","last_clean_scrub_stamp":"2024-04-25T00:01:10.157243+0000","objects_scrubbed":0,"log_size":7378,"log_dups_size":0,"ondisk_log_size":7378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:00.083638+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":155,"num_read_kb":39370,"num_write":6529,"num_write_kb":924891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"27'2542","reported_seq":2587,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:59.692895+0000","last_change":"2024-04-25T00:00:59.692933+0000","last_active":"2024-04-25T00:00:59.692895+0000","last_peered":"2024-04-25T00:00:59.692895+0000","last_clean":"2024-04-25T00:00:59.692895+0000","last_became_active":"2024-04-24T23:34:54.367355+0000","last_became_peered":"2024-04-24T23:34:54.367355+0000","last_unstale":"2024-04-25T00:00:59.692895+0000","last_undegraded":"2024-04-25T00:00:59.692895+0000","last_fullsized":"2024-04-25T00:00:59.692895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2542","last_scrub_stamp":"2024-04-25T00:00:59.692781+0000","last_deep_scrub":"27'2542","last_deep_scrub_stamp":"2024-04-25T00:00:59.692781+0000","last_clean_scrub_stamp":"2024-04-25T00:00:59.692781+0000","objects_scrubbed":0,"log_size":2542,"log_dups_size":0,"ondisk_log_size":2542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:15.110780+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"27'8390","reported_seq":8437,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:57.031782+0000","last_change":"2024-04-25T00:00:57.031834+0000","last_active":"2024-04-25T00:00:57.031782+0000","last_peered":"2024-04-25T00:00:57.031782+0000","last_clean":"2024-04-25T00:00:57.031782+0000","last_became_active":"2024-04-24T23:34:53.355710+0000","last_became_peered":"2024-04-24T23:34:53.355710+0000","last_unstale":"2024-04-25T00:00:57.031782+0000","last_undegraded":"2024-04-25T00:00:57.031782+0000","last_fullsized":"2024-04-25T00:00:57.031782+0000","mapping_epoch":18,"log_start":"27'600","ondisk_log_start":"27'600","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8390","last_scrub_stamp":"2024-04-25T00:00:57.031693+0000","last_deep_scrub":"27'8390","last_deep_scrub_stamp":"2024-04-25T00:00:57.031693+0000","last_clean_scrub_stamp":"2024-04-25T00:00:57.031693+0000","objects_scrubbed":2,"log_size":7790,"log_dups_size":0,"ondisk_log_size":7790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:12.449692+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":8401,"num_write_kb":63182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"27'9677","reported_seq":10050,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:08.156503+0000","last_change":"2024-04-25T00:01:08.156527+0000","last_active":"2024-04-25T00:01:08.156503+0000","last_peered":"2024-04-25T00:01:08.156503+0000","last_clean":"2024-04-25T00:01:08.156503+0000","last_became_active":"2024-04-24T23:34:58.402598+0000","last_became_peered":"2024-04-24T23:34:58.402598+0000","last_unstale":"2024-04-25T00:01:08.156503+0000","last_undegraded":"2024-04-25T00:01:08.156503+0000","last_fullsized":"2024-04-25T00:01:08.156503+0000","mapping_epoch":23,"log_start":"27'2299","ondisk_log_start":"27'2299","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9677","last_scrub_stamp":"2024-04-25T00:01:08.156410+0000","last_deep_scrub":"27'9677","last_deep_scrub_stamp":"2024-04-25T00:01:08.156410+0000","last_clean_scrub_stamp":"2024-04-25T00:01:08.156410+0000","objects_scrubbed":0,"log_size":7378,"log_dups_size":0,"ondisk_log_size":7378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:08.786202+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":334,"num_read_kb":84836,"num_write":6883,"num_write_kb":886687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"27'2549","reported_seq":2595,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:03.081113+0000","last_change":"2024-04-25T00:01:03.081113+0000","last_active":"2024-04-25T00:01:03.081113+0000","last_peered":"2024-04-25T00:01:03.081113+0000","last_clean":"2024-04-25T00:01:03.081113+0000","last_became_active":"2024-04-24T23:34:54.356326+0000","last_became_peered":"2024-04-24T23:34:54.356326+0000","last_unstale":"2024-04-25T00:01:03.081113+0000","last_undegraded":"2024-04-25T00:01:03.081113+0000","last_fullsized":"2024-04-25T00:01:03.081113+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2549","last_scrub_stamp":"2024-04-25T00:01:03.081031+0000","last_deep_scrub":"27'2549","last_deep_scrub_stamp":"2024-04-25T00:01:03.081031+0000","last_clean_scrub_stamp":"2024-04-25T00:01:03.081031+0000","objects_scrubbed":0,"log_size":2549,"log_dups_size":0,"ondisk_log_size":2549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:42:19.039285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"27'7179","reported_seq":8155,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:58.015120+0000","last_change":"2024-04-25T00:00:58.015150+0000","last_active":"2024-04-25T00:00:58.015120+0000","last_peered":"2024-04-25T00:00:58.015120+0000","last_clean":"2024-04-25T00:00:58.015120+0000","last_became_active":"2024-04-24T23:34:53.358937+0000","last_became_peered":"2024-04-24T23:34:53.358937+0000","last_unstale":"2024-04-25T00:00:58.015120+0000","last_undegraded":"2024-04-25T00:00:58.015120+0000","last_fullsized":"2024-04-25T00:00:58.015120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7179","last_scrub_stamp":"2024-04-25T00:00:58.015059+0000","last_deep_scrub":"27'7179","last_deep_scrub_stamp":"2024-04-25T00:00:58.015059+0000","last_clean_scrub_stamp":"2024-04-25T00:00:58.015059+0000","objects_scrubbed":2,"log_size":7179,"log_dups_size":0,"ondisk_log_size":7179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:13:14.047029+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":929,"num_read_kb":4700,"num_write":7175,"num_write_kb":61982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"27'9030","reported_seq":9633,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:09.134170+0000","last_change":"2024-04-25T00:01:09.134230+0000","last_active":"2024-04-25T00:01:09.134170+0000","last_peered":"2024-04-25T00:01:09.134170+0000","last_clean":"2024-04-25T00:01:09.134170+0000","last_became_active":"2024-04-24T23:34:58.402569+0000","last_became_peered":"2024-04-24T23:34:58.402569+0000","last_unstale":"2024-04-25T00:01:09.134170+0000","last_undegraded":"2024-04-25T00:01:09.134170+0000","last_fullsized":"2024-04-25T00:01:09.134170+0000","mapping_epoch":23,"log_start":"27'1638","ondisk_log_start":"27'1638","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9030","last_scrub_stamp":"2024-04-25T00:01:09.134031+0000","last_deep_scrub":"27'9030","last_deep_scrub_stamp":"2024-04-25T00:01:09.134031+0000","last_clean_scrub_stamp":"2024-04-25T00:01:09.134031+0000","objects_scrubbed":0,"log_size":7392,"log_dups_size":0,"ondisk_log_size":7392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:15:13.537860+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":564,"num_read_kb":143256,"num_write":6307,"num_write_kb":911438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"27'2483","reported_seq":2528,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:03.975184+0000","last_change":"2024-04-25T00:01:03.975224+0000","last_active":"2024-04-25T00:01:03.975184+0000","last_peered":"2024-04-25T00:01:03.975184+0000","last_clean":"2024-04-25T00:01:03.975184+0000","last_became_active":"2024-04-24T23:34:54.377228+0000","last_became_peered":"2024-04-24T23:34:54.377228+0000","last_unstale":"2024-04-25T00:01:03.975184+0000","last_undegraded":"2024-04-25T00:01:03.975184+0000","last_fullsized":"2024-04-25T00:01:03.975184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2483","last_scrub_stamp":"2024-04-25T00:01:03.975085+0000","last_deep_scrub":"27'2483","last_deep_scrub_stamp":"2024-04-25T00:01:03.975085+0000","last_clean_scrub_stamp":"2024-04-25T00:01:03.975085+0000","objects_scrubbed":0,"log_size":2483,"log_dups_size":0,"ondisk_log_size":2483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:29:04.141244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"27'7207","reported_seq":7254,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:01.942254+0000","last_change":"2024-04-25T00:01:01.942296+0000","last_active":"2024-04-25T00:01:01.942254+0000","last_peered":"2024-04-25T00:01:01.942254+0000","last_clean":"2024-04-25T00:01:01.942254+0000","last_became_active":"2024-04-24T23:34:53.362425+0000","last_became_peered":"2024-04-24T23:34:53.362425+0000","last_unstale":"2024-04-25T00:01:01.942254+0000","last_undegraded":"2024-04-25T00:01:01.942254+0000","last_fullsized":"2024-04-25T00:01:01.942254+0000","mapping_epoch":18,"log_start":"27'300","ondisk_log_start":"27'300","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7207","last_scrub_stamp":"2024-04-25T00:01:01.942161+0000","last_deep_scrub":"27'7207","last_deep_scrub_stamp":"2024-04-25T00:01:01.942161+0000","last_clean_scrub_stamp":"2024-04-25T00:01:01.942161+0000","objects_scrubbed":1,"log_size":6907,"log_dups_size":0,"ondisk_log_size":6907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:59:07.423372+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":7208,"num_write_kb":58030,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"27'9329","reported_seq":9861,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:09.964607+0000","last_change":"2024-04-25T00:01:09.964632+0000","last_active":"2024-04-25T00:01:09.964607+0000","last_peered":"2024-04-25T00:01:09.964607+0000","last_clean":"2024-04-25T00:01:09.964607+0000","last_became_active":"2024-04-24T23:34:58.727284+0000","last_became_peered":"2024-04-24T23:34:58.727284+0000","last_unstale":"2024-04-25T00:01:09.964607+0000","last_undegraded":"2024-04-25T00:01:09.964607+0000","last_fullsized":"2024-04-25T00:01:09.964607+0000","mapping_epoch":23,"log_start":"27'2114","ondisk_log_start":"27'2114","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9329","last_scrub_stamp":"2024-04-25T00:01:09.964510+0000","last_deep_scrub":"27'9329","last_deep_scrub_stamp":"2024-04-25T00:01:09.964510+0000","last_clean_scrub_stamp":"2024-04-25T00:01:09.964510+0000","objects_scrubbed":0,"log_size":7215,"log_dups_size":0,"ondisk_log_size":7215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:59:06.046664+0000","scrub_duration":30,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":493,"num_read_kb":125222,"num_write":6691,"num_write_kb":1110316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"27'2597","reported_seq":2642,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:08.253077+0000","last_change":"2024-04-25T00:01:08.253097+0000","last_active":"2024-04-25T00:01:08.253077+0000","last_peered":"2024-04-25T00:01:08.253077+0000","last_clean":"2024-04-25T00:01:08.253077+0000","last_became_active":"2024-04-24T23:34:54.382564+0000","last_became_peered":"2024-04-24T23:34:54.382564+0000","last_unstale":"2024-04-25T00:01:08.253077+0000","last_undegraded":"2024-04-25T00:01:08.253077+0000","last_fullsized":"2024-04-25T00:01:08.253077+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2597","last_scrub_stamp":"2024-04-25T00:01:08.253016+0000","last_deep_scrub":"27'2597","last_deep_scrub_stamp":"2024-04-25T00:01:08.253016+0000","last_clean_scrub_stamp":"2024-04-25T00:01:08.253016+0000","objects_scrubbed":0,"log_size":2597,"log_dups_size":0,"ondisk_log_size":2597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:28.022070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"27'5190","reported_seq":5237,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:57.975969+0000","last_change":"2024-04-25T00:00:57.976027+0000","last_active":"2024-04-25T00:00:57.975969+0000","last_peered":"2024-04-25T00:00:57.975969+0000","last_clean":"2024-04-25T00:00:57.975969+0000","last_became_active":"2024-04-24T23:34:53.357456+0000","last_became_peered":"2024-04-24T23:34:53.357456+0000","last_unstale":"2024-04-25T00:00:57.975969+0000","last_undegraded":"2024-04-25T00:00:57.975969+0000","last_fullsized":"2024-04-25T00:00:57.975969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5190","last_scrub_stamp":"2024-04-25T00:00:57.975892+0000","last_deep_scrub":"27'5190","last_deep_scrub_stamp":"2024-04-25T00:00:57.975892+0000","last_clean_scrub_stamp":"2024-04-25T00:00:57.975892+0000","objects_scrubbed":1,"log_size":5190,"log_dups_size":0,"ondisk_log_size":5190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:31:12.967557+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5191,"num_write_kb":40339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"27'119","reported_seq":205,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:00.011024+0000","last_change":"2024-04-25T00:01:00.011070+0000","last_active":"2024-04-25T00:01:00.011024+0000","last_peered":"2024-04-25T00:01:00.011024+0000","last_clean":"2024-04-25T00:01:00.011024+0000","last_became_active":"2024-04-24T23:34:49.778083+0000","last_became_peered":"2024-04-24T23:34:49.778083+0000","last_unstale":"2024-04-25T00:01:00.011024+0000","last_undegraded":"2024-04-25T00:01:00.011024+0000","last_fullsized":"2024-04-25T00:01:00.011024+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'119","last_scrub_stamp":"2024-04-25T00:01:00.010927+0000","last_deep_scrub":"27'119","last_deep_scrub_stamp":"2024-04-25T00:01:00.010927+0000","last_clean_scrub_stamp":"2024-04-25T00:01:00.010927+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:15.428926+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"27'8889","reported_seq":9415,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:10.091675+0000","last_change":"2024-04-25T00:01:10.091700+0000","last_active":"2024-04-25T00:01:10.091675+0000","last_peered":"2024-04-25T00:01:10.091675+0000","last_clean":"2024-04-25T00:01:10.091675+0000","last_became_active":"2024-04-24T23:34:58.729370+0000","last_became_peered":"2024-04-24T23:34:58.729370+0000","last_unstale":"2024-04-25T00:01:10.091675+0000","last_undegraded":"2024-04-25T00:01:10.091675+0000","last_fullsized":"2024-04-25T00:01:10.091675+0000","mapping_epoch":23,"log_start":"27'902","ondisk_log_start":"27'902","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8889","last_scrub_stamp":"2024-04-25T00:01:10.091591+0000","last_deep_scrub":"27'8889","last_deep_scrub_stamp":"2024-04-25T00:01:10.091591+0000","last_clean_scrub_stamp":"2024-04-25T00:01:10.091591+0000","objects_scrubbed":0,"log_size":7987,"log_dups_size":0,"ondisk_log_size":7987,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:00.756229+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":487,"num_read_kb":123698,"num_write":6268,"num_write_kb":912685,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"27'2533","reported_seq":2568,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:51.040389+0000","last_change":"2024-04-24T23:34:54.384106+0000","last_active":"2024-04-25T00:00:51.040389+0000","last_peered":"2024-04-25T00:00:51.040389+0000","last_clean":"2024-04-25T00:00:51.040389+0000","last_became_active":"2024-04-24T23:34:54.383589+0000","last_became_peered":"2024-04-24T23:34:54.383589+0000","last_unstale":"2024-04-25T00:00:51.040389+0000","last_undegraded":"2024-04-25T00:00:51.040389+0000","last_fullsized":"2024-04-25T00:00:51.040389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:34:53.338884+0000","last_clean_scrub_stamp":"2024-04-24T23:34:53.338884+0000","objects_scrubbed":0,"log_size":2533,"log_dups_size":0,"ondisk_log_size":2533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"27'8233","reported_seq":8279,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:56.027938+0000","last_change":"2024-04-25T00:00:56.027970+0000","last_active":"2024-04-25T00:00:56.027938+0000","last_peered":"2024-04-25T00:00:56.027938+0000","last_clean":"2024-04-25T00:00:56.027938+0000","last_became_active":"2024-04-24T23:34:53.353804+0000","last_became_peered":"2024-04-24T23:34:53.353804+0000","last_unstale":"2024-04-25T00:00:56.027938+0000","last_undegraded":"2024-04-25T00:00:56.027938+0000","last_fullsized":"2024-04-25T00:00:56.027938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8233","last_scrub_stamp":"2024-04-25T00:00:56.027870+0000","last_deep_scrub":"27'8233","last_deep_scrub_stamp":"2024-04-25T00:00:56.027870+0000","last_clean_scrub_stamp":"2024-04-25T00:00:56.027870+0000","objects_scrubbed":2,"log_size":8233,"log_dups_size":0,"ondisk_log_size":8233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:58:35.100912+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8225,"num_write_kb":58738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"27'9238","reported_seq":9434,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:09.117913+0000","last_change":"2024-04-25T00:01:09.117938+0000","last_active":"2024-04-25T00:01:09.117913+0000","last_peered":"2024-04-25T00:01:09.117913+0000","last_clean":"2024-04-25T00:01:09.117913+0000","last_became_active":"2024-04-24T23:34:58.727649+0000","last_became_peered":"2024-04-24T23:34:58.727649+0000","last_unstale":"2024-04-25T00:01:09.117913+0000","last_undegraded":"2024-04-25T00:01:09.117913+0000","last_fullsized":"2024-04-25T00:01:09.117913+0000","mapping_epoch":23,"log_start":"27'1312","ondisk_log_start":"27'1312","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9238","last_scrub_stamp":"2024-04-25T00:01:09.117822+0000","last_deep_scrub":"27'9238","last_deep_scrub_stamp":"2024-04-25T00:01:09.117822+0000","last_clean_scrub_stamp":"2024-04-25T00:01:09.117822+0000","objects_scrubbed":0,"log_size":7926,"log_dups_size":0,"ondisk_log_size":7926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:28.602762+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":158,"num_read_kb":40132,"num_write":6566,"num_write_kb":960960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"27'2561","reported_seq":2605,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:02.891955+0000","last_change":"2024-04-25T00:01:02.891995+0000","last_active":"2024-04-25T00:01:02.891955+0000","last_peered":"2024-04-25T00:01:02.891955+0000","last_clean":"2024-04-25T00:01:02.891955+0000","last_became_active":"2024-04-24T23:34:54.372553+0000","last_became_peered":"2024-04-24T23:34:54.372553+0000","last_unstale":"2024-04-25T00:01:02.891955+0000","last_undegraded":"2024-04-25T00:01:02.891955+0000","last_fullsized":"2024-04-25T00:01:02.891955+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2561","last_scrub_stamp":"2024-04-25T00:01:02.891883+0000","last_deep_scrub":"27'2561","last_deep_scrub_stamp":"2024-04-25T00:01:02.891883+0000","last_clean_scrub_stamp":"2024-04-25T00:01:02.891883+0000","objects_scrubbed":0,"log_size":2561,"log_dups_size":0,"ondisk_log_size":2561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:42:18.850137+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"27'8396","reported_seq":8442,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:57.135015+0000","last_change":"2024-04-25T00:00:57.135047+0000","last_active":"2024-04-25T00:00:57.135015+0000","last_peered":"2024-04-25T00:00:57.135015+0000","last_clean":"2024-04-25T00:00:57.135015+0000","last_became_active":"2024-04-24T23:34:53.353186+0000","last_became_peered":"2024-04-24T23:34:53.353186+0000","last_unstale":"2024-04-25T00:00:57.135015+0000","last_undegraded":"2024-04-25T00:00:57.135015+0000","last_fullsized":"2024-04-25T00:00:57.135015+0000","mapping_epoch":18,"log_start":"27'1000","ondisk_log_start":"27'1000","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8396","last_scrub_stamp":"2024-04-25T00:00:57.134940+0000","last_deep_scrub":"27'8396","last_deep_scrub_stamp":"2024-04-25T00:00:57.134940+0000","last_clean_scrub_stamp":"2024-04-25T00:00:57.134940+0000","objects_scrubbed":3,"log_size":7396,"log_dups_size":0,"ondisk_log_size":7396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:12.552939+0000","scrub_duration":38,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388678,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8393,"num_write_kb":58880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"27'9582","reported_seq":9946,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:08.003411+0000","last_change":"2024-04-25T00:01:08.003455+0000","last_active":"2024-04-25T00:01:08.003411+0000","last_peered":"2024-04-25T00:01:08.003411+0000","last_clean":"2024-04-25T00:01:08.003411+0000","last_became_active":"2024-04-24T23:34:58.400549+0000","last_became_peered":"2024-04-24T23:34:58.400549+0000","last_unstale":"2024-04-25T00:01:08.003411+0000","last_undegraded":"2024-04-25T00:01:08.003411+0000","last_fullsized":"2024-04-25T00:01:08.003411+0000","mapping_epoch":23,"log_start":"27'2356","ondisk_log_start":"27'2356","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9582","last_scrub_stamp":"2024-04-25T00:01:08.003283+0000","last_deep_scrub":"27'9582","last_deep_scrub_stamp":"2024-04-25T00:01:08.003283+0000","last_clean_scrub_stamp":"2024-04-25T00:01:08.003283+0000","objects_scrubbed":0,"log_size":7226,"log_dups_size":0,"ondisk_log_size":7226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:13:25.549872+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":326,"num_read_kb":82804,"num_write":6847,"num_write_kb":886980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"27'2433","reported_seq":2476,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:09.873327+0000","last_change":"2024-04-25T00:01:09.873348+0000","last_active":"2024-04-25T00:01:09.873327+0000","last_peered":"2024-04-25T00:01:09.873327+0000","last_clean":"2024-04-25T00:01:09.873327+0000","last_became_active":"2024-04-24T23:34:54.374963+0000","last_became_peered":"2024-04-24T23:34:54.374963+0000","last_unstale":"2024-04-25T00:01:09.873327+0000","last_undegraded":"2024-04-25T00:01:09.873327+0000","last_fullsized":"2024-04-25T00:01:09.873327+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2433","last_scrub_stamp":"2024-04-25T00:01:09.873258+0000","last_deep_scrub":"27'2433","last_deep_scrub_stamp":"2024-04-25T00:01:09.873258+0000","last_clean_scrub_stamp":"2024-04-25T00:01:09.873258+0000","objects_scrubbed":0,"log_size":2433,"log_dups_size":0,"ondisk_log_size":2433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:51.544421+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"27'8944","reported_seq":8990,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:59.112330+0000","last_change":"2024-04-25T00:00:59.112379+0000","last_active":"2024-04-25T00:00:59.112330+0000","last_peered":"2024-04-25T00:00:59.112330+0000","last_clean":"2024-04-25T00:00:59.112330+0000","last_became_active":"2024-04-24T23:34:53.360247+0000","last_became_peered":"2024-04-24T23:34:53.360247+0000","last_unstale":"2024-04-25T00:00:59.112330+0000","last_undegraded":"2024-04-25T00:00:59.112330+0000","last_fullsized":"2024-04-25T00:00:59.112330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8944","last_scrub_stamp":"2024-04-25T00:00:59.112263+0000","last_deep_scrub":"27'8944","last_deep_scrub_stamp":"2024-04-25T00:00:59.112263+0000","last_clean_scrub_stamp":"2024-04-25T00:00:59.112263+0000","objects_scrubbed":7,"log_size":8944,"log_dups_size":0,"ondisk_log_size":8944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:47:18.749387+0000","scrub_duration":101,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8935,"num_write_kb":79464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"27'9258","reported_seq":10451,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:11.889222+0000","last_change":"2024-04-25T00:01:11.889248+0000","last_active":"2024-04-25T00:01:11.889222+0000","last_peered":"2024-04-25T00:01:11.889222+0000","last_clean":"2024-04-25T00:01:11.889222+0000","last_became_active":"2024-04-24T23:34:58.729345+0000","last_became_peered":"2024-04-24T23:34:58.729345+0000","last_unstale":"2024-04-25T00:01:11.889222+0000","last_undegraded":"2024-04-25T00:01:11.889222+0000","last_fullsized":"2024-04-25T00:01:11.889222+0000","mapping_epoch":23,"log_start":"27'1527","ondisk_log_start":"27'1527","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9258","last_scrub_stamp":"2024-04-25T00:01:11.889135+0000","last_deep_scrub":"27'9258","last_deep_scrub_stamp":"2024-04-25T00:01:11.889135+0000","last_clean_scrub_stamp":"2024-04-25T00:01:11.889135+0000","objects_scrubbed":0,"log_size":7731,"log_dups_size":0,"ondisk_log_size":7731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:17:58.223321+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1154,"num_read_kb":293116,"num_write":6516,"num_write_kb":1022776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"27'2529","reported_seq":2572,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:11.803877+0000","last_change":"2024-04-25T00:01:11.803899+0000","last_active":"2024-04-25T00:01:11.803877+0000","last_peered":"2024-04-25T00:01:11.803877+0000","last_clean":"2024-04-25T00:01:11.803877+0000","last_became_active":"2024-04-24T23:34:54.377270+0000","last_became_peered":"2024-04-24T23:34:54.377270+0000","last_unstale":"2024-04-25T00:01:11.803877+0000","last_undegraded":"2024-04-25T00:01:11.803877+0000","last_fullsized":"2024-04-25T00:01:11.803877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2529","last_scrub_stamp":"2024-04-25T00:01:11.803800+0000","last_deep_scrub":"27'2529","last_deep_scrub_stamp":"2024-04-25T00:01:11.803800+0000","last_clean_scrub_stamp":"2024-04-25T00:01:11.803800+0000","objects_scrubbed":0,"log_size":2529,"log_dups_size":0,"ondisk_log_size":2529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:15:16.207629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"27'4551","reported_seq":4597,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:56.705155+0000","last_change":"2024-04-25T00:00:56.705183+0000","last_active":"2024-04-25T00:00:56.705155+0000","last_peered":"2024-04-25T00:00:56.705155+0000","last_clean":"2024-04-25T00:00:56.705155+0000","last_became_active":"2024-04-24T23:34:53.352669+0000","last_became_peered":"2024-04-24T23:34:53.352669+0000","last_unstale":"2024-04-25T00:00:56.705155+0000","last_undegraded":"2024-04-25T00:00:56.705155+0000","last_fullsized":"2024-04-25T00:00:56.705155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4551","last_scrub_stamp":"2024-04-25T00:00:56.705093+0000","last_deep_scrub":"27'4551","last_deep_scrub_stamp":"2024-04-25T00:00:56.705093+0000","last_clean_scrub_stamp":"2024-04-25T00:00:56.705093+0000","objects_scrubbed":2,"log_size":4551,"log_dups_size":0,"ondisk_log_size":4551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:14:54.766742+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4539,"num_write_kb":43853,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"27'9062","reported_seq":9852,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:05.737990+0000","last_change":"2024-04-25T00:01:05.737990+0000","last_active":"2024-04-25T00:01:05.737990+0000","last_peered":"2024-04-25T00:01:05.737990+0000","last_clean":"2024-04-25T00:01:05.737990+0000","last_became_active":"2024-04-24T23:35:43.529375+0000","last_became_peered":"2024-04-24T23:35:43.529375+0000","last_unstale":"2024-04-25T00:01:05.737990+0000","last_undegraded":"2024-04-25T00:01:05.737990+0000","last_fullsized":"2024-04-25T00:01:05.737990+0000","mapping_epoch":26,"log_start":"27'1912","ondisk_log_start":"27'1912","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9062","last_scrub_stamp":"2024-04-25T00:01:05.737842+0000","last_deep_scrub":"27'9062","last_deep_scrub_stamp":"2024-04-25T00:01:05.737842+0000","last_clean_scrub_stamp":"2024-04-25T00:01:05.737842+0000","objects_scrubbed":0,"log_size":7150,"log_dups_size":0,"ondisk_log_size":7150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:06:36.752974+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":744,"num_read_kb":188976,"num_write":6372,"num_write_kb":873040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3,6],"acting":[4,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"27'2607","reported_seq":2650,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:10.824429+0000","last_change":"2024-04-25T00:01:10.824450+0000","last_active":"2024-04-25T00:01:10.824429+0000","last_peered":"2024-04-25T00:01:10.824429+0000","last_clean":"2024-04-25T00:01:10.824429+0000","last_became_active":"2024-04-24T23:34:54.376512+0000","last_became_peered":"2024-04-24T23:34:54.376512+0000","last_unstale":"2024-04-25T00:01:10.824429+0000","last_undegraded":"2024-04-25T00:01:10.824429+0000","last_fullsized":"2024-04-25T00:01:10.824429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2607","last_scrub_stamp":"2024-04-25T00:01:10.824364+0000","last_deep_scrub":"27'2607","last_deep_scrub_stamp":"2024-04-25T00:01:10.824364+0000","last_clean_scrub_stamp":"2024-04-25T00:01:10.824364+0000","objects_scrubbed":0,"log_size":2607,"log_dups_size":0,"ondisk_log_size":2607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:13:28.370952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"27'4002","reported_seq":4048,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:57.022182+0000","last_change":"2024-04-25T00:00:57.022228+0000","last_active":"2024-04-25T00:00:57.022182+0000","last_peered":"2024-04-25T00:00:57.022182+0000","last_clean":"2024-04-25T00:00:57.022182+0000","last_became_active":"2024-04-24T23:34:53.354988+0000","last_became_peered":"2024-04-24T23:34:53.354988+0000","last_unstale":"2024-04-25T00:00:57.022182+0000","last_undegraded":"2024-04-25T00:00:57.022182+0000","last_fullsized":"2024-04-25T00:00:57.022182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4002","last_scrub_stamp":"2024-04-25T00:00:57.022120+0000","last_deep_scrub":"27'4002","last_deep_scrub_stamp":"2024-04-25T00:00:57.022120+0000","last_clean_scrub_stamp":"2024-04-25T00:00:57.022120+0000","objects_scrubbed":6,"log_size":4002,"log_dups_size":0,"ondisk_log_size":4002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:19:11.793888+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4001,"num_write_kb":39473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"27'9856","reported_seq":10237,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:07.042224+0000","last_change":"2024-04-25T00:01:07.042265+0000","last_active":"2024-04-25T00:01:07.042224+0000","last_peered":"2024-04-25T00:01:07.042224+0000","last_clean":"2024-04-25T00:01:07.042224+0000","last_became_active":"2024-04-24T23:34:58.398457+0000","last_became_peered":"2024-04-24T23:34:58.398457+0000","last_unstale":"2024-04-25T00:01:07.042224+0000","last_undegraded":"2024-04-25T00:01:07.042224+0000","last_fullsized":"2024-04-25T00:01:07.042224+0000","mapping_epoch":23,"log_start":"27'2619","ondisk_log_start":"27'2619","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9856","last_scrub_stamp":"2024-04-25T00:01:07.042125+0000","last_deep_scrub":"27'9856","last_deep_scrub_stamp":"2024-04-25T00:01:07.042125+0000","last_clean_scrub_stamp":"2024-04-25T00:01:07.042125+0000","objects_scrubbed":0,"log_size":7237,"log_dups_size":0,"ondisk_log_size":7237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:48.713288+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":344,"num_read_kb":87376,"num_write":7128,"num_write_kb":1170154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"27'2472","reported_seq":2516,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:02.176088+0000","last_change":"2024-04-25T00:01:02.176113+0000","last_active":"2024-04-25T00:01:02.176088+0000","last_peered":"2024-04-25T00:01:02.176088+0000","last_clean":"2024-04-25T00:01:02.176088+0000","last_became_active":"2024-04-24T23:34:54.371703+0000","last_became_peered":"2024-04-24T23:34:54.371703+0000","last_unstale":"2024-04-25T00:01:02.176088+0000","last_undegraded":"2024-04-25T00:01:02.176088+0000","last_fullsized":"2024-04-25T00:01:02.176088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2472","last_scrub_stamp":"2024-04-25T00:01:02.176002+0000","last_deep_scrub":"27'2472","last_deep_scrub_stamp":"2024-04-25T00:01:02.176002+0000","last_clean_scrub_stamp":"2024-04-25T00:01:02.176002+0000","objects_scrubbed":0,"log_size":2472,"log_dups_size":0,"ondisk_log_size":2472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:18:51.065456+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"27'4715","reported_seq":4761,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:59.279111+0000","last_change":"2024-04-25T00:00:59.279163+0000","last_active":"2024-04-25T00:00:59.279111+0000","last_peered":"2024-04-25T00:00:59.279111+0000","last_clean":"2024-04-25T00:00:59.279111+0000","last_became_active":"2024-04-24T23:34:53.363515+0000","last_became_peered":"2024-04-24T23:34:53.363515+0000","last_unstale":"2024-04-25T00:00:59.279111+0000","last_undegraded":"2024-04-25T00:00:59.279111+0000","last_fullsized":"2024-04-25T00:00:59.279111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4715","last_scrub_stamp":"2024-04-25T00:00:59.279022+0000","last_deep_scrub":"27'4715","last_deep_scrub_stamp":"2024-04-25T00:00:59.279022+0000","last_clean_scrub_stamp":"2024-04-25T00:00:59.279022+0000","objects_scrubbed":3,"log_size":4715,"log_dups_size":0,"ondisk_log_size":4715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:38:12.523634+0000","scrub_duration":55,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4705,"num_write_kb":43976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"27'9394","reported_seq":9798,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:11.316884+0000","last_change":"2024-04-25T00:01:11.316909+0000","last_active":"2024-04-25T00:01:11.316884+0000","last_peered":"2024-04-25T00:01:11.316884+0000","last_clean":"2024-04-25T00:01:11.316884+0000","last_became_active":"2024-04-24T23:34:58.727425+0000","last_became_peered":"2024-04-24T23:34:58.727425+0000","last_unstale":"2024-04-25T00:01:11.316884+0000","last_undegraded":"2024-04-25T00:01:11.316884+0000","last_fullsized":"2024-04-25T00:01:11.316884+0000","mapping_epoch":23,"log_start":"27'3135","ondisk_log_start":"27'3135","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9394","last_scrub_stamp":"2024-04-25T00:01:11.316797+0000","last_deep_scrub":"27'9394","last_deep_scrub_stamp":"2024-04-25T00:01:11.316797+0000","last_clean_scrub_stamp":"2024-04-25T00:01:11.316797+0000","objects_scrubbed":0,"log_size":6259,"log_dups_size":0,"ondisk_log_size":6259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:27.090436+0000","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":367,"num_read_kb":93218,"num_write":6677,"num_write_kb":924431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"27'2582","reported_seq":2625,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:12.849850+0000","last_change":"2024-04-25T00:01:12.849871+0000","last_active":"2024-04-25T00:01:12.849850+0000","last_peered":"2024-04-25T00:01:12.849850+0000","last_clean":"2024-04-25T00:01:12.849850+0000","last_became_active":"2024-04-24T23:34:54.379016+0000","last_became_peered":"2024-04-24T23:34:54.379016+0000","last_unstale":"2024-04-25T00:01:12.849850+0000","last_undegraded":"2024-04-25T00:01:12.849850+0000","last_fullsized":"2024-04-25T00:01:12.849850+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2582","last_scrub_stamp":"2024-04-25T00:01:12.849779+0000","last_deep_scrub":"27'2582","last_deep_scrub_stamp":"2024-04-25T00:01:12.849779+0000","last_clean_scrub_stamp":"2024-04-25T00:01:12.849779+0000","objects_scrubbed":0,"log_size":2582,"log_dups_size":0,"ondisk_log_size":2582,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:24.897738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"27'2768","reported_seq":2814,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:59.985281+0000","last_change":"2024-04-25T00:00:59.985307+0000","last_active":"2024-04-25T00:00:59.985281+0000","last_peered":"2024-04-25T00:00:59.985281+0000","last_clean":"2024-04-25T00:00:59.985281+0000","last_became_active":"2024-04-24T23:34:53.359581+0000","last_became_peered":"2024-04-24T23:34:53.359581+0000","last_unstale":"2024-04-25T00:00:59.985281+0000","last_undegraded":"2024-04-25T00:00:59.985281+0000","last_fullsized":"2024-04-25T00:00:59.985281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2768","last_scrub_stamp":"2024-04-25T00:00:59.985218+0000","last_deep_scrub":"27'2768","last_deep_scrub_stamp":"2024-04-25T00:00:59.985218+0000","last_clean_scrub_stamp":"2024-04-25T00:00:59.985218+0000","objects_scrubbed":2,"log_size":2768,"log_dups_size":0,"ondisk_log_size":2768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:08:07.821723+0000","scrub_duration":29,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2773,"num_write_kb":30370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"27'9089","reported_seq":9568,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:12.295066+0000","last_change":"2024-04-25T00:01:12.295091+0000","last_active":"2024-04-25T00:01:12.295066+0000","last_peered":"2024-04-25T00:01:12.295066+0000","last_clean":"2024-04-25T00:01:12.295066+0000","last_became_active":"2024-04-24T23:34:58.727686+0000","last_became_peered":"2024-04-24T23:34:58.727686+0000","last_unstale":"2024-04-25T00:01:12.295066+0000","last_undegraded":"2024-04-25T00:01:12.295066+0000","last_fullsized":"2024-04-25T00:01:12.295066+0000","mapping_epoch":23,"log_start":"27'2830","ondisk_log_start":"27'2830","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9089","last_scrub_stamp":"2024-04-25T00:01:12.294980+0000","last_deep_scrub":"27'9089","last_deep_scrub_stamp":"2024-04-25T00:01:12.294980+0000","last_clean_scrub_stamp":"2024-04-25T00:01:12.294980+0000","objects_scrubbed":0,"log_size":6259,"log_dups_size":0,"ondisk_log_size":6259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:30.991247+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":441,"num_read_kb":112014,"num_write":6536,"num_write_kb":907183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"27'2539","reported_seq":2583,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:02.980632+0000","last_change":"2024-04-25T00:01:02.980653+0000","last_active":"2024-04-25T00:01:02.980632+0000","last_peered":"2024-04-25T00:01:02.980632+0000","last_clean":"2024-04-25T00:01:02.980632+0000","last_became_active":"2024-04-24T23:34:54.371720+0000","last_became_peered":"2024-04-24T23:34:54.371720+0000","last_unstale":"2024-04-25T00:01:02.980632+0000","last_undegraded":"2024-04-25T00:01:02.980632+0000","last_fullsized":"2024-04-25T00:01:02.980632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2539","last_scrub_stamp":"2024-04-25T00:01:02.980564+0000","last_deep_scrub":"27'2539","last_deep_scrub_stamp":"2024-04-25T00:01:02.980564+0000","last_clean_scrub_stamp":"2024-04-25T00:01:02.980564+0000","objects_scrubbed":0,"log_size":2539,"log_dups_size":0,"ondisk_log_size":2539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:42:18.938818+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"27'6538","reported_seq":6583,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:56.140250+0000","last_change":"2024-04-25T00:00:56.140323+0000","last_active":"2024-04-25T00:00:56.140250+0000","last_peered":"2024-04-25T00:00:56.140250+0000","last_clean":"2024-04-25T00:00:56.140250+0000","last_became_active":"2024-04-24T23:34:53.353452+0000","last_became_peered":"2024-04-24T23:34:53.353452+0000","last_unstale":"2024-04-25T00:00:56.140250+0000","last_undegraded":"2024-04-25T00:00:56.140250+0000","last_fullsized":"2024-04-25T00:00:56.140250+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6538","last_scrub_stamp":"2024-04-25T00:00:56.140125+0000","last_deep_scrub":"27'6538","last_deep_scrub_stamp":"2024-04-25T00:00:56.140125+0000","last_clean_scrub_stamp":"2024-04-25T00:00:56.140125+0000","objects_scrubbed":7,"log_size":6538,"log_dups_size":0,"ondisk_log_size":6538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:02:42.997386+0000","scrub_duration":75,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971554,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6522,"num_write_kb":57515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"27'9615","reported_seq":10073,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:10.881545+0000","last_change":"2024-04-25T00:01:10.881571+0000","last_active":"2024-04-25T00:01:10.881545+0000","last_peered":"2024-04-25T00:01:10.881545+0000","last_clean":"2024-04-25T00:01:10.881545+0000","last_became_active":"2024-04-24T23:34:58.727921+0000","last_became_peered":"2024-04-24T23:34:58.727921+0000","last_unstale":"2024-04-25T00:01:10.881545+0000","last_undegraded":"2024-04-25T00:01:10.881545+0000","last_fullsized":"2024-04-25T00:01:10.881545+0000","mapping_epoch":23,"log_start":"27'1833","ondisk_log_start":"27'1833","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9615","last_scrub_stamp":"2024-04-25T00:01:10.881459+0000","last_deep_scrub":"27'9615","last_deep_scrub_stamp":"2024-04-25T00:01:10.881459+0000","last_clean_scrub_stamp":"2024-04-25T00:01:10.881459+0000","objects_scrubbed":0,"log_size":7782,"log_dups_size":0,"ondisk_log_size":7782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:04:35.139000+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":421,"num_read_kb":106934,"num_write":7001,"num_write_kb":1058941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"27'2478","reported_seq":2520,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:59.038378+0000","last_change":"2024-04-25T00:00:59.038416+0000","last_active":"2024-04-25T00:00:59.038378+0000","last_peered":"2024-04-25T00:00:59.038378+0000","last_clean":"2024-04-25T00:00:59.038378+0000","last_became_active":"2024-04-24T23:34:54.367903+0000","last_became_peered":"2024-04-24T23:34:54.367903+0000","last_unstale":"2024-04-25T00:00:59.038378+0000","last_undegraded":"2024-04-25T00:00:59.038378+0000","last_fullsized":"2024-04-25T00:00:59.038378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2478","last_scrub_stamp":"2024-04-25T00:00:59.038288+0000","last_deep_scrub":"27'2478","last_deep_scrub_stamp":"2024-04-25T00:00:59.038288+0000","last_clean_scrub_stamp":"2024-04-25T00:00:59.038288+0000","objects_scrubbed":0,"log_size":2478,"log_dups_size":0,"ondisk_log_size":2478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:10:54.490651+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'9572","reported_seq":10219,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:58.284915+0000","last_change":"2024-04-25T00:00:58.284949+0000","last_active":"2024-04-25T00:00:58.284915+0000","last_peered":"2024-04-25T00:00:58.284915+0000","last_clean":"2024-04-25T00:00:58.284915+0000","last_became_active":"2024-04-24T23:34:53.362230+0000","last_became_peered":"2024-04-24T23:34:53.362230+0000","last_unstale":"2024-04-25T00:00:58.284915+0000","last_undegraded":"2024-04-25T00:00:58.284915+0000","last_fullsized":"2024-04-25T00:00:58.284915+0000","mapping_epoch":18,"log_start":"27'1600","ondisk_log_start":"27'1600","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9572","last_scrub_stamp":"2024-04-25T00:00:58.284856+0000","last_deep_scrub":"27'9572","last_deep_scrub_stamp":"2024-04-25T00:00:58.284856+0000","last_clean_scrub_stamp":"2024-04-25T00:00:58.284856+0000","objects_scrubbed":5,"log_size":7972,"log_dups_size":0,"ondisk_log_size":7972,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:02:45.142117+0000","scrub_duration":36,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389280,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":602,"num_read_kb":4494,"num_write":9568,"num_write_kb":76044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"27'9012","reported_seq":9427,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:07.123038+0000","last_change":"2024-04-25T00:01:07.123063+0000","last_active":"2024-04-25T00:01:07.123038+0000","last_peered":"2024-04-25T00:01:07.123038+0000","last_clean":"2024-04-25T00:01:07.123038+0000","last_became_active":"2024-04-24T23:34:58.400429+0000","last_became_peered":"2024-04-24T23:34:58.400429+0000","last_unstale":"2024-04-25T00:01:07.123038+0000","last_undegraded":"2024-04-25T00:01:07.123038+0000","last_fullsized":"2024-04-25T00:01:07.123038+0000","mapping_epoch":23,"log_start":"27'1625","ondisk_log_start":"27'1625","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9012","last_scrub_stamp":"2024-04-25T00:01:07.122949+0000","last_deep_scrub":"27'9012","last_deep_scrub_stamp":"2024-04-25T00:01:07.122949+0000","last_clean_scrub_stamp":"2024-04-25T00:01:07.122949+0000","objects_scrubbed":0,"log_size":7387,"log_dups_size":0,"ondisk_log_size":7387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:57.787587+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":377,"num_read_kb":95758,"num_write":6342,"num_write_kb":955265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"27'2558","reported_seq":2602,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:00.661276+0000","last_change":"2024-04-25T00:01:00.661297+0000","last_active":"2024-04-25T00:01:00.661276+0000","last_peered":"2024-04-25T00:01:00.661276+0000","last_clean":"2024-04-25T00:01:00.661276+0000","last_became_active":"2024-04-24T23:34:54.373347+0000","last_became_peered":"2024-04-24T23:34:54.373347+0000","last_unstale":"2024-04-25T00:01:00.661276+0000","last_undegraded":"2024-04-25T00:01:00.661276+0000","last_fullsized":"2024-04-25T00:01:00.661276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2558","last_scrub_stamp":"2024-04-25T00:01:00.661203+0000","last_deep_scrub":"27'2558","last_deep_scrub_stamp":"2024-04-25T00:01:00.661203+0000","last_clean_scrub_stamp":"2024-04-25T00:01:00.661203+0000","objects_scrubbed":0,"log_size":2558,"log_dups_size":0,"ondisk_log_size":2558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:59:35.495255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"27'5926","reported_seq":5971,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:58.078866+0000","last_change":"2024-04-25T00:00:58.078892+0000","last_active":"2024-04-25T00:00:58.078866+0000","last_peered":"2024-04-25T00:00:58.078866+0000","last_clean":"2024-04-25T00:00:58.078866+0000","last_became_active":"2024-04-24T23:34:53.354394+0000","last_became_peered":"2024-04-24T23:34:53.354394+0000","last_unstale":"2024-04-25T00:00:58.078866+0000","last_undegraded":"2024-04-25T00:00:58.078866+0000","last_fullsized":"2024-04-25T00:00:58.078866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5926","last_scrub_stamp":"2024-04-25T00:00:58.078802+0000","last_deep_scrub":"27'5926","last_deep_scrub_stamp":"2024-04-25T00:00:58.078802+0000","last_clean_scrub_stamp":"2024-04-25T00:00:58.078802+0000","objects_scrubbed":2,"log_size":5926,"log_dups_size":0,"ondisk_log_size":5926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:31:13.070467+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5926,"num_write_kb":41236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"27'9157","reported_seq":9573,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:03.711930+0000","last_change":"2024-04-25T00:01:03.711976+0000","last_active":"2024-04-25T00:01:03.711930+0000","last_peered":"2024-04-25T00:01:03.711930+0000","last_clean":"2024-04-25T00:01:03.711930+0000","last_became_active":"2024-04-24T23:34:58.397353+0000","last_became_peered":"2024-04-24T23:34:58.397353+0000","last_unstale":"2024-04-25T00:01:03.711930+0000","last_undegraded":"2024-04-25T00:01:03.711930+0000","last_fullsized":"2024-04-25T00:01:03.711930+0000","mapping_epoch":23,"log_start":"27'2001","ondisk_log_start":"27'2001","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9157","last_scrub_stamp":"2024-04-25T00:01:03.711783+0000","last_deep_scrub":"27'9157","last_deep_scrub_stamp":"2024-04-25T00:01:03.711783+0000","last_clean_scrub_stamp":"2024-04-25T00:01:03.711783+0000","objects_scrubbed":0,"log_size":7156,"log_dups_size":0,"ondisk_log_size":7156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:45:03.473492+0000","scrub_duration":29,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":378,"num_read_kb":96012,"num_write":6474,"num_write_kb":975733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"27'2584","reported_seq":2627,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:05.001006+0000","last_change":"2024-04-25T00:01:05.001044+0000","last_active":"2024-04-25T00:01:05.001006+0000","last_peered":"2024-04-25T00:01:05.001006+0000","last_clean":"2024-04-25T00:01:05.001006+0000","last_became_active":"2024-04-24T23:34:54.383082+0000","last_became_peered":"2024-04-24T23:34:54.383082+0000","last_unstale":"2024-04-25T00:01:05.001006+0000","last_undegraded":"2024-04-25T00:01:05.001006+0000","last_fullsized":"2024-04-25T00:01:05.001006+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2584","last_scrub_stamp":"2024-04-25T00:01:05.000905+0000","last_deep_scrub":"27'2584","last_deep_scrub_stamp":"2024-04-25T00:01:05.000905+0000","last_clean_scrub_stamp":"2024-04-25T00:01:05.000905+0000","objects_scrubbed":0,"log_size":2584,"log_dups_size":0,"ondisk_log_size":2584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:31.816773+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"27'7103","reported_seq":7148,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:00.991905+0000","last_change":"2024-04-25T00:01:00.991975+0000","last_active":"2024-04-25T00:01:00.991905+0000","last_peered":"2024-04-25T00:01:00.991905+0000","last_clean":"2024-04-25T00:01:00.991905+0000","last_became_active":"2024-04-24T23:34:53.364259+0000","last_became_peered":"2024-04-24T23:34:53.364259+0000","last_unstale":"2024-04-25T00:01:00.991905+0000","last_undegraded":"2024-04-25T00:01:00.991905+0000","last_fullsized":"2024-04-25T00:01:00.991905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7103","last_scrub_stamp":"2024-04-25T00:01:00.991796+0000","last_deep_scrub":"27'7103","last_deep_scrub_stamp":"2024-04-25T00:01:00.991796+0000","last_clean_scrub_stamp":"2024-04-25T00:01:00.991796+0000","objects_scrubbed":2,"log_size":7103,"log_dups_size":0,"ondisk_log_size":7103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:45:00.753505+0000","scrub_duration":45,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":7102,"num_write_kb":53865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"27'9038","reported_seq":9145,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:09.256807+0000","last_change":"2024-04-25T00:01:09.256870+0000","last_active":"2024-04-25T00:01:09.256807+0000","last_peered":"2024-04-25T00:01:09.256807+0000","last_clean":"2024-04-25T00:01:09.256807+0000","last_became_active":"2024-04-24T23:34:58.405032+0000","last_became_peered":"2024-04-24T23:34:58.405032+0000","last_unstale":"2024-04-25T00:01:09.256807+0000","last_undegraded":"2024-04-25T00:01:09.256807+0000","last_fullsized":"2024-04-25T00:01:09.256807+0000","mapping_epoch":23,"log_start":"27'2712","ondisk_log_start":"27'2712","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9038","last_scrub_stamp":"2024-04-25T00:01:09.256679+0000","last_deep_scrub":"27'9038","last_deep_scrub_stamp":"2024-04-25T00:01:09.256679+0000","last_clean_scrub_stamp":"2024-04-25T00:01:09.256679+0000","objects_scrubbed":0,"log_size":6326,"log_dups_size":0,"ondisk_log_size":6326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:56.779919+0000","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":17526,"num_write":6404,"num_write_kb":965810,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"27'2558","reported_seq":2601,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:08.887028+0000","last_change":"2024-04-25T00:01:08.887064+0000","last_active":"2024-04-25T00:01:08.887028+0000","last_peered":"2024-04-25T00:01:08.887028+0000","last_clean":"2024-04-25T00:01:08.887028+0000","last_became_active":"2024-04-24T23:34:54.383354+0000","last_became_peered":"2024-04-24T23:34:54.383354+0000","last_unstale":"2024-04-25T00:01:08.887028+0000","last_undegraded":"2024-04-25T00:01:08.887028+0000","last_fullsized":"2024-04-25T00:01:08.887028+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2558","last_scrub_stamp":"2024-04-25T00:01:08.886937+0000","last_deep_scrub":"27'2558","last_deep_scrub_stamp":"2024-04-25T00:01:08.886937+0000","last_clean_scrub_stamp":"2024-04-25T00:01:08.886937+0000","objects_scrubbed":0,"log_size":2558,"log_dups_size":0,"ondisk_log_size":2558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:15:13.290766+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"27'6418","reported_seq":7309,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:57.253355+0000","last_change":"2024-04-25T00:00:57.253393+0000","last_active":"2024-04-25T00:00:57.253355+0000","last_peered":"2024-04-25T00:00:57.253355+0000","last_clean":"2024-04-25T00:00:57.253355+0000","last_became_active":"2024-04-24T23:34:53.362103+0000","last_became_peered":"2024-04-24T23:34:53.362103+0000","last_unstale":"2024-04-25T00:00:57.253355+0000","last_undegraded":"2024-04-25T00:00:57.253355+0000","last_fullsized":"2024-04-25T00:00:57.253355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6418","last_scrub_stamp":"2024-04-25T00:00:57.253246+0000","last_deep_scrub":"27'6418","last_deep_scrub_stamp":"2024-04-25T00:00:57.253246+0000","last_clean_scrub_stamp":"2024-04-25T00:00:57.253246+0000","objects_scrubbed":0,"log_size":6418,"log_dups_size":0,"ondisk_log_size":6418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:49:16.211875+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":847,"num_read_kb":4657,"num_write":6417,"num_write_kb":49337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"27'9416","reported_seq":9713,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:14.351499+0000","last_change":"2024-04-25T00:01:14.351523+0000","last_active":"2024-04-25T00:01:14.351499+0000","last_peered":"2024-04-25T00:01:14.351499+0000","last_clean":"2024-04-25T00:01:14.351499+0000","last_became_active":"2024-04-24T23:34:58.729217+0000","last_became_peered":"2024-04-24T23:34:58.729217+0000","last_unstale":"2024-04-25T00:01:14.351499+0000","last_undegraded":"2024-04-25T00:01:14.351499+0000","last_fullsized":"2024-04-25T00:01:14.351499+0000","mapping_epoch":23,"log_start":"27'3115","ondisk_log_start":"27'3115","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9416","last_scrub_stamp":"2024-04-25T00:01:14.351384+0000","last_deep_scrub":"27'9416","last_deep_scrub_stamp":"2024-04-25T00:01:14.351384+0000","last_clean_scrub_stamp":"2024-04-25T00:01:14.351384+0000","objects_scrubbed":0,"log_size":6301,"log_dups_size":0,"ondisk_log_size":6301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:05:00.012542+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":259,"num_read_kb":65786,"num_write":6628,"num_write_kb":1056336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"27'9147","reported_seq":9286,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:10.982447+0000","last_change":"2024-04-25T00:01:10.982484+0000","last_active":"2024-04-25T00:01:10.982447+0000","last_peered":"2024-04-25T00:01:10.982447+0000","last_clean":"2024-04-25T00:01:10.982447+0000","last_became_active":"2024-04-24T23:34:58.727914+0000","last_became_peered":"2024-04-24T23:34:58.727914+0000","last_unstale":"2024-04-25T00:01:10.982447+0000","last_undegraded":"2024-04-25T00:01:10.982447+0000","last_fullsized":"2024-04-25T00:01:10.982447+0000","mapping_epoch":23,"log_start":"27'1931","ondisk_log_start":"27'1931","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9147","last_scrub_stamp":"2024-04-25T00:01:10.982317+0000","last_deep_scrub":"27'9147","last_deep_scrub_stamp":"2024-04-25T00:01:10.982317+0000","last_clean_scrub_stamp":"2024-04-25T00:01:10.982317+0000","objects_scrubbed":0,"log_size":7216,"log_dups_size":0,"ondisk_log_size":7216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:19:22.482471+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":99,"num_read_kb":25146,"num_write":6551,"num_write_kb":920441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"27'2499","reported_seq":2542,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.125218+0000","last_change":"2024-04-25T00:00:55.125262+0000","last_active":"2024-04-25T00:00:55.125218+0000","last_peered":"2024-04-25T00:00:55.125218+0000","last_clean":"2024-04-25T00:00:55.125218+0000","last_became_active":"2024-04-24T23:34:54.363724+0000","last_became_peered":"2024-04-24T23:34:54.363724+0000","last_unstale":"2024-04-25T00:00:55.125218+0000","last_undegraded":"2024-04-25T00:00:55.125218+0000","last_fullsized":"2024-04-25T00:00:55.125218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2499","last_scrub_stamp":"2024-04-25T00:00:55.125079+0000","last_deep_scrub":"27'2499","last_deep_scrub_stamp":"2024-04-25T00:00:55.125079+0000","last_clean_scrub_stamp":"2024-04-25T00:00:55.125079+0000","objects_scrubbed":0,"log_size":2499,"log_dups_size":0,"ondisk_log_size":2499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:41.142244+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"27'9688","reported_seq":9733,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:58.982139+0000","last_change":"2024-04-25T00:00:58.982186+0000","last_active":"2024-04-25T00:00:58.982139+0000","last_peered":"2024-04-25T00:00:58.982139+0000","last_clean":"2024-04-25T00:00:58.982139+0000","last_became_active":"2024-04-24T23:34:53.359707+0000","last_became_peered":"2024-04-24T23:34:53.359707+0000","last_unstale":"2024-04-25T00:00:58.982139+0000","last_undegraded":"2024-04-25T00:00:58.982139+0000","last_fullsized":"2024-04-25T00:00:58.982139+0000","mapping_epoch":18,"log_start":"27'1900","ondisk_log_start":"27'1900","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9688","last_scrub_stamp":"2024-04-25T00:00:58.982041+0000","last_deep_scrub":"27'9688","last_deep_scrub_stamp":"2024-04-25T00:00:58.982041+0000","last_clean_scrub_stamp":"2024-04-25T00:00:58.982041+0000","objects_scrubbed":1,"log_size":7788,"log_dups_size":0,"ondisk_log_size":7788,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:59:04.463251+0000","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9677,"num_write_kb":67972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"27'9609","reported_seq":10709,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:11.120915+0000","last_change":"2024-04-25T00:01:11.120940+0000","last_active":"2024-04-25T00:01:11.120915+0000","last_peered":"2024-04-25T00:01:11.120915+0000","last_clean":"2024-04-25T00:01:11.120915+0000","last_became_active":"2024-04-24T23:35:43.834066+0000","last_became_peered":"2024-04-24T23:35:43.834066+0000","last_unstale":"2024-04-25T00:01:11.120915+0000","last_undegraded":"2024-04-25T00:01:11.120915+0000","last_fullsized":"2024-04-25T00:01:11.120915+0000","mapping_epoch":26,"log_start":"27'1708","ondisk_log_start":"27'1708","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9609","last_scrub_stamp":"2024-04-25T00:01:11.120817+0000","last_deep_scrub":"27'9609","last_deep_scrub_stamp":"2024-04-25T00:01:11.120817+0000","last_clean_scrub_stamp":"2024-04-25T00:01:11.120817+0000","objects_scrubbed":0,"log_size":7901,"log_dups_size":0,"ondisk_log_size":7901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:47.802601+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1054,"num_read_kb":267716,"num_write":6908,"num_write_kb":899799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2,0],"acting":[6,5,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"27'2603","reported_seq":2646,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:57.268395+0000","last_change":"2024-04-25T00:00:57.268434+0000","last_active":"2024-04-25T00:00:57.268395+0000","last_peered":"2024-04-25T00:00:57.268395+0000","last_clean":"2024-04-25T00:00:57.268395+0000","last_became_active":"2024-04-24T23:34:54.369155+0000","last_became_peered":"2024-04-24T23:34:54.369155+0000","last_unstale":"2024-04-25T00:00:57.268395+0000","last_undegraded":"2024-04-25T00:00:57.268395+0000","last_fullsized":"2024-04-25T00:00:57.268395+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2603","last_scrub_stamp":"2024-04-25T00:00:57.268303+0000","last_deep_scrub":"27'2603","last_deep_scrub_stamp":"2024-04-25T00:00:57.268303+0000","last_clean_scrub_stamp":"2024-04-25T00:00:57.268303+0000","objects_scrubbed":0,"log_size":2603,"log_dups_size":0,"ondisk_log_size":2603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:36:57.590443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"27'4846","reported_seq":4891,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:02.278021+0000","last_change":"2024-04-25T00:01:02.278048+0000","last_active":"2024-04-25T00:01:02.278021+0000","last_peered":"2024-04-25T00:01:02.278021+0000","last_clean":"2024-04-25T00:01:02.278021+0000","last_became_active":"2024-04-24T23:34:53.365029+0000","last_became_peered":"2024-04-24T23:34:53.365029+0000","last_unstale":"2024-04-25T00:01:02.278021+0000","last_undegraded":"2024-04-25T00:01:02.278021+0000","last_fullsized":"2024-04-25T00:01:02.278021+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4846","last_scrub_stamp":"2024-04-25T00:01:02.277955+0000","last_deep_scrub":"27'4846","last_deep_scrub_stamp":"2024-04-25T00:01:02.277955+0000","last_clean_scrub_stamp":"2024-04-25T00:01:02.277955+0000","objects_scrubbed":2,"log_size":4846,"log_dups_size":0,"ondisk_log_size":4846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:17:52.774906+0000","scrub_duration":30,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4849,"num_write_kb":44132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"27'9229","reported_seq":9274,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:00.249412+0000","last_change":"2024-04-25T00:01:00.249459+0000","last_active":"2024-04-25T00:01:00.249412+0000","last_peered":"2024-04-25T00:01:00.249412+0000","last_clean":"2024-04-25T00:01:00.249412+0000","last_became_active":"2024-04-24T23:34:53.364821+0000","last_became_peered":"2024-04-24T23:34:53.364821+0000","last_unstale":"2024-04-25T00:01:00.249412+0000","last_undegraded":"2024-04-25T00:01:00.249412+0000","last_fullsized":"2024-04-25T00:01:00.249412+0000","mapping_epoch":18,"log_start":"27'1300","ondisk_log_start":"27'1300","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9229","last_scrub_stamp":"2024-04-25T00:01:00.249298+0000","last_deep_scrub":"27'9229","last_deep_scrub_stamp":"2024-04-25T00:01:00.249298+0000","last_clean_scrub_stamp":"2024-04-25T00:01:00.249298+0000","objects_scrubbed":1,"log_size":7929,"log_dups_size":0,"ondisk_log_size":7929,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:15.667297+0000","scrub_duration":29,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9220,"num_write_kb":71599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"27'2536","reported_seq":2579,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.053278+0000","last_change":"2024-04-25T00:00:55.053300+0000","last_active":"2024-04-25T00:00:55.053278+0000","last_peered":"2024-04-25T00:00:55.053278+0000","last_clean":"2024-04-25T00:00:55.053278+0000","last_became_active":"2024-04-24T23:34:54.375689+0000","last_became_peered":"2024-04-24T23:34:54.375689+0000","last_unstale":"2024-04-25T00:00:55.053278+0000","last_undegraded":"2024-04-25T00:00:55.053278+0000","last_fullsized":"2024-04-25T00:00:55.053278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2536","last_scrub_stamp":"2024-04-25T00:00:55.053203+0000","last_deep_scrub":"27'2536","last_deep_scrub_stamp":"2024-04-25T00:00:55.053203+0000","last_clean_scrub_stamp":"2024-04-25T00:00:55.053203+0000","objects_scrubbed":0,"log_size":2536,"log_dups_size":0,"ondisk_log_size":2536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:49:14.011832+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"27'9729","reported_seq":9962,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:04.048851+0000","last_change":"2024-04-25T00:01:04.048875+0000","last_active":"2024-04-25T00:01:04.048851+0000","last_peered":"2024-04-25T00:01:04.048851+0000","last_clean":"2024-04-25T00:01:04.048851+0000","last_became_active":"2024-04-24T23:35:43.834061+0000","last_became_peered":"2024-04-24T23:35:43.834061+0000","last_unstale":"2024-04-25T00:01:04.048851+0000","last_undegraded":"2024-04-25T00:01:04.048851+0000","last_fullsized":"2024-04-25T00:01:04.048851+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9729","last_scrub_stamp":"2024-04-25T00:01:04.048769+0000","last_deep_scrub":"27'9729","last_deep_scrub_stamp":"2024-04-25T00:01:04.048769+0000","last_clean_scrub_stamp":"2024-04-25T00:01:04.048769+0000","objects_scrubbed":0,"log_size":9729,"log_dups_size":0,"ondisk_log_size":9729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:15:02.110418+0000","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":189,"num_read_kb":48006,"num_write":6998,"num_write_kb":973944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3,5],"acting":[2,0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"27'7147","reported_seq":7192,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:01.276351+0000","last_change":"2024-04-25T00:01:01.276399+0000","last_active":"2024-04-25T00:01:01.276351+0000","last_peered":"2024-04-25T00:01:01.276351+0000","last_clean":"2024-04-25T00:01:01.276351+0000","last_became_active":"2024-04-24T23:34:53.362548+0000","last_became_peered":"2024-04-24T23:34:53.362548+0000","last_unstale":"2024-04-25T00:01:01.276351+0000","last_undegraded":"2024-04-25T00:01:01.276351+0000","last_fullsized":"2024-04-25T00:01:01.276351+0000","mapping_epoch":18,"log_start":"27'800","ondisk_log_start":"27'800","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7147","last_scrub_stamp":"2024-04-25T00:01:01.276272+0000","last_deep_scrub":"27'7147","last_deep_scrub_stamp":"2024-04-25T00:01:01.276272+0000","last_clean_scrub_stamp":"2024-04-25T00:01:01.276272+0000","objects_scrubbed":0,"log_size":6347,"log_dups_size":0,"ondisk_log_size":6347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:49:20.637238+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7134,"num_write_kb":66117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"27'2573","reported_seq":2616,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:58.745714+0000","last_change":"2024-04-25T00:00:58.745751+0000","last_active":"2024-04-25T00:00:58.745714+0000","last_peered":"2024-04-25T00:00:58.745714+0000","last_clean":"2024-04-25T00:00:58.745714+0000","last_became_active":"2024-04-24T23:34:54.364814+0000","last_became_peered":"2024-04-24T23:34:54.364814+0000","last_unstale":"2024-04-25T00:00:58.745714+0000","last_undegraded":"2024-04-25T00:00:58.745714+0000","last_fullsized":"2024-04-25T00:00:58.745714+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2573","last_scrub_stamp":"2024-04-25T00:00:58.745637+0000","last_deep_scrub":"27'2573","last_deep_scrub_stamp":"2024-04-25T00:00:58.745637+0000","last_clean_scrub_stamp":"2024-04-25T00:00:58.745637+0000","objects_scrubbed":0,"log_size":2573,"log_dups_size":0,"ondisk_log_size":2573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:38:11.990248+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"27'9398","reported_seq":9613,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:10.294046+0000","last_change":"2024-04-25T00:01:10.294070+0000","last_active":"2024-04-25T00:01:10.294046+0000","last_peered":"2024-04-25T00:01:10.294046+0000","last_clean":"2024-04-25T00:01:10.294046+0000","last_became_active":"2024-04-24T23:34:58.406811+0000","last_became_peered":"2024-04-24T23:34:58.406811+0000","last_unstale":"2024-04-25T00:01:10.294046+0000","last_undegraded":"2024-04-25T00:01:10.294046+0000","last_fullsized":"2024-04-25T00:01:10.294046+0000","mapping_epoch":23,"log_start":"27'3112","ondisk_log_start":"27'3112","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9398","last_scrub_stamp":"2024-04-25T00:01:10.293962+0000","last_deep_scrub":"27'9398","last_deep_scrub_stamp":"2024-04-25T00:01:10.293962+0000","last_clean_scrub_stamp":"2024-04-25T00:01:10.293962+0000","objects_scrubbed":0,"log_size":6286,"log_dups_size":0,"ondisk_log_size":6286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:53.391877+0000","scrub_duration":41,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":178,"num_read_kb":45212,"num_write":6641,"num_write_kb":1045709,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"27'2471","reported_seq":2516,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:03.159347+0000","last_change":"2024-04-25T00:01:03.159369+0000","last_active":"2024-04-25T00:01:03.159347+0000","last_peered":"2024-04-25T00:01:03.159347+0000","last_clean":"2024-04-25T00:01:03.159347+0000","last_became_active":"2024-04-24T23:34:54.374844+0000","last_became_peered":"2024-04-24T23:34:54.374844+0000","last_unstale":"2024-04-25T00:01:03.159347+0000","last_undegraded":"2024-04-25T00:01:03.159347+0000","last_fullsized":"2024-04-25T00:01:03.159347+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2471","last_scrub_stamp":"2024-04-25T00:01:03.159211+0000","last_deep_scrub":"27'2471","last_deep_scrub_stamp":"2024-04-25T00:01:03.159211+0000","last_clean_scrub_stamp":"2024-04-25T00:01:03.159211+0000","objects_scrubbed":0,"log_size":2471,"log_dups_size":0,"ondisk_log_size":2471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:08:10.995716+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"27'2599","reported_seq":2644,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:00.987482+0000","last_change":"2024-04-25T00:01:00.987505+0000","last_active":"2024-04-25T00:01:00.987482+0000","last_peered":"2024-04-25T00:01:00.987482+0000","last_clean":"2024-04-25T00:01:00.987482+0000","last_became_active":"2024-04-24T23:34:54.383253+0000","last_became_peered":"2024-04-24T23:34:54.383253+0000","last_unstale":"2024-04-25T00:01:00.987482+0000","last_undegraded":"2024-04-25T00:01:00.987482+0000","last_fullsized":"2024-04-25T00:01:00.987482+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2599","last_scrub_stamp":"2024-04-25T00:01:00.987360+0000","last_deep_scrub":"27'2599","last_deep_scrub_stamp":"2024-04-25T00:01:00.987360+0000","last_clean_scrub_stamp":"2024-04-25T00:01:00.987360+0000","objects_scrubbed":0,"log_size":2599,"log_dups_size":0,"ondisk_log_size":2599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:44:48.841627+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"27'2500","reported_seq":2531,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:03.000975+0000","last_change":"2024-04-25T00:01:03.000997+0000","last_active":"2024-04-25T00:01:03.000975+0000","last_peered":"2024-04-25T00:01:03.000975+0000","last_clean":"2024-04-25T00:01:03.000975+0000","last_became_active":"2024-04-24T23:35:43.525929+0000","last_became_peered":"2024-04-24T23:35:43.525929+0000","last_unstale":"2024-04-25T00:01:03.000975+0000","last_undegraded":"2024-04-25T00:01:03.000975+0000","last_fullsized":"2024-04-25T00:01:03.000975+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2500","last_scrub_stamp":"2024-04-25T00:01:03.000914+0000","last_deep_scrub":"27'2500","last_deep_scrub_stamp":"2024-04-25T00:01:03.000914+0000","last_clean_scrub_stamp":"2024-04-25T00:01:03.000914+0000","objects_scrubbed":0,"log_size":2500,"log_dups_size":0,"ondisk_log_size":2500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:32:27.297166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"27'2582","reported_seq":2627,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:59.059928+0000","last_change":"2024-04-25T00:00:59.060129+0000","last_active":"2024-04-25T00:00:59.059928+0000","last_peered":"2024-04-25T00:00:59.059928+0000","last_clean":"2024-04-25T00:00:59.059928+0000","last_became_active":"2024-04-24T23:34:54.366005+0000","last_became_peered":"2024-04-24T23:34:54.366005+0000","last_unstale":"2024-04-25T00:00:59.059928+0000","last_undegraded":"2024-04-25T00:00:59.059928+0000","last_fullsized":"2024-04-25T00:00:59.059928+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2582","last_scrub_stamp":"2024-04-25T00:00:59.059802+0000","last_deep_scrub":"27'2582","last_deep_scrub_stamp":"2024-04-25T00:00:59.059802+0000","last_clean_scrub_stamp":"2024-04-25T00:00:59.059802+0000","objects_scrubbed":0,"log_size":2582,"log_dups_size":0,"ondisk_log_size":2582,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:59:33.893854+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"27'2585","reported_seq":2630,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:03.237339+0000","last_change":"2024-04-25T00:01:03.237381+0000","last_active":"2024-04-25T00:01:03.237339+0000","last_peered":"2024-04-25T00:01:03.237339+0000","last_clean":"2024-04-25T00:01:03.237339+0000","last_became_active":"2024-04-24T23:34:54.360266+0000","last_became_peered":"2024-04-24T23:34:54.360266+0000","last_unstale":"2024-04-25T00:01:03.237339+0000","last_undegraded":"2024-04-25T00:01:03.237339+0000","last_fullsized":"2024-04-25T00:01:03.237339+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2585","last_scrub_stamp":"2024-04-25T00:01:03.237248+0000","last_deep_scrub":"27'2585","last_deep_scrub_stamp":"2024-04-25T00:01:03.237248+0000","last_clean_scrub_stamp":"2024-04-25T00:01:03.237248+0000","objects_scrubbed":0,"log_size":2585,"log_dups_size":0,"ondisk_log_size":2585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:30:45.427659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"27'2453","reported_seq":2498,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:05.107266+0000","last_change":"2024-04-25T00:01:05.107305+0000","last_active":"2024-04-25T00:01:05.107266+0000","last_peered":"2024-04-25T00:01:05.107266+0000","last_clean":"2024-04-25T00:01:05.107266+0000","last_became_active":"2024-04-24T23:34:54.365493+0000","last_became_peered":"2024-04-24T23:34:54.365493+0000","last_unstale":"2024-04-25T00:01:05.107266+0000","last_undegraded":"2024-04-25T00:01:05.107266+0000","last_fullsized":"2024-04-25T00:01:05.107266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2453","last_scrub_stamp":"2024-04-25T00:01:05.107173+0000","last_deep_scrub":"27'2453","last_deep_scrub_stamp":"2024-04-25T00:01:05.107173+0000","last_clean_scrub_stamp":"2024-04-25T00:01:05.107173+0000","objects_scrubbed":0,"log_size":2453,"log_dups_size":0,"ondisk_log_size":2453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:12:53.127579+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"27'2590","reported_seq":2635,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:06.108968+0000","last_change":"2024-04-25T00:01:06.109006+0000","last_active":"2024-04-25T00:01:06.108968+0000","last_peered":"2024-04-25T00:01:06.108968+0000","last_clean":"2024-04-25T00:01:06.108968+0000","last_became_active":"2024-04-24T23:34:54.378568+0000","last_became_peered":"2024-04-24T23:34:54.378568+0000","last_unstale":"2024-04-25T00:01:06.108968+0000","last_undegraded":"2024-04-25T00:01:06.108968+0000","last_fullsized":"2024-04-25T00:01:06.108968+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2590","last_scrub_stamp":"2024-04-25T00:01:06.108860+0000","last_deep_scrub":"27'2590","last_deep_scrub_stamp":"2024-04-25T00:01:06.108860+0000","last_clean_scrub_stamp":"2024-04-25T00:01:06.108860+0000","objects_scrubbed":0,"log_size":2590,"log_dups_size":0,"ondisk_log_size":2590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:06:37.123991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"27'2557","reported_seq":2602,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:04.256443+0000","last_change":"2024-04-25T00:01:04.256479+0000","last_active":"2024-04-25T00:01:04.256443+0000","last_peered":"2024-04-25T00:01:04.256443+0000","last_clean":"2024-04-25T00:01:04.256443+0000","last_became_active":"2024-04-24T23:34:54.363439+0000","last_became_peered":"2024-04-24T23:34:54.363439+0000","last_unstale":"2024-04-25T00:01:04.256443+0000","last_undegraded":"2024-04-25T00:01:04.256443+0000","last_fullsized":"2024-04-25T00:01:04.256443+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2557","last_scrub_stamp":"2024-04-25T00:01:04.256331+0000","last_deep_scrub":"27'2557","last_deep_scrub_stamp":"2024-04-25T00:01:04.256331+0000","last_clean_scrub_stamp":"2024-04-25T00:01:04.256331+0000","objects_scrubbed":0,"log_size":2557,"log_dups_size":0,"ondisk_log_size":2557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:32:13.247335+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"27'2540","reported_seq":2585,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:04.914408+0000","last_change":"2024-04-25T00:01:04.914408+0000","last_active":"2024-04-25T00:01:04.914408+0000","last_peered":"2024-04-25T00:01:04.914408+0000","last_clean":"2024-04-25T00:01:04.914408+0000","last_became_active":"2024-04-24T23:34:54.375123+0000","last_became_peered":"2024-04-24T23:34:54.375123+0000","last_unstale":"2024-04-25T00:01:04.914408+0000","last_undegraded":"2024-04-25T00:01:04.914408+0000","last_fullsized":"2024-04-25T00:01:04.914408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2540","last_scrub_stamp":"2024-04-25T00:01:04.914323+0000","last_deep_scrub":"27'2540","last_deep_scrub_stamp":"2024-04-25T00:01:04.914323+0000","last_clean_scrub_stamp":"2024-04-25T00:01:04.914323+0000","objects_scrubbed":0,"log_size":2540,"log_dups_size":0,"ondisk_log_size":2540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:12:52.934728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"27'2526","reported_seq":2570,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:59.305901+0000","last_change":"2024-04-25T00:00:59.305924+0000","last_active":"2024-04-25T00:00:59.305901+0000","last_peered":"2024-04-25T00:00:59.305901+0000","last_clean":"2024-04-25T00:00:59.305901+0000","last_became_active":"2024-04-24T23:34:54.369219+0000","last_became_peered":"2024-04-24T23:34:54.369219+0000","last_unstale":"2024-04-25T00:00:59.305901+0000","last_undegraded":"2024-04-25T00:00:59.305901+0000","last_fullsized":"2024-04-25T00:00:59.305901+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2526","last_scrub_stamp":"2024-04-25T00:00:59.305832+0000","last_deep_scrub":"27'2526","last_deep_scrub_stamp":"2024-04-25T00:00:59.305832+0000","last_clean_scrub_stamp":"2024-04-25T00:00:59.305832+0000","objects_scrubbed":0,"log_size":2526,"log_dups_size":0,"ondisk_log_size":2526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:01.562527+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"27'2564","reported_seq":2594,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:04.973041+0000","last_change":"2024-04-25T00:01:04.973082+0000","last_active":"2024-04-25T00:01:04.973041+0000","last_peered":"2024-04-25T00:01:04.973041+0000","last_clean":"2024-04-25T00:01:04.973041+0000","last_became_active":"2024-04-24T23:35:43.933686+0000","last_became_peered":"2024-04-24T23:35:43.933686+0000","last_unstale":"2024-04-25T00:01:04.973041+0000","last_undegraded":"2024-04-25T00:01:04.973041+0000","last_fullsized":"2024-04-25T00:01:04.973041+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2564","last_scrub_stamp":"2024-04-25T00:01:04.972966+0000","last_deep_scrub":"27'2564","last_deep_scrub_stamp":"2024-04-25T00:01:04.972966+0000","last_clean_scrub_stamp":"2024-04-25T00:01:04.972966+0000","objects_scrubbed":0,"log_size":2564,"log_dups_size":0,"ondisk_log_size":2564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:02:51.830227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2b","version":"27'2598","reported_seq":2642,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:02.014639+0000","last_change":"2024-04-25T00:01:02.014662+0000","last_active":"2024-04-25T00:01:02.014639+0000","last_peered":"2024-04-25T00:01:02.014639+0000","last_clean":"2024-04-25T00:01:02.014639+0000","last_became_active":"2024-04-24T23:34:54.370793+0000","last_became_peered":"2024-04-24T23:34:54.370793+0000","last_unstale":"2024-04-25T00:01:02.014639+0000","last_undegraded":"2024-04-25T00:01:02.014639+0000","last_fullsized":"2024-04-25T00:01:02.014639+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2598","last_scrub_stamp":"2024-04-25T00:01:02.014571+0000","last_deep_scrub":"27'2598","last_deep_scrub_stamp":"2024-04-25T00:01:02.014571+0000","last_clean_scrub_stamp":"2024-04-25T00:01:02.014571+0000","objects_scrubbed":0,"log_size":2598,"log_dups_size":0,"ondisk_log_size":2598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:55:58.335701+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"27'2595","reported_seq":2639,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:04.137947+0000","last_change":"2024-04-25T00:01:04.137986+0000","last_active":"2024-04-25T00:01:04.137947+0000","last_peered":"2024-04-25T00:01:04.137947+0000","last_clean":"2024-04-25T00:01:04.137947+0000","last_became_active":"2024-04-24T23:34:54.376940+0000","last_became_peered":"2024-04-24T23:34:54.376940+0000","last_unstale":"2024-04-25T00:01:04.137947+0000","last_undegraded":"2024-04-25T00:01:04.137947+0000","last_fullsized":"2024-04-25T00:01:04.137947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2595","last_scrub_stamp":"2024-04-25T00:01:04.137832+0000","last_deep_scrub":"27'2595","last_deep_scrub_stamp":"2024-04-25T00:01:04.137832+0000","last_clean_scrub_stamp":"2024-04-25T00:01:04.137832+0000","objects_scrubbed":0,"log_size":2595,"log_dups_size":0,"ondisk_log_size":2595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:45:03.899541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"27'2588","reported_seq":2631,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:58.053198+0000","last_change":"2024-04-25T00:00:58.053220+0000","last_active":"2024-04-25T00:00:58.053198+0000","last_peered":"2024-04-25T00:00:58.053198+0000","last_clean":"2024-04-25T00:00:58.053198+0000","last_became_active":"2024-04-24T23:34:54.368228+0000","last_became_peered":"2024-04-24T23:34:54.368228+0000","last_unstale":"2024-04-25T00:00:58.053198+0000","last_undegraded":"2024-04-25T00:00:58.053198+0000","last_fullsized":"2024-04-25T00:00:58.053198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2588","last_scrub_stamp":"2024-04-25T00:00:58.053136+0000","last_deep_scrub":"27'2588","last_deep_scrub_stamp":"2024-04-25T00:00:58.053136+0000","last_clean_scrub_stamp":"2024-04-25T00:00:58.053136+0000","objects_scrubbed":0,"log_size":2588,"log_dups_size":0,"ondisk_log_size":2588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:44.070301+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"27'2532","reported_seq":2576,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:59.982893+0000","last_change":"2024-04-25T00:00:59.982913+0000","last_active":"2024-04-25T00:00:59.982893+0000","last_peered":"2024-04-25T00:00:59.982893+0000","last_clean":"2024-04-25T00:00:59.982893+0000","last_became_active":"2024-04-24T23:34:54.372766+0000","last_became_peered":"2024-04-24T23:34:54.372766+0000","last_unstale":"2024-04-25T00:00:59.982893+0000","last_undegraded":"2024-04-25T00:00:59.982893+0000","last_fullsized":"2024-04-25T00:00:59.982893+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2532","last_scrub_stamp":"2024-04-25T00:00:59.982822+0000","last_deep_scrub":"27'2532","last_deep_scrub_stamp":"2024-04-25T00:00:59.982822+0000","last_clean_scrub_stamp":"2024-04-25T00:00:59.982822+0000","objects_scrubbed":0,"log_size":2532,"log_dups_size":0,"ondisk_log_size":2532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:11:44.041900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"27'2601","reported_seq":2645,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:09.896631+0000","last_change":"2024-04-25T00:01:09.896670+0000","last_active":"2024-04-25T00:01:09.896631+0000","last_peered":"2024-04-25T00:01:09.896631+0000","last_clean":"2024-04-25T00:01:09.896631+0000","last_became_active":"2024-04-24T23:34:54.383463+0000","last_became_peered":"2024-04-24T23:34:54.383463+0000","last_unstale":"2024-04-25T00:01:09.896631+0000","last_undegraded":"2024-04-25T00:01:09.896631+0000","last_fullsized":"2024-04-25T00:01:09.896631+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2601","last_scrub_stamp":"2024-04-25T00:01:09.896541+0000","last_deep_scrub":"27'2601","last_deep_scrub_stamp":"2024-04-25T00:01:09.896541+0000","last_clean_scrub_stamp":"2024-04-25T00:01:09.896541+0000","objects_scrubbed":0,"log_size":2601,"log_dups_size":0,"ondisk_log_size":2601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:21.944500+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"27'2529","reported_seq":2573,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:07.890936+0000","last_change":"2024-04-25T00:01:07.890974+0000","last_active":"2024-04-25T00:01:07.890936+0000","last_peered":"2024-04-25T00:01:07.890936+0000","last_clean":"2024-04-25T00:01:07.890936+0000","last_became_active":"2024-04-24T23:34:54.380702+0000","last_became_peered":"2024-04-24T23:34:54.380702+0000","last_unstale":"2024-04-25T00:01:07.890936+0000","last_undegraded":"2024-04-25T00:01:07.890936+0000","last_fullsized":"2024-04-25T00:01:07.890936+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2529","last_scrub_stamp":"2024-04-25T00:01:07.890855+0000","last_deep_scrub":"27'2529","last_deep_scrub_stamp":"2024-04-25T00:01:07.890855+0000","last_clean_scrub_stamp":"2024-04-25T00:01:07.890855+0000","objects_scrubbed":0,"log_size":2529,"log_dups_size":0,"ondisk_log_size":2529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:13:25.437443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"27'2650","reported_seq":2694,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:56.127927+0000","last_change":"2024-04-25T00:00:56.127955+0000","last_active":"2024-04-25T00:00:56.127927+0000","last_peered":"2024-04-25T00:00:56.127927+0000","last_clean":"2024-04-25T00:00:56.127927+0000","last_became_active":"2024-04-24T23:34:54.358203+0000","last_became_peered":"2024-04-24T23:34:54.358203+0000","last_unstale":"2024-04-25T00:00:56.127927+0000","last_undegraded":"2024-04-25T00:00:56.127927+0000","last_fullsized":"2024-04-25T00:00:56.127927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2650","last_scrub_stamp":"2024-04-25T00:00:56.127849+0000","last_deep_scrub":"27'2650","last_deep_scrub_stamp":"2024-04-25T00:00:56.127849+0000","last_clean_scrub_stamp":"2024-04-25T00:00:56.127849+0000","objects_scrubbed":0,"log_size":2650,"log_dups_size":0,"ondisk_log_size":2650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:38:09.372461+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"27'2497","reported_seq":2541,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:00.049532+0000","last_change":"2024-04-25T00:01:00.049568+0000","last_active":"2024-04-25T00:01:00.049532+0000","last_peered":"2024-04-25T00:01:00.049532+0000","last_clean":"2024-04-25T00:01:00.049532+0000","last_became_active":"2024-04-24T23:34:54.367595+0000","last_became_peered":"2024-04-24T23:34:54.367595+0000","last_unstale":"2024-04-25T00:01:00.049532+0000","last_undegraded":"2024-04-25T00:01:00.049532+0000","last_fullsized":"2024-04-25T00:01:00.049532+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2497","last_scrub_stamp":"2024-04-25T00:01:00.049447+0000","last_deep_scrub":"27'2497","last_deep_scrub_stamp":"2024-04-25T00:01:00.049447+0000","last_clean_scrub_stamp":"2024-04-25T00:01:00.049447+0000","objects_scrubbed":0,"log_size":2497,"log_dups_size":0,"ondisk_log_size":2497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:18:48.938901+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"27'2502","reported_seq":2531,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:12.172793+0000","last_change":"2024-04-25T00:01:12.172815+0000","last_active":"2024-04-25T00:01:12.172793+0000","last_peered":"2024-04-25T00:01:12.172793+0000","last_clean":"2024-04-25T00:01:12.172793+0000","last_became_active":"2024-04-24T23:35:43.529631+0000","last_became_peered":"2024-04-24T23:35:43.529631+0000","last_unstale":"2024-04-25T00:01:12.172793+0000","last_undegraded":"2024-04-25T00:01:12.172793+0000","last_fullsized":"2024-04-25T00:01:12.172793+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2502","last_scrub_stamp":"2024-04-25T00:01:12.172724+0000","last_deep_scrub":"27'2502","last_deep_scrub_stamp":"2024-04-25T00:01:12.172724+0000","last_clean_scrub_stamp":"2024-04-25T00:01:12.172724+0000","objects_scrubbed":0,"log_size":2502,"log_dups_size":0,"ondisk_log_size":2502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:17:58.506910+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.34","version":"27'2450","reported_seq":2493,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:01.625828+0000","last_change":"2024-04-25T00:01:01.625856+0000","last_active":"2024-04-25T00:01:01.625828+0000","last_peered":"2024-04-25T00:01:01.625828+0000","last_clean":"2024-04-25T00:01:01.625828+0000","last_became_active":"2024-04-24T23:34:54.377277+0000","last_became_peered":"2024-04-24T23:34:54.377277+0000","last_unstale":"2024-04-25T00:01:01.625828+0000","last_undegraded":"2024-04-25T00:01:01.625828+0000","last_fullsized":"2024-04-25T00:01:01.625828+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2450","last_scrub_stamp":"2024-04-25T00:01:01.625754+0000","last_deep_scrub":"27'2450","last_deep_scrub_stamp":"2024-04-25T00:01:01.625754+0000","last_clean_scrub_stamp":"2024-04-25T00:01:01.625754+0000","objects_scrubbed":0,"log_size":2450,"log_dups_size":0,"ondisk_log_size":2450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:18:50.515208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"27'2496","reported_seq":2539,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:07.109209+0000","last_change":"2024-04-25T00:01:07.109233+0000","last_active":"2024-04-25T00:01:07.109209+0000","last_peered":"2024-04-25T00:01:07.109209+0000","last_clean":"2024-04-25T00:01:07.109209+0000","last_became_active":"2024-04-24T23:34:54.387781+0000","last_became_peered":"2024-04-24T23:34:54.387781+0000","last_unstale":"2024-04-25T00:01:07.109209+0000","last_undegraded":"2024-04-25T00:01:07.109209+0000","last_fullsized":"2024-04-25T00:01:07.109209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2496","last_scrub_stamp":"2024-04-25T00:01:07.109132+0000","last_deep_scrub":"27'2496","last_deep_scrub_stamp":"2024-04-25T00:01:07.109132+0000","last_clean_scrub_stamp":"2024-04-25T00:01:07.109132+0000","objects_scrubbed":0,"log_size":2496,"log_dups_size":0,"ondisk_log_size":2496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:07:54.441793+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"27'2613","reported_seq":2656,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:08.106753+0000","last_change":"2024-04-25T00:01:08.106791+0000","last_active":"2024-04-25T00:01:08.106753+0000","last_peered":"2024-04-25T00:01:08.106753+0000","last_clean":"2024-04-25T00:01:08.106753+0000","last_became_active":"2024-04-24T23:34:54.387913+0000","last_became_peered":"2024-04-24T23:34:54.387913+0000","last_unstale":"2024-04-25T00:01:08.106753+0000","last_undegraded":"2024-04-25T00:01:08.106753+0000","last_fullsized":"2024-04-25T00:01:08.106753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2613","last_scrub_stamp":"2024-04-25T00:01:08.106638+0000","last_deep_scrub":"27'2613","last_deep_scrub_stamp":"2024-04-25T00:01:08.106638+0000","last_clean_scrub_stamp":"2024-04-25T00:01:08.106638+0000","objects_scrubbed":0,"log_size":2613,"log_dups_size":0,"ondisk_log_size":2613,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:29:25.481550+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"27'2583","reported_seq":2625,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:13.852424+0000","last_change":"2024-04-25T00:01:13.852447+0000","last_active":"2024-04-25T00:01:13.852424+0000","last_peered":"2024-04-25T00:01:13.852424+0000","last_clean":"2024-04-25T00:01:13.852424+0000","last_became_active":"2024-04-24T23:34:54.383024+0000","last_became_peered":"2024-04-24T23:34:54.383024+0000","last_unstale":"2024-04-25T00:01:13.852424+0000","last_undegraded":"2024-04-25T00:01:13.852424+0000","last_fullsized":"2024-04-25T00:01:13.852424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2583","last_scrub_stamp":"2024-04-25T00:01:13.852347+0000","last_deep_scrub":"27'2583","last_deep_scrub_stamp":"2024-04-25T00:01:13.852347+0000","last_clean_scrub_stamp":"2024-04-25T00:01:13.852347+0000","objects_scrubbed":0,"log_size":2583,"log_dups_size":0,"ondisk_log_size":2583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:04:38.109888+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"27'2526","reported_seq":2569,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:58.299580+0000","last_change":"2024-04-25T00:00:58.299619+0000","last_active":"2024-04-25T00:00:58.299580+0000","last_peered":"2024-04-25T00:00:58.299580+0000","last_clean":"2024-04-25T00:00:58.299580+0000","last_became_active":"2024-04-24T23:34:54.382482+0000","last_became_peered":"2024-04-24T23:34:54.382482+0000","last_unstale":"2024-04-25T00:00:58.299580+0000","last_undegraded":"2024-04-25T00:00:58.299580+0000","last_fullsized":"2024-04-25T00:00:58.299580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2526","last_scrub_stamp":"2024-04-25T00:00:58.299502+0000","last_deep_scrub":"27'2526","last_deep_scrub_stamp":"2024-04-25T00:00:58.299502+0000","last_clean_scrub_stamp":"2024-04-25T00:00:58.299502+0000","objects_scrubbed":0,"log_size":2526,"log_dups_size":0,"ondisk_log_size":2526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:10.303461+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"27'2536","reported_seq":2578,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:07.847203+0000","last_change":"2024-04-25T00:01:07.847223+0000","last_active":"2024-04-25T00:01:07.847203+0000","last_peered":"2024-04-25T00:01:07.847203+0000","last_clean":"2024-04-25T00:01:07.847203+0000","last_became_active":"2024-04-24T23:34:54.368663+0000","last_became_peered":"2024-04-24T23:34:54.368663+0000","last_unstale":"2024-04-25T00:01:07.847203+0000","last_undegraded":"2024-04-25T00:01:07.847203+0000","last_fullsized":"2024-04-25T00:01:07.847203+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2536","last_scrub_stamp":"2024-04-25T00:01:07.847132+0000","last_deep_scrub":"27'2536","last_deep_scrub_stamp":"2024-04-25T00:01:07.847132+0000","last_clean_scrub_stamp":"2024-04-25T00:01:07.847132+0000","objects_scrubbed":0,"log_size":2536,"log_dups_size":0,"ondisk_log_size":2536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:34.663000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"27'2479","reported_seq":2522,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:06.868019+0000","last_change":"2024-04-25T00:01:06.868066+0000","last_active":"2024-04-25T00:01:06.868019+0000","last_peered":"2024-04-25T00:01:06.868019+0000","last_clean":"2024-04-25T00:01:06.868019+0000","last_became_active":"2024-04-24T23:34:54.380052+0000","last_became_peered":"2024-04-24T23:34:54.380052+0000","last_unstale":"2024-04-25T00:01:06.868019+0000","last_undegraded":"2024-04-25T00:01:06.868019+0000","last_fullsized":"2024-04-25T00:01:06.868019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2479","last_scrub_stamp":"2024-04-25T00:01:06.867940+0000","last_deep_scrub":"27'2479","last_deep_scrub_stamp":"2024-04-25T00:01:06.867940+0000","last_clean_scrub_stamp":"2024-04-25T00:01:06.867940+0000","objects_scrubbed":0,"log_size":2479,"log_dups_size":0,"ondisk_log_size":2479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:48.539103+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"27'2516","reported_seq":2558,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:06.916177+0000","last_change":"2024-04-25T00:01:06.916216+0000","last_active":"2024-04-25T00:01:06.916177+0000","last_peered":"2024-04-25T00:01:06.916177+0000","last_clean":"2024-04-25T00:01:06.916177+0000","last_became_active":"2024-04-24T23:34:54.364605+0000","last_became_peered":"2024-04-24T23:34:54.364605+0000","last_unstale":"2024-04-25T00:01:06.916177+0000","last_undegraded":"2024-04-25T00:01:06.916177+0000","last_fullsized":"2024-04-25T00:01:06.916177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2516","last_scrub_stamp":"2024-04-25T00:01:06.916086+0000","last_deep_scrub":"27'2516","last_deep_scrub_stamp":"2024-04-25T00:01:06.916086+0000","last_clean_scrub_stamp":"2024-04-25T00:01:06.916086+0000","objects_scrubbed":0,"log_size":2516,"log_dups_size":0,"ondisk_log_size":2516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:29:07.082246+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"27'2568","reported_seq":2599,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:02.018530+0000","last_change":"2024-04-25T00:01:02.018551+0000","last_active":"2024-04-25T00:01:02.018530+0000","last_peered":"2024-04-25T00:01:02.018530+0000","last_clean":"2024-04-25T00:01:02.018530+0000","last_became_active":"2024-04-24T23:35:43.525191+0000","last_became_peered":"2024-04-24T23:35:43.525191+0000","last_unstale":"2024-04-25T00:01:02.018530+0000","last_undegraded":"2024-04-25T00:01:02.018530+0000","last_fullsized":"2024-04-25T00:01:02.018530+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2568","last_scrub_stamp":"2024-04-25T00:01:02.018471+0000","last_deep_scrub":"27'2568","last_deep_scrub_stamp":"2024-04-25T00:01:02.018471+0000","last_clean_scrub_stamp":"2024-04-25T00:01:02.018471+0000","objects_scrubbed":0,"log_size":2568,"log_dups_size":0,"ondisk_log_size":2568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:38:01.492823+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"27'2637","reported_seq":2682,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:58.081112+0000","last_change":"2024-04-25T00:00:58.081137+0000","last_active":"2024-04-25T00:00:58.081112+0000","last_peered":"2024-04-25T00:00:58.081112+0000","last_clean":"2024-04-25T00:00:58.081112+0000","last_became_active":"2024-04-24T23:34:54.365814+0000","last_became_peered":"2024-04-24T23:34:54.365814+0000","last_unstale":"2024-04-25T00:00:58.081112+0000","last_undegraded":"2024-04-25T00:00:58.081112+0000","last_fullsized":"2024-04-25T00:00:58.081112+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2637","last_scrub_stamp":"2024-04-25T00:00:58.081028+0000","last_deep_scrub":"27'2637","last_deep_scrub_stamp":"2024-04-25T00:00:58.081028+0000","last_clean_scrub_stamp":"2024-04-25T00:00:58.081028+0000","objects_scrubbed":0,"log_size":2637,"log_dups_size":0,"ondisk_log_size":2637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:19:32.412193+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"27'2476","reported_seq":2521,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:06.101132+0000","last_change":"2024-04-25T00:01:06.101155+0000","last_active":"2024-04-25T00:01:06.101132+0000","last_peered":"2024-04-25T00:01:06.101132+0000","last_clean":"2024-04-25T00:01:06.101132+0000","last_became_active":"2024-04-24T23:34:54.368663+0000","last_became_peered":"2024-04-24T23:34:54.368663+0000","last_unstale":"2024-04-25T00:01:06.101132+0000","last_undegraded":"2024-04-25T00:01:06.101132+0000","last_fullsized":"2024-04-25T00:01:06.101132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2476","last_scrub_stamp":"2024-04-25T00:01:06.101057+0000","last_deep_scrub":"27'2476","last_deep_scrub_stamp":"2024-04-25T00:01:06.101057+0000","last_clean_scrub_stamp":"2024-04-25T00:01:06.101057+0000","objects_scrubbed":0,"log_size":2476,"log_dups_size":0,"ondisk_log_size":2476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:25.585996+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"27'2458","reported_seq":2502,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:08.893917+0000","last_change":"2024-04-25T00:01:08.893937+0000","last_active":"2024-04-25T00:01:08.893917+0000","last_peered":"2024-04-25T00:01:08.893917+0000","last_clean":"2024-04-25T00:01:08.893917+0000","last_became_active":"2024-04-24T23:34:54.373531+0000","last_became_peered":"2024-04-24T23:34:54.373531+0000","last_unstale":"2024-04-25T00:01:08.893917+0000","last_undegraded":"2024-04-25T00:01:08.893917+0000","last_fullsized":"2024-04-25T00:01:08.893917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2458","last_scrub_stamp":"2024-04-25T00:01:08.893844+0000","last_deep_scrub":"27'2458","last_deep_scrub_stamp":"2024-04-25T00:01:08.893844+0000","last_clean_scrub_stamp":"2024-04-25T00:01:08.893844+0000","objects_scrubbed":0,"log_size":2458,"log_dups_size":0,"ondisk_log_size":2458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:29:26.268756+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12310,"num_read_kb":3126740,"num_write":212524,"num_write_kb":30664179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":230531,"ondisk_log_size":230531,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1120,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":162975,"ondisk_log_size":162975,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":236421422,"num_objects":79,"num_object_clones":0,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":0,"num_read":3152,"num_read_kb":19253,"num_write":210114,"num_write_kb":1682543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5807,"num_omap_keys":12,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":287891456,"data_stored":472842844,"data_compressed":55745062,"data_compressed_allocated":81436672,"data_compressed_original":266452992,"omap_allocated":0,"internal_metadata":0},"log_size":195731,"ondisk_log_size":195731,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65248,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542463,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":961884,"kb_used_data":28436,"kb_used_omap":1,"kb_used_meta":933438,"kb_avail":93409956,"statfs":{"total":96636764160,"available":95651794944,"internally_reserved":0,"allocated":29118464,"data_stored":47674208,"data_compressed":5564865,"data_compressed_allocated":8151040,"data_compressed_original":26886144,"omap_allocated":1590,"internal_metadata":955840970},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542461,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1126144,"kb_used_data":40056,"kb_used_omap":1,"kb_used_meta":1086078,"kb_avail":93245696,"statfs":{"total":96636764160,"available":95483592704,"internally_reserved":0,"allocated":41017344,"data_stored":68055450,"data_compressed":8055094,"data_compressed_allocated":11661312,"data_compressed_original":38879232,"omap_allocated":1594,"internal_metadata":1112144326},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542462,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":980308,"kb_used_data":24716,"kb_used_omap":1,"kb_used_meta":955582,"kb_avail":93391532,"statfs":{"total":96636764160,"available":95632928768,"internally_reserved":0,"allocated":25309184,"data_stored":42060410,"data_compressed":5058379,"data_compressed_allocated":7299072,"data_compressed_original":24223744,"omap_allocated":1595,"internal_metadata":978516421},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542461,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":924500,"kb_used_data":40972,"kb_used_omap":1,"kb_used_meta":883518,"kb_avail":93447340,"statfs":{"total":96636764160,"available":95690076160,"internally_reserved":0,"allocated":41955328,"data_stored":67226270,"data_compressed":7785480,"data_compressed_allocated":11411456,"data_compressed_original":36864000,"omap_allocated":1595,"internal_metadata":904722885},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542463,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":660908,"kb_used_data":34148,"kb_used_omap":1,"kb_used_meta":626750,"kb_avail":93710932,"statfs":{"total":96636764160,"available":95959994368,"internally_reserved":0,"allocated":34967552,"data_stored":54643232,"data_compressed":6190216,"data_compressed_allocated":9224192,"data_compressed_original":29069312,"omap_allocated":1595,"internal_metadata":641792453},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542463,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1239756,"kb_used_data":39748,"kb_used_omap":1,"kb_used_meta":1199998,"kb_avail":93132084,"statfs":{"total":96636764160,"available":95367254016,"internally_reserved":0,"allocated":40701952,"data_stored":63742294,"data_compressed":7360432,"data_compressed_allocated":10985472,"data_compressed_original":34205696,"omap_allocated":1595,"internal_metadata":1228798405},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542463,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":827048,"kb_used_data":33824,"kb_used_omap":1,"kb_used_meta":793214,"kb_avail":93544792,"statfs":{"total":96636764160,"available":95789867008,"internally_reserved":0,"allocated":34635776,"data_stored":60138836,"data_compressed":7346778,"data_compressed_allocated":10571776,"data_compressed_original":36261888,"omap_allocated":1595,"internal_metadata":812251589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575168,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1145812,"kb_used_data":41612,"kb_used_omap":1,"kb_used_meta":1104190,"kb_avail":93226028,"statfs":{"total":96636764160,"available":95463452672,"internally_reserved":0,"allocated":42610688,"data_stored":71421120,"data_compressed":8449066,"data_compressed_allocated":12222464,"data_compressed_original":41209856,"omap_allocated":1593,"internal_metadata":1130691015},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32624,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32624,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":34283520,"data_stored":59431188,"data_compressed":7314154,"data_compressed_allocated":10526720,"data_compressed_original":35688448,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40415232,"data_stored":63625014,"data_compressed":7360432,"data_compressed_allocated":10985472,"data_compressed_original":34205696,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":34680832,"data_stored":54525952,"data_compressed":6190216,"data_compressed_allocated":9224192,"data_compressed_original":29069312,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":42323968,"data_stored":71303840,"data_compressed":8449066,"data_compressed_allocated":12222464,"data_compressed_original":41209856,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":41668608,"data_stored":67108990,"data_compressed":7785480,"data_compressed_allocated":11411456,"data_compressed_original":36864000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":25022464,"data_stored":41943130,"data_compressed":5058379,"data_compressed_allocated":7299072,"data_compressed_original":24223744,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40730624,"data_stored":67938170,"data_compressed":8055094,"data_compressed_allocated":11661312,"data_compressed_original":38879232,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28766208,"data_stored":46966560,"data_compressed":5532241,"data_compressed_allocated":8105984,"data_compressed_original":26312704,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:01:18.012 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-04-24T23:34:53.338884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=34, tm_sec=53, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:01:18.013 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-25T00:01:38.014 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:01:38.113 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.112+0000 7f740d9a5640 1 Processor -- start 2024-04-25T00:01:38.113 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.112+0000 7f740d9a5640 1 -- start start 2024-04-25T00:01:38.115 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.113+0000 7f740d9a5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f74081562d0 0x7f740806b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:38.115 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.113+0000 7f740d9a5640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74080c2c00 con 0x7f740806b6a0 2024-04-25T00:01:38.115 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.113+0000 7f740d9a5640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74080c2ee0 con 0x7f74080c3e20 2024-04-25T00:01:38.115 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.113+0000 7f740d9a5640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f74081566b0 con 0x7f74081562d0 2024-04-25T00:01:38.115 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.113+0000 7f7406ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f74081562d0 0x7f740806b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:38.115 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.113+0000 7f74067fc640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f740806b6a0 0x7f7408158c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:50252/0 (socket says 172.21.15.157:50252) 2024-04-25T00:01:38.115 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.113+0000 7f74067fc640 1 -- 172.21.15.157:0/1085365002 learned_addr learned my addr 172.21.15.157:0/1085365002 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-25T00:01:38.115 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.114+0000 7f7406ffd640 1 -- 172.21.15.157:0/1085365002 >> v1:172.21.15.169:6789/0 conn(0x7f74080c3e20 legacy=0x7f7408159310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:38.115 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.114+0000 7f7406ffd640 1 -- 172.21.15.157:0/1085365002 >> v1:172.21.15.157:6789/0 conn(0x7f740806b6a0 legacy=0x7f7408158c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:38.115 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.114+0000 7f7405ffb640 1 -- 172.21.15.157:0/1085365002 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 322742977 0 0) 0x7f74080c2c00 con 0x7f740806b6a0 2024-04-25T00:01:38.116 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.114+0000 7f7406ffd640 1 -- 172.21.15.157:0/1085365002 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74080c31c0 con 0x7f74081562d0 2024-04-25T00:01:38.116 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.114+0000 7f7406ffd640 1 --2- 172.21.15.157:0/1085365002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74081562d0 0x7f740806b160 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f73fc0097a0 tx=0x7f73fc02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc962ad2db562f38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:38.116 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.114+0000 7f7405ffb640 1 -- 172.21.15.157:0/1085365002 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73fc037070 con 0x7f74081562d0 2024-04-25T00:01:38.116 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.115+0000 7f7405ffb640 1 -- 172.21.15.157:0/1085365002 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73fc032440 con 0x7f74081562d0 2024-04-25T00:01:38.116 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.115+0000 7f7405ffb640 1 -- 172.21.15.157:0/1085365002 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73fc02ae60 con 0x7f74081562d0 2024-04-25T00:01:38.117 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.115+0000 7f740d9a5640 1 -- 172.21.15.157:0/1085365002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74081562d0 msgr2=0x7f740806b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:38.117 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.115+0000 7f740d9a5640 1 --2- 172.21.15.157:0/1085365002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74081562d0 0x7f740806b160 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f73fc0097a0 tx=0x7f73fc02a110 comp rx=0 tx=0).stop 2024-04-25T00:01:38.117 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.115+0000 7f740d9a5640 1 -- 172.21.15.157:0/1085365002 shutdown_connections 2024-04-25T00:01:38.117 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.115+0000 7f740d9a5640 1 --2- 172.21.15.157:0/1085365002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74081562d0 0x7f740806b160 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:38.117 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.115+0000 7f740d9a5640 1 -- 172.21.15.157:0/1085365002 >> 172.21.15.157:0/1085365002 conn(0x7f74080a97a0 msgr2=0x7f74080a9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:38.117 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.115+0000 7f740d9a5640 1 -- 172.21.15.157:0/1085365002 shutdown_connections 2024-04-25T00:01:38.117 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.115+0000 7f740d9a5640 1 -- 172.21.15.157:0/1085365002 wait complete. 2024-04-25T00:01:38.117 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.116+0000 7f740d9a5640 1 Processor -- start 2024-04-25T00:01:38.117 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.116+0000 7f740d9a5640 1 -- start start 2024-04-25T00:01:38.119 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.117+0000 7f740d9a5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f740806b6a0 0x7f7408150f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:38.120 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.117+0000 7f740d9a5640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f74080c3e20 0x7f740814abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:38.120 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.117+0000 7f740d9a5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74081562d0 0x7f740814b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:38.120 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.117+0000 7f740d9a5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7408072e30 con 0x7f740806b6a0 2024-04-25T00:01:38.120 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.117+0000 7f740d9a5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f74080c3780 con 0x7f74081562d0 2024-04-25T00:01:38.120 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.117+0000 7f740d9a5640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7408072fb0 con 0x7f74080c3e20 2024-04-25T00:01:38.120 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.117+0000 7f7406ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f740806b6a0 0x7f7408150f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:38.120 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.117+0000 7f7406ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f740806b6a0 0x7f7408150f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44156/0 (socket says 172.21.15.157:44156) 2024-04-25T00:01:38.120 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.117+0000 7f7406ffd640 1 -- 172.21.15.157:0/1077587873 learned_addr learned my addr 172.21.15.157:0/1077587873 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:01:38.120 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.117+0000 7f74067fc640 1 --2- 172.21.15.157:0/1077587873 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f74080c3e20 0x7f740814abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:38.120 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.117+0000 7f74077fe640 1 --2- 172.21.15.157:0/1077587873 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74081562d0 0x7f740814b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:38.120 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.117+0000 7f7406ffd640 1 -- 172.21.15.157:0/1077587873 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f74080c3e20 msgr2=0x7f740814abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:38.120 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.117+0000 7f7406ffd640 1 --2- 172.21.15.157:0/1077587873 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f74080c3e20 0x7f740814abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:38.120 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.117+0000 7f7406ffd640 1 -- 172.21.15.157:0/1077587873 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74081562d0 msgr2=0x7f740814b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:38.121 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.117+0000 7f7406ffd640 1 --2- 172.21.15.157:0/1077587873 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74081562d0 0x7f740814b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:38.121 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.117+0000 7f7406ffd640 1 -- 172.21.15.157:0/1077587873 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74081593d0 con 0x7f740806b6a0 2024-04-25T00:01:38.121 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.118+0000 7f7406ffd640 1 --2- 172.21.15.157:0/1077587873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f740806b6a0 0x7f7408150f00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f73fc000c00 tx=0x7f73fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:38.121 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.118+0000 7f73effff640 1 -- 172.21.15.157:0/1077587873 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73fc037040 con 0x7f740806b6a0 2024-04-25T00:01:38.121 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.118+0000 7f73effff640 1 -- 172.21.15.157:0/1077587873 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73fc042070 con 0x7f740806b6a0 2024-04-25T00:01:38.121 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.118+0000 7f73effff640 1 -- 172.21.15.157:0/1077587873 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73fc030a70 con 0x7f740806b6a0 2024-04-25T00:01:38.121 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.118+0000 7f740d9a5640 1 -- 172.21.15.157:0/1077587873 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74081ebff0 con 0x7f740806b6a0 2024-04-25T00:01:38.122 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.118+0000 7f740d9a5640 1 -- 172.21.15.157:0/1077587873 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f740814b670 con 0x7f740806b6a0 2024-04-25T00:01:38.122 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.119+0000 7f740d9a5640 1 -- 172.21.15.157:0/1077587873 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74080688d0 con 0x7f740806b6a0 2024-04-25T00:01:38.122 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.119+0000 7f73effff640 1 -- 172.21.15.157:0/1077587873 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f73fc030c10 con 0x7f740806b6a0 2024-04-25T00:01:38.123 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.120+0000 7f73effff640 1 --2- 172.21.15.157:0/1077587873 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f73f406be30 0x7f73f406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:38.123 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.120+0000 7f73effff640 1 -- 172.21.15.157:0/1077587873 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f73fc0abb80 con 0x7f740806b6a0 2024-04-25T00:01:38.123 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.120+0000 7f74067fc640 1 --2- 172.21.15.157:0/1077587873 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f73f406be30 0x7f73f406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:38.123 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.120+0000 7f74067fc640 1 --2- 172.21.15.157:0/1077587873 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f73f406be30 0x7f73f406e2f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7408155640 tx=0x7f73f8007e40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:38.124 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.124+0000 7f73effff640 1 -- 172.21.15.157:0/1077587873 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f73fc079b70 con 0x7f740806b6a0 2024-04-25T00:01:38.296 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.296+0000 7f740d9a5640 1 -- 172.21.15.157:0/1077587873 --> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f74080a8100 con 0x7f73f406be30 2024-04-25T00:01:38.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.304+0000 7f73effff640 1 -- 172.21.15.157:0/1077587873 <== mgr.4105 v2:172.21.15.169:6832/662378289 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+337114 (secure 0 0 0) 0x7f74080a8100 con 0x7f73f406be30 2024-04-25T00:01:38.308 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-25T00:01:38.310 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2024-04-25T00:01:38.312 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.307+0000 7f740d9a5640 1 -- 172.21.15.157:0/1077587873 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f73f406be30 msgr2=0x7f73f406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:38.312 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.307+0000 7f740d9a5640 1 --2- 172.21.15.157:0/1077587873 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f73f406be30 0x7f73f406e2f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7408155640 tx=0x7f73f8007e40 comp rx=0 tx=0).stop 2024-04-25T00:01:38.312 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.307+0000 7f740d9a5640 1 -- 172.21.15.157:0/1077587873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f740806b6a0 msgr2=0x7f7408150f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:38.312 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.307+0000 7f740d9a5640 1 --2- 172.21.15.157:0/1077587873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f740806b6a0 0x7f7408150f00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f73fc000c00 tx=0x7f73fc0046c0 comp rx=0 tx=0).stop 2024-04-25T00:01:38.312 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.307+0000 7f740d9a5640 1 -- 172.21.15.157:0/1077587873 shutdown_connections 2024-04-25T00:01:38.312 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.307+0000 7f740d9a5640 1 --2- 172.21.15.157:0/1077587873 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f73f406be30 0x7f73f406e2f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:38.312 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.307+0000 7f740d9a5640 1 --2- 172.21.15.157:0/1077587873 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74081562d0 0x7f740814b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:38.312 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.307+0000 7f740d9a5640 1 --2- 172.21.15.157:0/1077587873 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f74080c3e20 0x7f740814abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:38.312 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.307+0000 7f740d9a5640 1 --2- 172.21.15.157:0/1077587873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f740806b6a0 0x7f7408150f00 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:38.312 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.307+0000 7f740d9a5640 1 -- 172.21.15.157:0/1077587873 >> 172.21.15.157:0/1077587873 conn(0x7f74080a97a0 msgr2=0x7f74080c57f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:38.312 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.307+0000 7f740d9a5640 1 -- 172.21.15.157:0/1077587873 shutdown_connections 2024-04-25T00:01:38.313 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.307+0000 7f740d9a5640 1 -- 172.21.15.157:0/1077587873 wait complete. 2024-04-25T00:01:38.319 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":826,"stamp":"2024-04-25T00:01:38.182860+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":237011790,"num_objects":81,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":0,"num_read":15578,"num_read_kb":3146093,"num_write":423917,"num_write_kb":32349138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5807,"num_omap_keys":12,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":589356,"ondisk_log_size":589356,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":7866360,"kb_used_data":283512,"kb_used_omap":12,"kb_used_meta":7582771,"kb_avail":747108360,"statfs":{"total":773094113280,"available":765038960640,"internally_reserved":0,"allocated":290316288,"data_stored":474961820,"data_compressed":55810310,"data_compressed_allocated":81526784,"data_compressed_original":267599872,"omap_allocated":12752,"internal_metadata":7764758064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002582"},"pg_stats":[{"pgid":"2.1f","version":"27'2680","reported_seq":2727,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:57.060012+0000","last_change":"2024-04-25T00:00:57.060042+0000","last_active":"2024-04-25T00:00:57.060012+0000","last_peered":"2024-04-25T00:00:57.060012+0000","last_clean":"2024-04-25T00:00:57.060012+0000","last_became_active":"2024-04-24T23:34:53.362104+0000","last_became_peered":"2024-04-24T23:34:53.362104+0000","last_unstale":"2024-04-25T00:00:57.060012+0000","last_undegraded":"2024-04-25T00:00:57.060012+0000","last_fullsized":"2024-04-25T00:00:57.060012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2680","last_scrub_stamp":"2024-04-25T00:00:57.059949+0000","last_deep_scrub":"27'2680","last_deep_scrub_stamp":"2024-04-25T00:00:57.059949+0000","last_clean_scrub_stamp":"2024-04-25T00:00:57.059949+0000","objects_scrubbed":2,"log_size":2680,"log_dups_size":0,"ondisk_log_size":2680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:38:10.304560+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":2687,"num_write_kb":18218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":561,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"27'2551","reported_seq":2597,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:56.270864+0000","last_change":"2024-04-25T00:00:56.270864+0000","last_active":"2024-04-25T00:00:56.270864+0000","last_peered":"2024-04-25T00:00:56.270864+0000","last_clean":"2024-04-25T00:00:56.270864+0000","last_became_active":"2024-04-24T23:34:54.373289+0000","last_became_peered":"2024-04-24T23:34:54.373289+0000","last_unstale":"2024-04-25T00:00:56.270864+0000","last_undegraded":"2024-04-25T00:00:56.270864+0000","last_fullsized":"2024-04-25T00:00:56.270864+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2551","last_scrub_stamp":"2024-04-25T00:00:56.270712+0000","last_deep_scrub":"27'2551","last_deep_scrub_stamp":"2024-04-25T00:00:56.270712+0000","last_clean_scrub_stamp":"2024-04-25T00:00:56.270712+0000","objects_scrubbed":0,"log_size":2551,"log_dups_size":0,"ondisk_log_size":2551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:16.906001+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"27'9462","reported_seq":9818,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:15.374476+0000","last_change":"2024-04-25T00:01:15.374522+0000","last_active":"2024-04-25T00:01:15.374476+0000","last_peered":"2024-04-25T00:01:15.374476+0000","last_clean":"2024-04-25T00:01:15.374476+0000","last_became_active":"2024-04-24T23:34:58.729816+0000","last_became_peered":"2024-04-24T23:34:58.729816+0000","last_unstale":"2024-04-25T00:01:15.374476+0000","last_undegraded":"2024-04-25T00:01:15.374476+0000","last_fullsized":"2024-04-25T00:01:15.374476+0000","mapping_epoch":23,"log_start":"27'3200","ondisk_log_start":"27'3200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9462","last_scrub_stamp":"2024-04-25T00:01:15.374342+0000","last_deep_scrub":"27'9462","last_deep_scrub_stamp":"2024-04-25T00:01:15.374342+0000","last_clean_scrub_stamp":"2024-04-25T00:01:15.374342+0000","objects_scrubbed":0,"log_size":6262,"log_dups_size":0,"ondisk_log_size":6262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:53:20.396466+0000","scrub_duration":41,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":319,"num_read_kb":81026,"num_write":6761,"num_write_kb":1031213,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"27'6592","reported_seq":6640,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:00.297107+0000","last_change":"2024-04-25T00:01:00.297107+0000","last_active":"2024-04-25T00:01:00.297107+0000","last_peered":"2024-04-25T00:01:00.297107+0000","last_clean":"2024-04-25T00:01:00.297107+0000","last_became_active":"2024-04-24T23:34:53.361366+0000","last_became_peered":"2024-04-24T23:34:53.361366+0000","last_unstale":"2024-04-25T00:01:00.297107+0000","last_undegraded":"2024-04-25T00:01:00.297107+0000","last_fullsized":"2024-04-25T00:01:00.297107+0000","mapping_epoch":18,"log_start":"27'300","ondisk_log_start":"27'300","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6592","last_scrub_stamp":"2024-04-25T00:01:00.296977+0000","last_deep_scrub":"27'6592","last_deep_scrub_stamp":"2024-04-25T00:01:00.296977+0000","last_clean_scrub_stamp":"2024-04-25T00:01:00.296977+0000","objects_scrubbed":1,"log_size":6292,"log_dups_size":0,"ondisk_log_size":6292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:01.320608+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6593,"num_write_kb":53610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"27'2529","reported_seq":2574,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:05.990749+0000","last_change":"2024-04-25T00:01:05.990807+0000","last_active":"2024-04-25T00:01:05.990749+0000","last_peered":"2024-04-25T00:01:05.990749+0000","last_clean":"2024-04-25T00:01:05.990749+0000","last_became_active":"2024-04-24T23:34:54.383160+0000","last_became_peered":"2024-04-24T23:34:54.383160+0000","last_unstale":"2024-04-25T00:01:05.990749+0000","last_undegraded":"2024-04-25T00:01:05.990749+0000","last_fullsized":"2024-04-25T00:01:05.990749+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2529","last_scrub_stamp":"2024-04-25T00:01:05.990668+0000","last_deep_scrub":"27'2529","last_deep_scrub_stamp":"2024-04-25T00:01:05.990668+0000","last_clean_scrub_stamp":"2024-04-25T00:01:05.990668+0000","objects_scrubbed":0,"log_size":2529,"log_dups_size":0,"ondisk_log_size":2529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:26:32.280888+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"27'9228","reported_seq":9434,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:08.713941+0000","last_change":"2024-04-25T00:01:08.713983+0000","last_active":"2024-04-25T00:01:08.713941+0000","last_peered":"2024-04-25T00:01:08.713941+0000","last_clean":"2024-04-25T00:01:08.713941+0000","last_became_active":"2024-04-24T23:34:58.403984+0000","last_became_peered":"2024-04-24T23:34:58.403984+0000","last_unstale":"2024-04-25T00:01:08.713941+0000","last_undegraded":"2024-04-25T00:01:08.713941+0000","last_fullsized":"2024-04-25T00:01:08.713941+0000","mapping_epoch":23,"log_start":"27'2060","ondisk_log_start":"27'2060","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9228","last_scrub_stamp":"2024-04-25T00:01:08.713806+0000","last_deep_scrub":"27'9228","last_deep_scrub_stamp":"2024-04-25T00:01:08.713806+0000","last_clean_scrub_stamp":"2024-04-25T00:01:08.713806+0000","objects_scrubbed":0,"log_size":7168,"log_dups_size":0,"ondisk_log_size":7168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:28.198746+0000","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":169,"num_read_kb":42926,"num_write":6485,"num_write_kb":1085292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"27'10055","reported_seq":10219,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:05.932579+0000","last_change":"2024-04-25T00:01:05.932614+0000","last_active":"2024-04-25T00:01:05.932579+0000","last_peered":"2024-04-25T00:01:05.932579+0000","last_clean":"2024-04-25T00:01:05.932579+0000","last_became_active":"2024-04-24T23:34:53.366430+0000","last_became_peered":"2024-04-24T23:34:53.366430+0000","last_unstale":"2024-04-25T00:01:05.932579+0000","last_undegraded":"2024-04-25T00:01:05.932579+0000","last_fullsized":"2024-04-25T00:01:05.932579+0000","mapping_epoch":18,"log_start":"27'3200","ondisk_log_start":"27'3200","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'10055","last_scrub_stamp":"2024-04-25T00:01:05.932513+0000","last_deep_scrub":"27'10055","last_deep_scrub_stamp":"2024-04-25T00:01:05.932513+0000","last_clean_scrub_stamp":"2024-04-25T00:01:05.932513+0000","objects_scrubbed":3,"log_size":6855,"log_dups_size":0,"ondisk_log_size":6855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:42:21.890767+0000","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5022938,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":120,"num_read_kb":893,"num_write":10045,"num_write_kb":77054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"27'2638","reported_seq":2681,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:03.880821+0000","last_change":"2024-04-25T00:01:03.880870+0000","last_active":"2024-04-25T00:01:03.880821+0000","last_peered":"2024-04-25T00:01:03.880821+0000","last_clean":"2024-04-25T00:01:03.880821+0000","last_became_active":"2024-04-24T23:34:54.372844+0000","last_became_peered":"2024-04-24T23:34:54.372844+0000","last_unstale":"2024-04-25T00:01:03.880821+0000","last_undegraded":"2024-04-25T00:01:03.880821+0000","last_fullsized":"2024-04-25T00:01:03.880821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2638","last_scrub_stamp":"2024-04-25T00:01:03.880722+0000","last_deep_scrub":"27'2638","last_deep_scrub_stamp":"2024-04-25T00:01:03.880722+0000","last_clean_scrub_stamp":"2024-04-25T00:01:03.880722+0000","objects_scrubbed":0,"log_size":2638,"log_dups_size":0,"ondisk_log_size":2638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:29:04.046881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"27'9403","reported_seq":10028,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:04.693593+0000","last_change":"2024-04-25T00:01:04.693617+0000","last_active":"2024-04-25T00:01:04.693593+0000","last_peered":"2024-04-25T00:01:04.693593+0000","last_clean":"2024-04-25T00:01:04.693593+0000","last_became_active":"2024-04-24T23:34:58.397817+0000","last_became_peered":"2024-04-24T23:34:58.397817+0000","last_unstale":"2024-04-25T00:01:04.693593+0000","last_undegraded":"2024-04-25T00:01:04.693593+0000","last_fullsized":"2024-04-25T00:01:04.693593+0000","mapping_epoch":23,"log_start":"27'2212","ondisk_log_start":"27'2212","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9403","last_scrub_stamp":"2024-04-25T00:01:04.693512+0000","last_deep_scrub":"27'9403","last_deep_scrub_stamp":"2024-04-25T00:01:04.693512+0000","last_clean_scrub_stamp":"2024-04-25T00:01:04.693512+0000","objects_scrubbed":0,"log_size":7191,"log_dups_size":0,"ondisk_log_size":7191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:18:20.800593+0000","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":588,"num_read_kb":149352,"num_write":6710,"num_write_kb":891395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"27'5159","reported_seq":5203,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:02.952659+0000","last_change":"2024-04-25T00:01:02.952701+0000","last_active":"2024-04-25T00:01:02.952659+0000","last_peered":"2024-04-25T00:01:02.952659+0000","last_clean":"2024-04-25T00:01:02.952659+0000","last_became_active":"2024-04-24T23:34:53.362337+0000","last_became_peered":"2024-04-24T23:34:53.362337+0000","last_unstale":"2024-04-25T00:01:02.952659+0000","last_undegraded":"2024-04-25T00:01:02.952659+0000","last_fullsized":"2024-04-25T00:01:02.952659+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5159","last_scrub_stamp":"2024-04-25T00:01:02.952565+0000","last_deep_scrub":"27'5159","last_deep_scrub_stamp":"2024-04-25T00:01:02.952565+0000","last_clean_scrub_stamp":"2024-04-25T00:01:02.952565+0000","objects_scrubbed":1,"log_size":5159,"log_dups_size":0,"ondisk_log_size":5159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:57:16.859658+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":5165,"num_write_kb":44324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"27'2566","reported_seq":2609,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:05.884139+0000","last_change":"2024-04-25T00:01:05.884161+0000","last_active":"2024-04-25T00:01:05.884139+0000","last_peered":"2024-04-25T00:01:05.884139+0000","last_clean":"2024-04-25T00:01:05.884139+0000","last_became_active":"2024-04-24T23:34:54.376019+0000","last_became_peered":"2024-04-24T23:34:54.376019+0000","last_unstale":"2024-04-25T00:01:05.884139+0000","last_undegraded":"2024-04-25T00:01:05.884139+0000","last_fullsized":"2024-04-25T00:01:05.884139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2566","last_scrub_stamp":"2024-04-25T00:01:05.884075+0000","last_deep_scrub":"27'2566","last_deep_scrub_stamp":"2024-04-25T00:01:05.884075+0000","last_clean_scrub_stamp":"2024-04-25T00:01:05.884075+0000","objects_scrubbed":0,"log_size":2566,"log_dups_size":0,"ondisk_log_size":2566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:26:32.174295+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"27'8765","reported_seq":9274,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:02.681788+0000","last_change":"2024-04-25T00:01:02.681812+0000","last_active":"2024-04-25T00:01:02.681788+0000","last_peered":"2024-04-25T00:01:02.681788+0000","last_clean":"2024-04-25T00:01:02.681788+0000","last_became_active":"2024-04-24T23:34:58.396658+0000","last_became_peered":"2024-04-24T23:34:58.396658+0000","last_unstale":"2024-04-25T00:01:02.681788+0000","last_undegraded":"2024-04-25T00:01:02.681788+0000","last_fullsized":"2024-04-25T00:01:02.681788+0000","mapping_epoch":23,"log_start":"27'1609","ondisk_log_start":"27'1609","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8765","last_scrub_stamp":"2024-04-25T00:01:02.681708+0000","last_deep_scrub":"27'8765","last_deep_scrub_stamp":"2024-04-25T00:01:02.681708+0000","last_clean_scrub_stamp":"2024-04-25T00:01:02.681708+0000","objects_scrubbed":0,"log_size":7156,"log_dups_size":0,"ondisk_log_size":7156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:08:10.518212+0000","scrub_duration":45,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":472,"num_read_kb":119888,"num_write":5982,"num_write_kb":849049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"27'2561","reported_seq":2606,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:05.138692+0000","last_change":"2024-04-25T00:01:05.138692+0000","last_active":"2024-04-25T00:01:05.138692+0000","last_peered":"2024-04-25T00:01:05.138692+0000","last_clean":"2024-04-25T00:01:05.138692+0000","last_became_active":"2024-04-24T23:34:54.377576+0000","last_became_peered":"2024-04-24T23:34:54.377576+0000","last_unstale":"2024-04-25T00:01:05.138692+0000","last_undegraded":"2024-04-25T00:01:05.138692+0000","last_fullsized":"2024-04-25T00:01:05.138692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2561","last_scrub_stamp":"2024-04-25T00:01:05.138606+0000","last_deep_scrub":"27'2561","last_deep_scrub_stamp":"2024-04-25T00:01:05.138606+0000","last_clean_scrub_stamp":"2024-04-25T00:01:05.138606+0000","objects_scrubbed":0,"log_size":2561,"log_dups_size":0,"ondisk_log_size":2561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:18:21.245687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"27'6838","reported_seq":6883,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:01.026815+0000","last_change":"2024-04-25T00:01:01.026856+0000","last_active":"2024-04-25T00:01:01.026815+0000","last_peered":"2024-04-25T00:01:01.026815+0000","last_clean":"2024-04-25T00:01:01.026815+0000","last_became_active":"2024-04-24T23:34:53.360635+0000","last_became_peered":"2024-04-24T23:34:53.360635+0000","last_unstale":"2024-04-25T00:01:01.026815+0000","last_undegraded":"2024-04-25T00:01:01.026815+0000","last_fullsized":"2024-04-25T00:01:01.026815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6838","last_scrub_stamp":"2024-04-25T00:01:01.026748+0000","last_deep_scrub":"27'6838","last_deep_scrub_stamp":"2024-04-25T00:01:01.026748+0000","last_clean_scrub_stamp":"2024-04-25T00:01:01.026748+0000","objects_scrubbed":4,"log_size":6838,"log_dups_size":0,"ondisk_log_size":6838,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:31:16.018413+0000","scrub_duration":64,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6840,"num_write_kb":53782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":246,"num_omap_keys":1,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"27'10213","reported_seq":10504,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:09.697769+0000","last_change":"2024-04-25T00:01:09.697805+0000","last_active":"2024-04-25T00:01:09.697769+0000","last_peered":"2024-04-25T00:01:09.697769+0000","last_clean":"2024-04-25T00:01:09.697769+0000","last_became_active":"2024-04-24T23:34:58.406844+0000","last_became_peered":"2024-04-24T23:34:58.406844+0000","last_unstale":"2024-04-25T00:01:09.697769+0000","last_undegraded":"2024-04-25T00:01:09.697769+0000","last_fullsized":"2024-04-25T00:01:09.697769+0000","mapping_epoch":23,"log_start":"27'3041","ondisk_log_start":"27'3041","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'10213","last_scrub_stamp":"2024-04-25T00:01:09.697684+0000","last_deep_scrub":"27'10213","last_deep_scrub_stamp":"2024-04-25T00:01:09.697684+0000","last_clean_scrub_stamp":"2024-04-25T00:01:09.697684+0000","objects_scrubbed":0,"log_size":7172,"log_dups_size":0,"ondisk_log_size":7172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:00.362322+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":253,"num_read_kb":64262,"num_write":7465,"num_write_kb":1030177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"27'2579","reported_seq":2623,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:06.203818+0000","last_change":"2024-04-25T00:01:06.203841+0000","last_active":"2024-04-25T00:01:06.203818+0000","last_peered":"2024-04-25T00:01:06.203818+0000","last_clean":"2024-04-25T00:01:06.203818+0000","last_became_active":"2024-04-24T23:34:54.366426+0000","last_became_peered":"2024-04-24T23:34:54.366426+0000","last_unstale":"2024-04-25T00:01:06.203818+0000","last_undegraded":"2024-04-25T00:01:06.203818+0000","last_fullsized":"2024-04-25T00:01:06.203818+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2579","last_scrub_stamp":"2024-04-25T00:01:06.203744+0000","last_deep_scrub":"27'2579","last_deep_scrub_stamp":"2024-04-25T00:01:06.203744+0000","last_clean_scrub_stamp":"2024-04-25T00:01:06.203744+0000","objects_scrubbed":0,"log_size":2579,"log_dups_size":0,"ondisk_log_size":2579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:54.928622+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"27'7915","reported_seq":7962,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:59.098554+0000","last_change":"2024-04-25T00:00:59.098554+0000","last_active":"2024-04-25T00:00:59.098554+0000","last_peered":"2024-04-25T00:00:59.098554+0000","last_clean":"2024-04-25T00:00:59.098554+0000","last_became_active":"2024-04-24T23:34:53.358841+0000","last_became_peered":"2024-04-24T23:34:53.358841+0000","last_unstale":"2024-04-25T00:00:59.098554+0000","last_undegraded":"2024-04-25T00:00:59.098554+0000","last_fullsized":"2024-04-25T00:00:59.098554+0000","mapping_epoch":18,"log_start":"27'500","ondisk_log_start":"27'500","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7915","last_scrub_stamp":"2024-04-25T00:00:59.098404+0000","last_deep_scrub":"27'7915","last_deep_scrub_stamp":"2024-04-25T00:00:59.098404+0000","last_clean_scrub_stamp":"2024-04-25T00:00:59.098404+0000","objects_scrubbed":2,"log_size":7415,"log_dups_size":0,"ondisk_log_size":7415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:59:04.579615+0000","scrub_duration":41,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7912,"num_write_kb":58475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"27'9714","reported_seq":10155,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:06.714444+0000","last_change":"2024-04-25T00:01:06.714487+0000","last_active":"2024-04-25T00:01:06.714444+0000","last_peered":"2024-04-25T00:01:06.714444+0000","last_clean":"2024-04-25T00:01:06.714444+0000","last_became_active":"2024-04-24T23:34:58.399375+0000","last_became_peered":"2024-04-24T23:34:58.399375+0000","last_unstale":"2024-04-25T00:01:06.714444+0000","last_undegraded":"2024-04-25T00:01:06.714444+0000","last_fullsized":"2024-04-25T00:01:06.714444+0000","mapping_epoch":23,"log_start":"27'2556","ondisk_log_start":"27'2556","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9714","last_scrub_stamp":"2024-04-25T00:01:06.714325+0000","last_deep_scrub":"27'9714","last_deep_scrub_stamp":"2024-04-25T00:01:06.714325+0000","last_clean_scrub_stamp":"2024-04-25T00:01:06.714325+0000","objects_scrubbed":0,"log_size":7158,"log_dups_size":0,"ondisk_log_size":7158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:07:54.046986+0000","scrub_duration":29,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":403,"num_read_kb":102362,"num_write":6994,"num_write_kb":969161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"27'2649","reported_seq":2694,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:05.228281+0000","last_change":"2024-04-25T00:01:05.228304+0000","last_active":"2024-04-25T00:01:05.228281+0000","last_peered":"2024-04-25T00:01:05.228281+0000","last_clean":"2024-04-25T00:01:05.228281+0000","last_became_active":"2024-04-24T23:34:54.364744+0000","last_became_peered":"2024-04-24T23:34:54.364744+0000","last_unstale":"2024-04-25T00:01:05.228281+0000","last_undegraded":"2024-04-25T00:01:05.228281+0000","last_fullsized":"2024-04-25T00:01:05.228281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2649","last_scrub_stamp":"2024-04-25T00:01:05.228201+0000","last_deep_scrub":"27'2649","last_deep_scrub_stamp":"2024-04-25T00:01:05.228201+0000","last_clean_scrub_stamp":"2024-04-25T00:01:05.228201+0000","objects_scrubbed":0,"log_size":2649,"log_dups_size":0,"ondisk_log_size":2649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:08:45.592988+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"27'4022","reported_seq":4069,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:01.102491+0000","last_change":"2024-04-25T00:01:01.102519+0000","last_active":"2024-04-25T00:01:01.102491+0000","last_peered":"2024-04-25T00:01:01.102491+0000","last_clean":"2024-04-25T00:01:01.102491+0000","last_became_active":"2024-04-24T23:34:53.360570+0000","last_became_peered":"2024-04-24T23:34:53.360570+0000","last_unstale":"2024-04-25T00:01:01.102491+0000","last_undegraded":"2024-04-25T00:01:01.102491+0000","last_fullsized":"2024-04-25T00:01:01.102491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4022","last_scrub_stamp":"2024-04-25T00:01:01.102425+0000","last_deep_scrub":"27'4022","last_deep_scrub_stamp":"2024-04-25T00:01:01.102425+0000","last_clean_scrub_stamp":"2024-04-25T00:01:01.102425+0000","objects_scrubbed":0,"log_size":4022,"log_dups_size":0,"ondisk_log_size":4022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:45:00.864134+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4022,"num_write_kb":27320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"27'9656","reported_seq":10086,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:11.200114+0000","last_change":"2024-04-25T00:01:11.200158+0000","last_active":"2024-04-25T00:01:11.200114+0000","last_peered":"2024-04-25T00:01:11.200114+0000","last_clean":"2024-04-25T00:01:11.200114+0000","last_became_active":"2024-04-24T23:34:58.731485+0000","last_became_peered":"2024-04-24T23:34:58.731485+0000","last_unstale":"2024-04-25T00:01:11.200114+0000","last_undegraded":"2024-04-25T00:01:11.200114+0000","last_fullsized":"2024-04-25T00:01:11.200114+0000","mapping_epoch":23,"log_start":"27'2316","ondisk_log_start":"27'2316","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9656","last_scrub_stamp":"2024-04-25T00:01:11.199988+0000","last_deep_scrub":"27'9656","last_deep_scrub_stamp":"2024-04-25T00:01:11.199988+0000","last_clean_scrub_stamp":"2024-04-25T00:01:11.199988+0000","objects_scrubbed":0,"log_size":7340,"log_dups_size":0,"ondisk_log_size":7340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:04:35.457528+0000","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":392,"num_read_kb":99568,"num_write":7039,"num_write_kb":904216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"27'2554","reported_seq":2599,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:57.711687+0000","last_change":"2024-04-25T00:00:57.711687+0000","last_active":"2024-04-25T00:00:57.711687+0000","last_peered":"2024-04-25T00:00:57.711687+0000","last_clean":"2024-04-25T00:00:57.711687+0000","last_became_active":"2024-04-24T23:34:54.362141+0000","last_became_peered":"2024-04-24T23:34:54.362141+0000","last_unstale":"2024-04-25T00:00:57.711687+0000","last_undegraded":"2024-04-25T00:00:57.711687+0000","last_fullsized":"2024-04-25T00:00:57.711687+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2554","last_scrub_stamp":"2024-04-25T00:00:57.711603+0000","last_deep_scrub":"27'2554","last_deep_scrub_stamp":"2024-04-25T00:00:57.711603+0000","last_clean_scrub_stamp":"2024-04-25T00:00:57.711603+0000","objects_scrubbed":0,"log_size":2554,"log_dups_size":0,"ondisk_log_size":2554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:02:44.568864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"27'2571","reported_seq":2617,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:04.967048+0000","last_change":"2024-04-25T00:01:04.967073+0000","last_active":"2024-04-25T00:01:04.967048+0000","last_peered":"2024-04-25T00:01:04.967048+0000","last_clean":"2024-04-25T00:01:04.967048+0000","last_became_active":"2024-04-24T23:34:53.366110+0000","last_became_peered":"2024-04-24T23:34:53.366110+0000","last_unstale":"2024-04-25T00:01:04.967048+0000","last_undegraded":"2024-04-25T00:01:04.967048+0000","last_fullsized":"2024-04-25T00:01:04.967048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2571","last_scrub_stamp":"2024-04-25T00:01:04.966976+0000","last_deep_scrub":"27'2571","last_deep_scrub_stamp":"2024-04-25T00:01:04.966976+0000","last_clean_scrub_stamp":"2024-04-25T00:01:04.966976+0000","objects_scrubbed":1,"log_size":2571,"log_dups_size":0,"ondisk_log_size":2571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:49:24.660148+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2576,"num_write_kb":22186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"27'9201","reported_seq":9390,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:07.712985+0000","last_change":"2024-04-25T00:01:07.713013+0000","last_active":"2024-04-25T00:01:07.712985+0000","last_peered":"2024-04-25T00:01:07.712985+0000","last_clean":"2024-04-25T00:01:07.712985+0000","last_became_active":"2024-04-24T23:34:58.402365+0000","last_became_peered":"2024-04-24T23:34:58.402365+0000","last_unstale":"2024-04-25T00:01:07.712985+0000","last_undegraded":"2024-04-25T00:01:07.712985+0000","last_fullsized":"2024-04-25T00:01:07.712985+0000","mapping_epoch":23,"log_start":"27'2049","ondisk_log_start":"27'2049","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9201","last_scrub_stamp":"2024-04-25T00:01:07.712900+0000","last_deep_scrub":"27'9201","last_deep_scrub_stamp":"2024-04-25T00:01:07.712900+0000","last_clean_scrub_stamp":"2024-04-25T00:01:07.712900+0000","objects_scrubbed":0,"log_size":7152,"log_dups_size":0,"ondisk_log_size":7152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:29:25.087812+0000","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":151,"num_read_kb":38354,"num_write":6553,"num_write_kb":977677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"27'2524","reported_seq":2569,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:01.012496+0000","last_change":"2024-04-25T00:01:01.012517+0000","last_active":"2024-04-25T00:01:01.012496+0000","last_peered":"2024-04-25T00:01:01.012496+0000","last_clean":"2024-04-25T00:01:01.012496+0000","last_became_active":"2024-04-24T23:34:54.369835+0000","last_became_peered":"2024-04-24T23:34:54.369835+0000","last_unstale":"2024-04-25T00:01:01.012496+0000","last_undegraded":"2024-04-25T00:01:01.012496+0000","last_fullsized":"2024-04-25T00:01:01.012496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2524","last_scrub_stamp":"2024-04-25T00:01:01.012421+0000","last_deep_scrub":"27'2524","last_deep_scrub_stamp":"2024-04-25T00:01:01.012421+0000","last_clean_scrub_stamp":"2024-04-25T00:01:01.012421+0000","objects_scrubbed":0,"log_size":2524,"log_dups_size":0,"ondisk_log_size":2524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:37:04.673623+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"27'4488","reported_seq":4535,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:01.248458+0000","last_change":"2024-04-25T00:01:01.248513+0000","last_active":"2024-04-25T00:01:01.248458+0000","last_peered":"2024-04-25T00:01:01.248458+0000","last_clean":"2024-04-25T00:01:01.248458+0000","last_became_active":"2024-04-24T23:34:53.365783+0000","last_became_peered":"2024-04-24T23:34:53.365783+0000","last_unstale":"2024-04-25T00:01:01.248458+0000","last_undegraded":"2024-04-25T00:01:01.248458+0000","last_fullsized":"2024-04-25T00:01:01.248458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4488","last_scrub_stamp":"2024-04-25T00:01:01.248363+0000","last_deep_scrub":"27'4488","last_deep_scrub_stamp":"2024-04-25T00:01:01.248363+0000","last_clean_scrub_stamp":"2024-04-25T00:01:01.248363+0000","objects_scrubbed":4,"log_size":4488,"log_dups_size":0,"ondisk_log_size":4488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:59:36.082415+0000","scrub_duration":48,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":4494,"num_write_kb":39872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"27'9221","reported_seq":9450,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:10.696821+0000","last_change":"2024-04-25T00:01:10.696849+0000","last_active":"2024-04-25T00:01:10.696821+0000","last_peered":"2024-04-25T00:01:10.696821+0000","last_clean":"2024-04-25T00:01:10.696821+0000","last_became_active":"2024-04-24T23:34:58.727653+0000","last_became_peered":"2024-04-24T23:34:58.727653+0000","last_unstale":"2024-04-25T00:01:10.696821+0000","last_undegraded":"2024-04-25T00:01:10.696821+0000","last_fullsized":"2024-04-25T00:01:10.696821+0000","mapping_epoch":23,"log_start":"27'2006","ondisk_log_start":"27'2006","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9221","last_scrub_stamp":"2024-04-25T00:01:10.696738+0000","last_deep_scrub":"27'9221","last_deep_scrub_stamp":"2024-04-25T00:01:10.696738+0000","last_clean_scrub_stamp":"2024-04-25T00:01:10.696738+0000","objects_scrubbed":0,"log_size":7215,"log_dups_size":0,"ondisk_log_size":7215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:47.378522+0000","scrub_duration":29,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":190,"num_read_kb":48260,"num_write":6573,"num_write_kb":840416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"27'2508","reported_seq":2553,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:07.250632+0000","last_change":"2024-04-25T00:01:07.250655+0000","last_active":"2024-04-25T00:01:07.250632+0000","last_peered":"2024-04-25T00:01:07.250632+0000","last_clean":"2024-04-25T00:01:07.250632+0000","last_became_active":"2024-04-24T23:34:54.367991+0000","last_became_peered":"2024-04-24T23:34:54.367991+0000","last_unstale":"2024-04-25T00:01:07.250632+0000","last_undegraded":"2024-04-25T00:01:07.250632+0000","last_fullsized":"2024-04-25T00:01:07.250632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2508","last_scrub_stamp":"2024-04-25T00:01:07.250562+0000","last_deep_scrub":"27'2508","last_deep_scrub_stamp":"2024-04-25T00:01:07.250562+0000","last_clean_scrub_stamp":"2024-04-25T00:01:07.250562+0000","objects_scrubbed":0,"log_size":2508,"log_dups_size":0,"ondisk_log_size":2508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:45:34.404171+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"27'7476","reported_seq":8148,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:02.196749+0000","last_change":"2024-04-25T00:01:02.196778+0000","last_active":"2024-04-25T00:01:02.196749+0000","last_peered":"2024-04-25T00:01:02.196749+0000","last_clean":"2024-04-25T00:01:02.196749+0000","last_became_active":"2024-04-24T23:34:53.363842+0000","last_became_peered":"2024-04-24T23:34:53.363842+0000","last_unstale":"2024-04-25T00:01:02.196749+0000","last_undegraded":"2024-04-25T00:01:02.196749+0000","last_fullsized":"2024-04-25T00:01:02.196749+0000","mapping_epoch":18,"log_start":"27'100","ondisk_log_start":"27'100","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7476","last_scrub_stamp":"2024-04-25T00:01:02.196688+0000","last_deep_scrub":"27'7476","last_deep_scrub_stamp":"2024-04-25T00:01:02.196688+0000","last_clean_scrub_stamp":"2024-04-25T00:01:02.196688+0000","objects_scrubbed":3,"log_size":7376,"log_dups_size":0,"ondisk_log_size":7376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:49:21.889860+0000","scrub_duration":111,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":626,"num_read_kb":4509,"num_write":10321,"num_write_kb":61532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"27'8920","reported_seq":9238,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:13.312090+0000","last_change":"2024-04-25T00:01:13.312133+0000","last_active":"2024-04-25T00:01:13.312090+0000","last_peered":"2024-04-25T00:01:13.312090+0000","last_clean":"2024-04-25T00:01:13.312090+0000","last_became_active":"2024-04-24T23:34:58.729082+0000","last_became_peered":"2024-04-24T23:34:58.729082+0000","last_unstale":"2024-04-25T00:01:13.312090+0000","last_undegraded":"2024-04-25T00:01:13.312090+0000","last_fullsized":"2024-04-25T00:01:13.312090+0000","mapping_epoch":23,"log_start":"27'2609","ondisk_log_start":"27'2609","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8920","last_scrub_stamp":"2024-04-25T00:01:13.311992+0000","last_deep_scrub":"27'8920","last_deep_scrub_stamp":"2024-04-25T00:01:13.311992+0000","last_clean_scrub_stamp":"2024-04-25T00:01:13.311992+0000","objects_scrubbed":0,"log_size":6311,"log_dups_size":0,"ondisk_log_size":6311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:17:28.784349+0000","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":278,"num_read_kb":70612,"num_write":6238,"num_write_kb":899702,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"27'2598","reported_seq":2643,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:04.112185+0000","last_change":"2024-04-25T00:01:04.112210+0000","last_active":"2024-04-25T00:01:04.112185+0000","last_peered":"2024-04-25T00:01:04.112185+0000","last_clean":"2024-04-25T00:01:04.112185+0000","last_became_active":"2024-04-24T23:34:54.356560+0000","last_became_peered":"2024-04-24T23:34:54.356560+0000","last_unstale":"2024-04-25T00:01:04.112185+0000","last_undegraded":"2024-04-25T00:01:04.112185+0000","last_fullsized":"2024-04-25T00:01:04.112185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2598","last_scrub_stamp":"2024-04-25T00:01:04.112109+0000","last_deep_scrub":"27'2598","last_deep_scrub_stamp":"2024-04-25T00:01:04.112109+0000","last_clean_scrub_stamp":"2024-04-25T00:01:04.112109+0000","objects_scrubbed":0,"log_size":2598,"log_dups_size":0,"ondisk_log_size":2598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:07:51.444770+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"27'6263","reported_seq":6309,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:03.994566+0000","last_change":"2024-04-25T00:01:03.994625+0000","last_active":"2024-04-25T00:01:03.994566+0000","last_peered":"2024-04-25T00:01:03.994566+0000","last_clean":"2024-04-25T00:01:03.994566+0000","last_became_active":"2024-04-24T23:34:53.365636+0000","last_became_peered":"2024-04-24T23:34:53.365636+0000","last_unstale":"2024-04-25T00:01:03.994566+0000","last_undegraded":"2024-04-25T00:01:03.994566+0000","last_fullsized":"2024-04-25T00:01:03.994566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6263","last_scrub_stamp":"2024-04-25T00:01:03.994496+0000","last_deep_scrub":"27'6263","last_deep_scrub_stamp":"2024-04-25T00:01:03.994496+0000","last_clean_scrub_stamp":"2024-04-25T00:01:03.994496+0000","objects_scrubbed":2,"log_size":6263,"log_dups_size":0,"ondisk_log_size":6263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:45:03.756205+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":6274,"num_write_kb":49208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"27'9152","reported_seq":9695,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:09.005063+0000","last_change":"2024-04-25T00:01:09.005107+0000","last_active":"2024-04-25T00:01:09.005063+0000","last_peered":"2024-04-25T00:01:09.005063+0000","last_clean":"2024-04-25T00:01:09.005063+0000","last_became_active":"2024-04-24T23:34:58.727885+0000","last_became_peered":"2024-04-24T23:34:58.727885+0000","last_unstale":"2024-04-25T00:01:09.005063+0000","last_undegraded":"2024-04-25T00:01:09.005063+0000","last_fullsized":"2024-04-25T00:01:09.005063+0000","mapping_epoch":23,"log_start":"27'1968","ondisk_log_start":"27'1968","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9152","last_scrub_stamp":"2024-04-25T00:01:09.004937+0000","last_deep_scrub":"27'9152","last_deep_scrub_stamp":"2024-04-25T00:01:09.004937+0000","last_clean_scrub_stamp":"2024-04-25T00:01:09.004937+0000","objects_scrubbed":0,"log_size":7184,"log_dups_size":0,"ondisk_log_size":7184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:45.686721+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":504,"num_read_kb":128016,"num_write":6457,"num_write_kb":842362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"27'2505","reported_seq":2550,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:01.936748+0000","last_change":"2024-04-25T00:01:01.936790+0000","last_active":"2024-04-25T00:01:01.936748+0000","last_peered":"2024-04-25T00:01:01.936748+0000","last_clean":"2024-04-25T00:01:01.936748+0000","last_became_active":"2024-04-24T23:34:54.371947+0000","last_became_peered":"2024-04-24T23:34:54.371947+0000","last_unstale":"2024-04-25T00:01:01.936748+0000","last_undegraded":"2024-04-25T00:01:01.936748+0000","last_fullsized":"2024-04-25T00:01:01.936748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2505","last_scrub_stamp":"2024-04-25T00:01:01.936631+0000","last_deep_scrub":"27'2505","last_deep_scrub_stamp":"2024-04-25T00:01:01.936631+0000","last_clean_scrub_stamp":"2024-04-25T00:01:01.936631+0000","objects_scrubbed":0,"log_size":2505,"log_dups_size":0,"ondisk_log_size":2505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:49:21.629803+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"27'7230","reported_seq":7277,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:00.113053+0000","last_change":"2024-04-25T00:01:00.113141+0000","last_active":"2024-04-25T00:01:00.113053+0000","last_peered":"2024-04-25T00:01:00.113053+0000","last_clean":"2024-04-25T00:01:00.113053+0000","last_became_active":"2024-04-24T23:34:53.360082+0000","last_became_peered":"2024-04-24T23:34:53.360082+0000","last_unstale":"2024-04-25T00:01:00.113053+0000","last_undegraded":"2024-04-25T00:01:00.113053+0000","last_fullsized":"2024-04-25T00:01:00.113053+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7230","last_scrub_stamp":"2024-04-25T00:01:00.112956+0000","last_deep_scrub":"27'7230","last_deep_scrub_stamp":"2024-04-25T00:01:00.112956+0000","last_clean_scrub_stamp":"2024-04-25T00:01:00.112956+0000","objects_scrubbed":5,"log_size":7230,"log_dups_size":0,"ondisk_log_size":7230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:08:07.949460+0000","scrub_duration":63,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17487510,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":7224,"num_write_kb":70858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"27'9197","reported_seq":9391,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:10.157328+0000","last_change":"2024-04-25T00:01:10.157355+0000","last_active":"2024-04-25T00:01:10.157328+0000","last_peered":"2024-04-25T00:01:10.157328+0000","last_clean":"2024-04-25T00:01:10.157328+0000","last_became_active":"2024-04-24T23:34:58.403735+0000","last_became_peered":"2024-04-24T23:34:58.403735+0000","last_unstale":"2024-04-25T00:01:10.157328+0000","last_undegraded":"2024-04-25T00:01:10.157328+0000","last_fullsized":"2024-04-25T00:01:10.157328+0000","mapping_epoch":23,"log_start":"27'1819","ondisk_log_start":"27'1819","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9197","last_scrub_stamp":"2024-04-25T00:01:10.157243+0000","last_deep_scrub":"27'9197","last_deep_scrub_stamp":"2024-04-25T00:01:10.157243+0000","last_clean_scrub_stamp":"2024-04-25T00:01:10.157243+0000","objects_scrubbed":0,"log_size":7378,"log_dups_size":0,"ondisk_log_size":7378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:00.083638+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":155,"num_read_kb":39370,"num_write":6529,"num_write_kb":924891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"27'2542","reported_seq":2587,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:59.692895+0000","last_change":"2024-04-25T00:00:59.692933+0000","last_active":"2024-04-25T00:00:59.692895+0000","last_peered":"2024-04-25T00:00:59.692895+0000","last_clean":"2024-04-25T00:00:59.692895+0000","last_became_active":"2024-04-24T23:34:54.367355+0000","last_became_peered":"2024-04-24T23:34:54.367355+0000","last_unstale":"2024-04-25T00:00:59.692895+0000","last_undegraded":"2024-04-25T00:00:59.692895+0000","last_fullsized":"2024-04-25T00:00:59.692895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2542","last_scrub_stamp":"2024-04-25T00:00:59.692781+0000","last_deep_scrub":"27'2542","last_deep_scrub_stamp":"2024-04-25T00:00:59.692781+0000","last_clean_scrub_stamp":"2024-04-25T00:00:59.692781+0000","objects_scrubbed":0,"log_size":2542,"log_dups_size":0,"ondisk_log_size":2542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:15.110780+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"27'8390","reported_seq":8437,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:57.031782+0000","last_change":"2024-04-25T00:00:57.031834+0000","last_active":"2024-04-25T00:00:57.031782+0000","last_peered":"2024-04-25T00:00:57.031782+0000","last_clean":"2024-04-25T00:00:57.031782+0000","last_became_active":"2024-04-24T23:34:53.355710+0000","last_became_peered":"2024-04-24T23:34:53.355710+0000","last_unstale":"2024-04-25T00:00:57.031782+0000","last_undegraded":"2024-04-25T00:00:57.031782+0000","last_fullsized":"2024-04-25T00:00:57.031782+0000","mapping_epoch":18,"log_start":"27'600","ondisk_log_start":"27'600","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8390","last_scrub_stamp":"2024-04-25T00:00:57.031693+0000","last_deep_scrub":"27'8390","last_deep_scrub_stamp":"2024-04-25T00:00:57.031693+0000","last_clean_scrub_stamp":"2024-04-25T00:00:57.031693+0000","objects_scrubbed":2,"log_size":7790,"log_dups_size":0,"ondisk_log_size":7790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:12.449692+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":8401,"num_write_kb":63182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"27'9677","reported_seq":10050,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:08.156503+0000","last_change":"2024-04-25T00:01:08.156527+0000","last_active":"2024-04-25T00:01:08.156503+0000","last_peered":"2024-04-25T00:01:08.156503+0000","last_clean":"2024-04-25T00:01:08.156503+0000","last_became_active":"2024-04-24T23:34:58.402598+0000","last_became_peered":"2024-04-24T23:34:58.402598+0000","last_unstale":"2024-04-25T00:01:08.156503+0000","last_undegraded":"2024-04-25T00:01:08.156503+0000","last_fullsized":"2024-04-25T00:01:08.156503+0000","mapping_epoch":23,"log_start":"27'2299","ondisk_log_start":"27'2299","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9677","last_scrub_stamp":"2024-04-25T00:01:08.156410+0000","last_deep_scrub":"27'9677","last_deep_scrub_stamp":"2024-04-25T00:01:08.156410+0000","last_clean_scrub_stamp":"2024-04-25T00:01:08.156410+0000","objects_scrubbed":0,"log_size":7378,"log_dups_size":0,"ondisk_log_size":7378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:08.786202+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":334,"num_read_kb":84836,"num_write":6883,"num_write_kb":886687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"27'2549","reported_seq":2595,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:03.081113+0000","last_change":"2024-04-25T00:01:03.081113+0000","last_active":"2024-04-25T00:01:03.081113+0000","last_peered":"2024-04-25T00:01:03.081113+0000","last_clean":"2024-04-25T00:01:03.081113+0000","last_became_active":"2024-04-24T23:34:54.356326+0000","last_became_peered":"2024-04-24T23:34:54.356326+0000","last_unstale":"2024-04-25T00:01:03.081113+0000","last_undegraded":"2024-04-25T00:01:03.081113+0000","last_fullsized":"2024-04-25T00:01:03.081113+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2549","last_scrub_stamp":"2024-04-25T00:01:03.081031+0000","last_deep_scrub":"27'2549","last_deep_scrub_stamp":"2024-04-25T00:01:03.081031+0000","last_clean_scrub_stamp":"2024-04-25T00:01:03.081031+0000","objects_scrubbed":0,"log_size":2549,"log_dups_size":0,"ondisk_log_size":2549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:42:19.039285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"27'7179","reported_seq":8155,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:58.015120+0000","last_change":"2024-04-25T00:00:58.015150+0000","last_active":"2024-04-25T00:00:58.015120+0000","last_peered":"2024-04-25T00:00:58.015120+0000","last_clean":"2024-04-25T00:00:58.015120+0000","last_became_active":"2024-04-24T23:34:53.358937+0000","last_became_peered":"2024-04-24T23:34:53.358937+0000","last_unstale":"2024-04-25T00:00:58.015120+0000","last_undegraded":"2024-04-25T00:00:58.015120+0000","last_fullsized":"2024-04-25T00:00:58.015120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7179","last_scrub_stamp":"2024-04-25T00:00:58.015059+0000","last_deep_scrub":"27'7179","last_deep_scrub_stamp":"2024-04-25T00:00:58.015059+0000","last_clean_scrub_stamp":"2024-04-25T00:00:58.015059+0000","objects_scrubbed":2,"log_size":7179,"log_dups_size":0,"ondisk_log_size":7179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:13:14.047029+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":929,"num_read_kb":4700,"num_write":7175,"num_write_kb":61982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"27'9030","reported_seq":9633,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:09.134170+0000","last_change":"2024-04-25T00:01:09.134230+0000","last_active":"2024-04-25T00:01:09.134170+0000","last_peered":"2024-04-25T00:01:09.134170+0000","last_clean":"2024-04-25T00:01:09.134170+0000","last_became_active":"2024-04-24T23:34:58.402569+0000","last_became_peered":"2024-04-24T23:34:58.402569+0000","last_unstale":"2024-04-25T00:01:09.134170+0000","last_undegraded":"2024-04-25T00:01:09.134170+0000","last_fullsized":"2024-04-25T00:01:09.134170+0000","mapping_epoch":23,"log_start":"27'1638","ondisk_log_start":"27'1638","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9030","last_scrub_stamp":"2024-04-25T00:01:09.134031+0000","last_deep_scrub":"27'9030","last_deep_scrub_stamp":"2024-04-25T00:01:09.134031+0000","last_clean_scrub_stamp":"2024-04-25T00:01:09.134031+0000","objects_scrubbed":0,"log_size":7392,"log_dups_size":0,"ondisk_log_size":7392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:15:13.537860+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":564,"num_read_kb":143256,"num_write":6307,"num_write_kb":911438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"27'2483","reported_seq":2528,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:03.975184+0000","last_change":"2024-04-25T00:01:03.975224+0000","last_active":"2024-04-25T00:01:03.975184+0000","last_peered":"2024-04-25T00:01:03.975184+0000","last_clean":"2024-04-25T00:01:03.975184+0000","last_became_active":"2024-04-24T23:34:54.377228+0000","last_became_peered":"2024-04-24T23:34:54.377228+0000","last_unstale":"2024-04-25T00:01:03.975184+0000","last_undegraded":"2024-04-25T00:01:03.975184+0000","last_fullsized":"2024-04-25T00:01:03.975184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2483","last_scrub_stamp":"2024-04-25T00:01:03.975085+0000","last_deep_scrub":"27'2483","last_deep_scrub_stamp":"2024-04-25T00:01:03.975085+0000","last_clean_scrub_stamp":"2024-04-25T00:01:03.975085+0000","objects_scrubbed":0,"log_size":2483,"log_dups_size":0,"ondisk_log_size":2483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:29:04.141244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"27'7207","reported_seq":7254,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:01.942254+0000","last_change":"2024-04-25T00:01:01.942296+0000","last_active":"2024-04-25T00:01:01.942254+0000","last_peered":"2024-04-25T00:01:01.942254+0000","last_clean":"2024-04-25T00:01:01.942254+0000","last_became_active":"2024-04-24T23:34:53.362425+0000","last_became_peered":"2024-04-24T23:34:53.362425+0000","last_unstale":"2024-04-25T00:01:01.942254+0000","last_undegraded":"2024-04-25T00:01:01.942254+0000","last_fullsized":"2024-04-25T00:01:01.942254+0000","mapping_epoch":18,"log_start":"27'300","ondisk_log_start":"27'300","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7207","last_scrub_stamp":"2024-04-25T00:01:01.942161+0000","last_deep_scrub":"27'7207","last_deep_scrub_stamp":"2024-04-25T00:01:01.942161+0000","last_clean_scrub_stamp":"2024-04-25T00:01:01.942161+0000","objects_scrubbed":1,"log_size":6907,"log_dups_size":0,"ondisk_log_size":6907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:59:07.423372+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":7208,"num_write_kb":58030,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"27'9329","reported_seq":9861,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:09.964607+0000","last_change":"2024-04-25T00:01:09.964632+0000","last_active":"2024-04-25T00:01:09.964607+0000","last_peered":"2024-04-25T00:01:09.964607+0000","last_clean":"2024-04-25T00:01:09.964607+0000","last_became_active":"2024-04-24T23:34:58.727284+0000","last_became_peered":"2024-04-24T23:34:58.727284+0000","last_unstale":"2024-04-25T00:01:09.964607+0000","last_undegraded":"2024-04-25T00:01:09.964607+0000","last_fullsized":"2024-04-25T00:01:09.964607+0000","mapping_epoch":23,"log_start":"27'2114","ondisk_log_start":"27'2114","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9329","last_scrub_stamp":"2024-04-25T00:01:09.964510+0000","last_deep_scrub":"27'9329","last_deep_scrub_stamp":"2024-04-25T00:01:09.964510+0000","last_clean_scrub_stamp":"2024-04-25T00:01:09.964510+0000","objects_scrubbed":0,"log_size":7215,"log_dups_size":0,"ondisk_log_size":7215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:59:06.046664+0000","scrub_duration":30,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":493,"num_read_kb":125222,"num_write":6691,"num_write_kb":1110316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"27'2597","reported_seq":2642,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:08.253077+0000","last_change":"2024-04-25T00:01:08.253097+0000","last_active":"2024-04-25T00:01:08.253077+0000","last_peered":"2024-04-25T00:01:08.253077+0000","last_clean":"2024-04-25T00:01:08.253077+0000","last_became_active":"2024-04-24T23:34:54.382564+0000","last_became_peered":"2024-04-24T23:34:54.382564+0000","last_unstale":"2024-04-25T00:01:08.253077+0000","last_undegraded":"2024-04-25T00:01:08.253077+0000","last_fullsized":"2024-04-25T00:01:08.253077+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2597","last_scrub_stamp":"2024-04-25T00:01:08.253016+0000","last_deep_scrub":"27'2597","last_deep_scrub_stamp":"2024-04-25T00:01:08.253016+0000","last_clean_scrub_stamp":"2024-04-25T00:01:08.253016+0000","objects_scrubbed":0,"log_size":2597,"log_dups_size":0,"ondisk_log_size":2597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:28.022070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"27'5190","reported_seq":5237,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:57.975969+0000","last_change":"2024-04-25T00:00:57.976027+0000","last_active":"2024-04-25T00:00:57.975969+0000","last_peered":"2024-04-25T00:00:57.975969+0000","last_clean":"2024-04-25T00:00:57.975969+0000","last_became_active":"2024-04-24T23:34:53.357456+0000","last_became_peered":"2024-04-24T23:34:53.357456+0000","last_unstale":"2024-04-25T00:00:57.975969+0000","last_undegraded":"2024-04-25T00:00:57.975969+0000","last_fullsized":"2024-04-25T00:00:57.975969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5190","last_scrub_stamp":"2024-04-25T00:00:57.975892+0000","last_deep_scrub":"27'5190","last_deep_scrub_stamp":"2024-04-25T00:00:57.975892+0000","last_clean_scrub_stamp":"2024-04-25T00:00:57.975892+0000","objects_scrubbed":1,"log_size":5190,"log_dups_size":0,"ondisk_log_size":5190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:31:12.967557+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5191,"num_write_kb":40339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"27'119","reported_seq":205,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:00.011024+0000","last_change":"2024-04-25T00:01:00.011070+0000","last_active":"2024-04-25T00:01:00.011024+0000","last_peered":"2024-04-25T00:01:00.011024+0000","last_clean":"2024-04-25T00:01:00.011024+0000","last_became_active":"2024-04-24T23:34:49.778083+0000","last_became_peered":"2024-04-24T23:34:49.778083+0000","last_unstale":"2024-04-25T00:01:00.011024+0000","last_undegraded":"2024-04-25T00:01:00.011024+0000","last_fullsized":"2024-04-25T00:01:00.011024+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'119","last_scrub_stamp":"2024-04-25T00:01:00.010927+0000","last_deep_scrub":"27'119","last_deep_scrub_stamp":"2024-04-25T00:01:00.010927+0000","last_clean_scrub_stamp":"2024-04-25T00:01:00.010927+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:15.428926+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"27'8889","reported_seq":9415,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:10.091675+0000","last_change":"2024-04-25T00:01:10.091700+0000","last_active":"2024-04-25T00:01:10.091675+0000","last_peered":"2024-04-25T00:01:10.091675+0000","last_clean":"2024-04-25T00:01:10.091675+0000","last_became_active":"2024-04-24T23:34:58.729370+0000","last_became_peered":"2024-04-24T23:34:58.729370+0000","last_unstale":"2024-04-25T00:01:10.091675+0000","last_undegraded":"2024-04-25T00:01:10.091675+0000","last_fullsized":"2024-04-25T00:01:10.091675+0000","mapping_epoch":23,"log_start":"27'902","ondisk_log_start":"27'902","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8889","last_scrub_stamp":"2024-04-25T00:01:10.091591+0000","last_deep_scrub":"27'8889","last_deep_scrub_stamp":"2024-04-25T00:01:10.091591+0000","last_clean_scrub_stamp":"2024-04-25T00:01:10.091591+0000","objects_scrubbed":0,"log_size":7987,"log_dups_size":0,"ondisk_log_size":7987,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:00.756229+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":487,"num_read_kb":123698,"num_write":6268,"num_write_kb":912685,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"27'2533","reported_seq":2576,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:14.828168+0000","last_change":"2024-04-25T00:01:14.828194+0000","last_active":"2024-04-25T00:01:14.828168+0000","last_peered":"2024-04-25T00:01:14.828168+0000","last_clean":"2024-04-25T00:01:14.828168+0000","last_became_active":"2024-04-24T23:34:54.383589+0000","last_became_peered":"2024-04-24T23:34:54.383589+0000","last_unstale":"2024-04-25T00:01:14.828168+0000","last_undegraded":"2024-04-25T00:01:14.828168+0000","last_fullsized":"2024-04-25T00:01:14.828168+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2533","last_scrub_stamp":"2024-04-25T00:01:14.828096+0000","last_deep_scrub":"27'2533","last_deep_scrub_stamp":"2024-04-25T00:01:14.828096+0000","last_clean_scrub_stamp":"2024-04-25T00:01:14.828096+0000","objects_scrubbed":0,"log_size":2533,"log_dups_size":0,"ondisk_log_size":2533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:02:44.566867+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"27'8233","reported_seq":8279,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:56.027938+0000","last_change":"2024-04-25T00:00:56.027970+0000","last_active":"2024-04-25T00:00:56.027938+0000","last_peered":"2024-04-25T00:00:56.027938+0000","last_clean":"2024-04-25T00:00:56.027938+0000","last_became_active":"2024-04-24T23:34:53.353804+0000","last_became_peered":"2024-04-24T23:34:53.353804+0000","last_unstale":"2024-04-25T00:00:56.027938+0000","last_undegraded":"2024-04-25T00:00:56.027938+0000","last_fullsized":"2024-04-25T00:00:56.027938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8233","last_scrub_stamp":"2024-04-25T00:00:56.027870+0000","last_deep_scrub":"27'8233","last_deep_scrub_stamp":"2024-04-25T00:00:56.027870+0000","last_clean_scrub_stamp":"2024-04-25T00:00:56.027870+0000","objects_scrubbed":2,"log_size":8233,"log_dups_size":0,"ondisk_log_size":8233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:58:35.100912+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8225,"num_write_kb":58738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"27'9238","reported_seq":9435,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:09.117938+0000","last_change":"2024-04-25T00:01:09.117938+0000","last_active":"2024-04-25T00:01:09.117938+0000","last_peered":"2024-04-25T00:01:09.117938+0000","last_clean":"2024-04-25T00:01:09.117938+0000","last_became_active":"2024-04-24T23:34:58.727649+0000","last_became_peered":"2024-04-24T23:34:58.727649+0000","last_unstale":"2024-04-25T00:01:09.117938+0000","last_undegraded":"2024-04-25T00:01:09.117938+0000","last_fullsized":"2024-04-25T00:01:09.117938+0000","mapping_epoch":23,"log_start":"27'1312","ondisk_log_start":"27'1312","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9238","last_scrub_stamp":"2024-04-25T00:01:09.117822+0000","last_deep_scrub":"27'9238","last_deep_scrub_stamp":"2024-04-25T00:01:09.117822+0000","last_clean_scrub_stamp":"2024-04-25T00:01:09.117822+0000","objects_scrubbed":0,"log_size":7926,"log_dups_size":0,"ondisk_log_size":7926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:28.602762+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":158,"num_read_kb":40132,"num_write":6566,"num_write_kb":960960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"27'2561","reported_seq":2605,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:02.891955+0000","last_change":"2024-04-25T00:01:02.891995+0000","last_active":"2024-04-25T00:01:02.891955+0000","last_peered":"2024-04-25T00:01:02.891955+0000","last_clean":"2024-04-25T00:01:02.891955+0000","last_became_active":"2024-04-24T23:34:54.372553+0000","last_became_peered":"2024-04-24T23:34:54.372553+0000","last_unstale":"2024-04-25T00:01:02.891955+0000","last_undegraded":"2024-04-25T00:01:02.891955+0000","last_fullsized":"2024-04-25T00:01:02.891955+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2561","last_scrub_stamp":"2024-04-25T00:01:02.891883+0000","last_deep_scrub":"27'2561","last_deep_scrub_stamp":"2024-04-25T00:01:02.891883+0000","last_clean_scrub_stamp":"2024-04-25T00:01:02.891883+0000","objects_scrubbed":0,"log_size":2561,"log_dups_size":0,"ondisk_log_size":2561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:42:18.850137+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"27'8396","reported_seq":8442,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:57.135015+0000","last_change":"2024-04-25T00:00:57.135047+0000","last_active":"2024-04-25T00:00:57.135015+0000","last_peered":"2024-04-25T00:00:57.135015+0000","last_clean":"2024-04-25T00:00:57.135015+0000","last_became_active":"2024-04-24T23:34:53.353186+0000","last_became_peered":"2024-04-24T23:34:53.353186+0000","last_unstale":"2024-04-25T00:00:57.135015+0000","last_undegraded":"2024-04-25T00:00:57.135015+0000","last_fullsized":"2024-04-25T00:00:57.135015+0000","mapping_epoch":18,"log_start":"27'1000","ondisk_log_start":"27'1000","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8396","last_scrub_stamp":"2024-04-25T00:00:57.134940+0000","last_deep_scrub":"27'8396","last_deep_scrub_stamp":"2024-04-25T00:00:57.134940+0000","last_clean_scrub_stamp":"2024-04-25T00:00:57.134940+0000","objects_scrubbed":3,"log_size":7396,"log_dups_size":0,"ondisk_log_size":7396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:12.552939+0000","scrub_duration":38,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388678,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8393,"num_write_kb":58880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"27'9582","reported_seq":9946,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:08.003411+0000","last_change":"2024-04-25T00:01:08.003455+0000","last_active":"2024-04-25T00:01:08.003411+0000","last_peered":"2024-04-25T00:01:08.003411+0000","last_clean":"2024-04-25T00:01:08.003411+0000","last_became_active":"2024-04-24T23:34:58.400549+0000","last_became_peered":"2024-04-24T23:34:58.400549+0000","last_unstale":"2024-04-25T00:01:08.003411+0000","last_undegraded":"2024-04-25T00:01:08.003411+0000","last_fullsized":"2024-04-25T00:01:08.003411+0000","mapping_epoch":23,"log_start":"27'2356","ondisk_log_start":"27'2356","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9582","last_scrub_stamp":"2024-04-25T00:01:08.003283+0000","last_deep_scrub":"27'9582","last_deep_scrub_stamp":"2024-04-25T00:01:08.003283+0000","last_clean_scrub_stamp":"2024-04-25T00:01:08.003283+0000","objects_scrubbed":0,"log_size":7226,"log_dups_size":0,"ondisk_log_size":7226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:13:25.549872+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":326,"num_read_kb":82804,"num_write":6847,"num_write_kb":886980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"27'2433","reported_seq":2476,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:09.873327+0000","last_change":"2024-04-25T00:01:09.873348+0000","last_active":"2024-04-25T00:01:09.873327+0000","last_peered":"2024-04-25T00:01:09.873327+0000","last_clean":"2024-04-25T00:01:09.873327+0000","last_became_active":"2024-04-24T23:34:54.374963+0000","last_became_peered":"2024-04-24T23:34:54.374963+0000","last_unstale":"2024-04-25T00:01:09.873327+0000","last_undegraded":"2024-04-25T00:01:09.873327+0000","last_fullsized":"2024-04-25T00:01:09.873327+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2433","last_scrub_stamp":"2024-04-25T00:01:09.873258+0000","last_deep_scrub":"27'2433","last_deep_scrub_stamp":"2024-04-25T00:01:09.873258+0000","last_clean_scrub_stamp":"2024-04-25T00:01:09.873258+0000","objects_scrubbed":0,"log_size":2433,"log_dups_size":0,"ondisk_log_size":2433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:51.544421+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"27'8944","reported_seq":8990,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:59.112330+0000","last_change":"2024-04-25T00:00:59.112379+0000","last_active":"2024-04-25T00:00:59.112330+0000","last_peered":"2024-04-25T00:00:59.112330+0000","last_clean":"2024-04-25T00:00:59.112330+0000","last_became_active":"2024-04-24T23:34:53.360247+0000","last_became_peered":"2024-04-24T23:34:53.360247+0000","last_unstale":"2024-04-25T00:00:59.112330+0000","last_undegraded":"2024-04-25T00:00:59.112330+0000","last_fullsized":"2024-04-25T00:00:59.112330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8944","last_scrub_stamp":"2024-04-25T00:00:59.112263+0000","last_deep_scrub":"27'8944","last_deep_scrub_stamp":"2024-04-25T00:00:59.112263+0000","last_clean_scrub_stamp":"2024-04-25T00:00:59.112263+0000","objects_scrubbed":7,"log_size":8944,"log_dups_size":0,"ondisk_log_size":8944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:47:18.749387+0000","scrub_duration":101,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8935,"num_write_kb":79464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"27'9258","reported_seq":10451,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:11.889222+0000","last_change":"2024-04-25T00:01:11.889248+0000","last_active":"2024-04-25T00:01:11.889222+0000","last_peered":"2024-04-25T00:01:11.889222+0000","last_clean":"2024-04-25T00:01:11.889222+0000","last_became_active":"2024-04-24T23:34:58.729345+0000","last_became_peered":"2024-04-24T23:34:58.729345+0000","last_unstale":"2024-04-25T00:01:11.889222+0000","last_undegraded":"2024-04-25T00:01:11.889222+0000","last_fullsized":"2024-04-25T00:01:11.889222+0000","mapping_epoch":23,"log_start":"27'1527","ondisk_log_start":"27'1527","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9258","last_scrub_stamp":"2024-04-25T00:01:11.889135+0000","last_deep_scrub":"27'9258","last_deep_scrub_stamp":"2024-04-25T00:01:11.889135+0000","last_clean_scrub_stamp":"2024-04-25T00:01:11.889135+0000","objects_scrubbed":0,"log_size":7731,"log_dups_size":0,"ondisk_log_size":7731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:17:58.223321+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1154,"num_read_kb":293116,"num_write":6516,"num_write_kb":1022776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"27'2529","reported_seq":2572,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:11.803877+0000","last_change":"2024-04-25T00:01:11.803899+0000","last_active":"2024-04-25T00:01:11.803877+0000","last_peered":"2024-04-25T00:01:11.803877+0000","last_clean":"2024-04-25T00:01:11.803877+0000","last_became_active":"2024-04-24T23:34:54.377270+0000","last_became_peered":"2024-04-24T23:34:54.377270+0000","last_unstale":"2024-04-25T00:01:11.803877+0000","last_undegraded":"2024-04-25T00:01:11.803877+0000","last_fullsized":"2024-04-25T00:01:11.803877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2529","last_scrub_stamp":"2024-04-25T00:01:11.803800+0000","last_deep_scrub":"27'2529","last_deep_scrub_stamp":"2024-04-25T00:01:11.803800+0000","last_clean_scrub_stamp":"2024-04-25T00:01:11.803800+0000","objects_scrubbed":0,"log_size":2529,"log_dups_size":0,"ondisk_log_size":2529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:15:16.207629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"27'4551","reported_seq":4597,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:56.705155+0000","last_change":"2024-04-25T00:00:56.705183+0000","last_active":"2024-04-25T00:00:56.705155+0000","last_peered":"2024-04-25T00:00:56.705155+0000","last_clean":"2024-04-25T00:00:56.705155+0000","last_became_active":"2024-04-24T23:34:53.352669+0000","last_became_peered":"2024-04-24T23:34:53.352669+0000","last_unstale":"2024-04-25T00:00:56.705155+0000","last_undegraded":"2024-04-25T00:00:56.705155+0000","last_fullsized":"2024-04-25T00:00:56.705155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4551","last_scrub_stamp":"2024-04-25T00:00:56.705093+0000","last_deep_scrub":"27'4551","last_deep_scrub_stamp":"2024-04-25T00:00:56.705093+0000","last_clean_scrub_stamp":"2024-04-25T00:00:56.705093+0000","objects_scrubbed":2,"log_size":4551,"log_dups_size":0,"ondisk_log_size":4551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:14:54.766742+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4539,"num_write_kb":43853,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"27'9062","reported_seq":9852,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:05.737990+0000","last_change":"2024-04-25T00:01:05.737990+0000","last_active":"2024-04-25T00:01:05.737990+0000","last_peered":"2024-04-25T00:01:05.737990+0000","last_clean":"2024-04-25T00:01:05.737990+0000","last_became_active":"2024-04-24T23:35:43.529375+0000","last_became_peered":"2024-04-24T23:35:43.529375+0000","last_unstale":"2024-04-25T00:01:05.737990+0000","last_undegraded":"2024-04-25T00:01:05.737990+0000","last_fullsized":"2024-04-25T00:01:05.737990+0000","mapping_epoch":26,"log_start":"27'1912","ondisk_log_start":"27'1912","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9062","last_scrub_stamp":"2024-04-25T00:01:05.737842+0000","last_deep_scrub":"27'9062","last_deep_scrub_stamp":"2024-04-25T00:01:05.737842+0000","last_clean_scrub_stamp":"2024-04-25T00:01:05.737842+0000","objects_scrubbed":0,"log_size":7150,"log_dups_size":0,"ondisk_log_size":7150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:06:36.752974+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":744,"num_read_kb":188976,"num_write":6372,"num_write_kb":873040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3,6],"acting":[4,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"27'2607","reported_seq":2650,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:10.824429+0000","last_change":"2024-04-25T00:01:10.824450+0000","last_active":"2024-04-25T00:01:10.824429+0000","last_peered":"2024-04-25T00:01:10.824429+0000","last_clean":"2024-04-25T00:01:10.824429+0000","last_became_active":"2024-04-24T23:34:54.376512+0000","last_became_peered":"2024-04-24T23:34:54.376512+0000","last_unstale":"2024-04-25T00:01:10.824429+0000","last_undegraded":"2024-04-25T00:01:10.824429+0000","last_fullsized":"2024-04-25T00:01:10.824429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2607","last_scrub_stamp":"2024-04-25T00:01:10.824364+0000","last_deep_scrub":"27'2607","last_deep_scrub_stamp":"2024-04-25T00:01:10.824364+0000","last_clean_scrub_stamp":"2024-04-25T00:01:10.824364+0000","objects_scrubbed":0,"log_size":2607,"log_dups_size":0,"ondisk_log_size":2607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:13:28.370952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"27'4002","reported_seq":4048,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:57.022182+0000","last_change":"2024-04-25T00:00:57.022228+0000","last_active":"2024-04-25T00:00:57.022182+0000","last_peered":"2024-04-25T00:00:57.022182+0000","last_clean":"2024-04-25T00:00:57.022182+0000","last_became_active":"2024-04-24T23:34:53.354988+0000","last_became_peered":"2024-04-24T23:34:53.354988+0000","last_unstale":"2024-04-25T00:00:57.022182+0000","last_undegraded":"2024-04-25T00:00:57.022182+0000","last_fullsized":"2024-04-25T00:00:57.022182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4002","last_scrub_stamp":"2024-04-25T00:00:57.022120+0000","last_deep_scrub":"27'4002","last_deep_scrub_stamp":"2024-04-25T00:00:57.022120+0000","last_clean_scrub_stamp":"2024-04-25T00:00:57.022120+0000","objects_scrubbed":6,"log_size":4002,"log_dups_size":0,"ondisk_log_size":4002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:19:11.793888+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4001,"num_write_kb":39473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"27'9856","reported_seq":10238,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:07.042265+0000","last_change":"2024-04-25T00:01:07.042265+0000","last_active":"2024-04-25T00:01:07.042265+0000","last_peered":"2024-04-25T00:01:07.042265+0000","last_clean":"2024-04-25T00:01:07.042265+0000","last_became_active":"2024-04-24T23:34:58.398457+0000","last_became_peered":"2024-04-24T23:34:58.398457+0000","last_unstale":"2024-04-25T00:01:07.042265+0000","last_undegraded":"2024-04-25T00:01:07.042265+0000","last_fullsized":"2024-04-25T00:01:07.042265+0000","mapping_epoch":23,"log_start":"27'2619","ondisk_log_start":"27'2619","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9856","last_scrub_stamp":"2024-04-25T00:01:07.042125+0000","last_deep_scrub":"27'9856","last_deep_scrub_stamp":"2024-04-25T00:01:07.042125+0000","last_clean_scrub_stamp":"2024-04-25T00:01:07.042125+0000","objects_scrubbed":0,"log_size":7237,"log_dups_size":0,"ondisk_log_size":7237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:48.713288+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":344,"num_read_kb":87376,"num_write":7128,"num_write_kb":1170154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"27'2472","reported_seq":2516,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:02.176088+0000","last_change":"2024-04-25T00:01:02.176113+0000","last_active":"2024-04-25T00:01:02.176088+0000","last_peered":"2024-04-25T00:01:02.176088+0000","last_clean":"2024-04-25T00:01:02.176088+0000","last_became_active":"2024-04-24T23:34:54.371703+0000","last_became_peered":"2024-04-24T23:34:54.371703+0000","last_unstale":"2024-04-25T00:01:02.176088+0000","last_undegraded":"2024-04-25T00:01:02.176088+0000","last_fullsized":"2024-04-25T00:01:02.176088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2472","last_scrub_stamp":"2024-04-25T00:01:02.176002+0000","last_deep_scrub":"27'2472","last_deep_scrub_stamp":"2024-04-25T00:01:02.176002+0000","last_clean_scrub_stamp":"2024-04-25T00:01:02.176002+0000","objects_scrubbed":0,"log_size":2472,"log_dups_size":0,"ondisk_log_size":2472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:18:51.065456+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"27'4715","reported_seq":4761,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:59.279111+0000","last_change":"2024-04-25T00:00:59.279163+0000","last_active":"2024-04-25T00:00:59.279111+0000","last_peered":"2024-04-25T00:00:59.279111+0000","last_clean":"2024-04-25T00:00:59.279111+0000","last_became_active":"2024-04-24T23:34:53.363515+0000","last_became_peered":"2024-04-24T23:34:53.363515+0000","last_unstale":"2024-04-25T00:00:59.279111+0000","last_undegraded":"2024-04-25T00:00:59.279111+0000","last_fullsized":"2024-04-25T00:00:59.279111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4715","last_scrub_stamp":"2024-04-25T00:00:59.279022+0000","last_deep_scrub":"27'4715","last_deep_scrub_stamp":"2024-04-25T00:00:59.279022+0000","last_clean_scrub_stamp":"2024-04-25T00:00:59.279022+0000","objects_scrubbed":3,"log_size":4715,"log_dups_size":0,"ondisk_log_size":4715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:38:12.523634+0000","scrub_duration":55,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4705,"num_write_kb":43976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"27'9394","reported_seq":9798,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:11.316884+0000","last_change":"2024-04-25T00:01:11.316909+0000","last_active":"2024-04-25T00:01:11.316884+0000","last_peered":"2024-04-25T00:01:11.316884+0000","last_clean":"2024-04-25T00:01:11.316884+0000","last_became_active":"2024-04-24T23:34:58.727425+0000","last_became_peered":"2024-04-24T23:34:58.727425+0000","last_unstale":"2024-04-25T00:01:11.316884+0000","last_undegraded":"2024-04-25T00:01:11.316884+0000","last_fullsized":"2024-04-25T00:01:11.316884+0000","mapping_epoch":23,"log_start":"27'3135","ondisk_log_start":"27'3135","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9394","last_scrub_stamp":"2024-04-25T00:01:11.316797+0000","last_deep_scrub":"27'9394","last_deep_scrub_stamp":"2024-04-25T00:01:11.316797+0000","last_clean_scrub_stamp":"2024-04-25T00:01:11.316797+0000","objects_scrubbed":0,"log_size":6259,"log_dups_size":0,"ondisk_log_size":6259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:27.090436+0000","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":367,"num_read_kb":93218,"num_write":6677,"num_write_kb":924431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"27'2582","reported_seq":2625,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:12.849850+0000","last_change":"2024-04-25T00:01:12.849871+0000","last_active":"2024-04-25T00:01:12.849850+0000","last_peered":"2024-04-25T00:01:12.849850+0000","last_clean":"2024-04-25T00:01:12.849850+0000","last_became_active":"2024-04-24T23:34:54.379016+0000","last_became_peered":"2024-04-24T23:34:54.379016+0000","last_unstale":"2024-04-25T00:01:12.849850+0000","last_undegraded":"2024-04-25T00:01:12.849850+0000","last_fullsized":"2024-04-25T00:01:12.849850+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2582","last_scrub_stamp":"2024-04-25T00:01:12.849779+0000","last_deep_scrub":"27'2582","last_deep_scrub_stamp":"2024-04-25T00:01:12.849779+0000","last_clean_scrub_stamp":"2024-04-25T00:01:12.849779+0000","objects_scrubbed":0,"log_size":2582,"log_dups_size":0,"ondisk_log_size":2582,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:24.897738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"27'2768","reported_seq":2814,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:59.985281+0000","last_change":"2024-04-25T00:00:59.985307+0000","last_active":"2024-04-25T00:00:59.985281+0000","last_peered":"2024-04-25T00:00:59.985281+0000","last_clean":"2024-04-25T00:00:59.985281+0000","last_became_active":"2024-04-24T23:34:53.359581+0000","last_became_peered":"2024-04-24T23:34:53.359581+0000","last_unstale":"2024-04-25T00:00:59.985281+0000","last_undegraded":"2024-04-25T00:00:59.985281+0000","last_fullsized":"2024-04-25T00:00:59.985281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2768","last_scrub_stamp":"2024-04-25T00:00:59.985218+0000","last_deep_scrub":"27'2768","last_deep_scrub_stamp":"2024-04-25T00:00:59.985218+0000","last_clean_scrub_stamp":"2024-04-25T00:00:59.985218+0000","objects_scrubbed":2,"log_size":2768,"log_dups_size":0,"ondisk_log_size":2768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:08:07.821723+0000","scrub_duration":29,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2773,"num_write_kb":30370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"27'9089","reported_seq":9568,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:12.295066+0000","last_change":"2024-04-25T00:01:12.295091+0000","last_active":"2024-04-25T00:01:12.295066+0000","last_peered":"2024-04-25T00:01:12.295066+0000","last_clean":"2024-04-25T00:01:12.295066+0000","last_became_active":"2024-04-24T23:34:58.727686+0000","last_became_peered":"2024-04-24T23:34:58.727686+0000","last_unstale":"2024-04-25T00:01:12.295066+0000","last_undegraded":"2024-04-25T00:01:12.295066+0000","last_fullsized":"2024-04-25T00:01:12.295066+0000","mapping_epoch":23,"log_start":"27'2830","ondisk_log_start":"27'2830","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9089","last_scrub_stamp":"2024-04-25T00:01:12.294980+0000","last_deep_scrub":"27'9089","last_deep_scrub_stamp":"2024-04-25T00:01:12.294980+0000","last_clean_scrub_stamp":"2024-04-25T00:01:12.294980+0000","objects_scrubbed":0,"log_size":6259,"log_dups_size":0,"ondisk_log_size":6259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:30.991247+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":441,"num_read_kb":112014,"num_write":6536,"num_write_kb":907183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"27'2539","reported_seq":2583,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:02.980632+0000","last_change":"2024-04-25T00:01:02.980653+0000","last_active":"2024-04-25T00:01:02.980632+0000","last_peered":"2024-04-25T00:01:02.980632+0000","last_clean":"2024-04-25T00:01:02.980632+0000","last_became_active":"2024-04-24T23:34:54.371720+0000","last_became_peered":"2024-04-24T23:34:54.371720+0000","last_unstale":"2024-04-25T00:01:02.980632+0000","last_undegraded":"2024-04-25T00:01:02.980632+0000","last_fullsized":"2024-04-25T00:01:02.980632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2539","last_scrub_stamp":"2024-04-25T00:01:02.980564+0000","last_deep_scrub":"27'2539","last_deep_scrub_stamp":"2024-04-25T00:01:02.980564+0000","last_clean_scrub_stamp":"2024-04-25T00:01:02.980564+0000","objects_scrubbed":0,"log_size":2539,"log_dups_size":0,"ondisk_log_size":2539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:42:18.938818+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"27'6538","reported_seq":6583,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:56.140250+0000","last_change":"2024-04-25T00:00:56.140323+0000","last_active":"2024-04-25T00:00:56.140250+0000","last_peered":"2024-04-25T00:00:56.140250+0000","last_clean":"2024-04-25T00:00:56.140250+0000","last_became_active":"2024-04-24T23:34:53.353452+0000","last_became_peered":"2024-04-24T23:34:53.353452+0000","last_unstale":"2024-04-25T00:00:56.140250+0000","last_undegraded":"2024-04-25T00:00:56.140250+0000","last_fullsized":"2024-04-25T00:00:56.140250+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6538","last_scrub_stamp":"2024-04-25T00:00:56.140125+0000","last_deep_scrub":"27'6538","last_deep_scrub_stamp":"2024-04-25T00:00:56.140125+0000","last_clean_scrub_stamp":"2024-04-25T00:00:56.140125+0000","objects_scrubbed":7,"log_size":6538,"log_dups_size":0,"ondisk_log_size":6538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:02:42.997386+0000","scrub_duration":75,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971554,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6522,"num_write_kb":57515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"27'9615","reported_seq":10074,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:10.881571+0000","last_change":"2024-04-25T00:01:10.881571+0000","last_active":"2024-04-25T00:01:10.881571+0000","last_peered":"2024-04-25T00:01:10.881571+0000","last_clean":"2024-04-25T00:01:10.881571+0000","last_became_active":"2024-04-24T23:34:58.727921+0000","last_became_peered":"2024-04-24T23:34:58.727921+0000","last_unstale":"2024-04-25T00:01:10.881571+0000","last_undegraded":"2024-04-25T00:01:10.881571+0000","last_fullsized":"2024-04-25T00:01:10.881571+0000","mapping_epoch":23,"log_start":"27'1833","ondisk_log_start":"27'1833","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9615","last_scrub_stamp":"2024-04-25T00:01:10.881459+0000","last_deep_scrub":"27'9615","last_deep_scrub_stamp":"2024-04-25T00:01:10.881459+0000","last_clean_scrub_stamp":"2024-04-25T00:01:10.881459+0000","objects_scrubbed":0,"log_size":7782,"log_dups_size":0,"ondisk_log_size":7782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:04:35.139000+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":421,"num_read_kb":106934,"num_write":7001,"num_write_kb":1058941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"27'2478","reported_seq":2521,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:59.038416+0000","last_change":"2024-04-25T00:00:59.038416+0000","last_active":"2024-04-25T00:00:59.038416+0000","last_peered":"2024-04-25T00:00:59.038416+0000","last_clean":"2024-04-25T00:00:59.038416+0000","last_became_active":"2024-04-24T23:34:54.367903+0000","last_became_peered":"2024-04-24T23:34:54.367903+0000","last_unstale":"2024-04-25T00:00:59.038416+0000","last_undegraded":"2024-04-25T00:00:59.038416+0000","last_fullsized":"2024-04-25T00:00:59.038416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2478","last_scrub_stamp":"2024-04-25T00:00:59.038288+0000","last_deep_scrub":"27'2478","last_deep_scrub_stamp":"2024-04-25T00:00:59.038288+0000","last_clean_scrub_stamp":"2024-04-25T00:00:59.038288+0000","objects_scrubbed":0,"log_size":2478,"log_dups_size":0,"ondisk_log_size":2478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:10:54.490651+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'9572","reported_seq":10219,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:58.284915+0000","last_change":"2024-04-25T00:00:58.284949+0000","last_active":"2024-04-25T00:00:58.284915+0000","last_peered":"2024-04-25T00:00:58.284915+0000","last_clean":"2024-04-25T00:00:58.284915+0000","last_became_active":"2024-04-24T23:34:53.362230+0000","last_became_peered":"2024-04-24T23:34:53.362230+0000","last_unstale":"2024-04-25T00:00:58.284915+0000","last_undegraded":"2024-04-25T00:00:58.284915+0000","last_fullsized":"2024-04-25T00:00:58.284915+0000","mapping_epoch":18,"log_start":"27'1600","ondisk_log_start":"27'1600","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9572","last_scrub_stamp":"2024-04-25T00:00:58.284856+0000","last_deep_scrub":"27'9572","last_deep_scrub_stamp":"2024-04-25T00:00:58.284856+0000","last_clean_scrub_stamp":"2024-04-25T00:00:58.284856+0000","objects_scrubbed":5,"log_size":7972,"log_dups_size":0,"ondisk_log_size":7972,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:02:45.142117+0000","scrub_duration":36,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389280,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":602,"num_read_kb":4494,"num_write":9568,"num_write_kb":76044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"27'9012","reported_seq":9427,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:07.123038+0000","last_change":"2024-04-25T00:01:07.123063+0000","last_active":"2024-04-25T00:01:07.123038+0000","last_peered":"2024-04-25T00:01:07.123038+0000","last_clean":"2024-04-25T00:01:07.123038+0000","last_became_active":"2024-04-24T23:34:58.400429+0000","last_became_peered":"2024-04-24T23:34:58.400429+0000","last_unstale":"2024-04-25T00:01:07.123038+0000","last_undegraded":"2024-04-25T00:01:07.123038+0000","last_fullsized":"2024-04-25T00:01:07.123038+0000","mapping_epoch":23,"log_start":"27'1625","ondisk_log_start":"27'1625","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9012","last_scrub_stamp":"2024-04-25T00:01:07.122949+0000","last_deep_scrub":"27'9012","last_deep_scrub_stamp":"2024-04-25T00:01:07.122949+0000","last_clean_scrub_stamp":"2024-04-25T00:01:07.122949+0000","objects_scrubbed":0,"log_size":7387,"log_dups_size":0,"ondisk_log_size":7387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:57.787587+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":377,"num_read_kb":95758,"num_write":6342,"num_write_kb":955265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"27'2558","reported_seq":2602,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:00.661276+0000","last_change":"2024-04-25T00:01:00.661297+0000","last_active":"2024-04-25T00:01:00.661276+0000","last_peered":"2024-04-25T00:01:00.661276+0000","last_clean":"2024-04-25T00:01:00.661276+0000","last_became_active":"2024-04-24T23:34:54.373347+0000","last_became_peered":"2024-04-24T23:34:54.373347+0000","last_unstale":"2024-04-25T00:01:00.661276+0000","last_undegraded":"2024-04-25T00:01:00.661276+0000","last_fullsized":"2024-04-25T00:01:00.661276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2558","last_scrub_stamp":"2024-04-25T00:01:00.661203+0000","last_deep_scrub":"27'2558","last_deep_scrub_stamp":"2024-04-25T00:01:00.661203+0000","last_clean_scrub_stamp":"2024-04-25T00:01:00.661203+0000","objects_scrubbed":0,"log_size":2558,"log_dups_size":0,"ondisk_log_size":2558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:59:35.495255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"27'5926","reported_seq":5972,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:58.078892+0000","last_change":"2024-04-25T00:00:58.078892+0000","last_active":"2024-04-25T00:00:58.078892+0000","last_peered":"2024-04-25T00:00:58.078892+0000","last_clean":"2024-04-25T00:00:58.078892+0000","last_became_active":"2024-04-24T23:34:53.354394+0000","last_became_peered":"2024-04-24T23:34:53.354394+0000","last_unstale":"2024-04-25T00:00:58.078892+0000","last_undegraded":"2024-04-25T00:00:58.078892+0000","last_fullsized":"2024-04-25T00:00:58.078892+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5926","last_scrub_stamp":"2024-04-25T00:00:58.078802+0000","last_deep_scrub":"27'5926","last_deep_scrub_stamp":"2024-04-25T00:00:58.078802+0000","last_clean_scrub_stamp":"2024-04-25T00:00:58.078802+0000","objects_scrubbed":2,"log_size":5926,"log_dups_size":0,"ondisk_log_size":5926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:31:13.070467+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5926,"num_write_kb":41236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"27'9157","reported_seq":9573,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:03.711930+0000","last_change":"2024-04-25T00:01:03.711976+0000","last_active":"2024-04-25T00:01:03.711930+0000","last_peered":"2024-04-25T00:01:03.711930+0000","last_clean":"2024-04-25T00:01:03.711930+0000","last_became_active":"2024-04-24T23:34:58.397353+0000","last_became_peered":"2024-04-24T23:34:58.397353+0000","last_unstale":"2024-04-25T00:01:03.711930+0000","last_undegraded":"2024-04-25T00:01:03.711930+0000","last_fullsized":"2024-04-25T00:01:03.711930+0000","mapping_epoch":23,"log_start":"27'2001","ondisk_log_start":"27'2001","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9157","last_scrub_stamp":"2024-04-25T00:01:03.711783+0000","last_deep_scrub":"27'9157","last_deep_scrub_stamp":"2024-04-25T00:01:03.711783+0000","last_clean_scrub_stamp":"2024-04-25T00:01:03.711783+0000","objects_scrubbed":0,"log_size":7156,"log_dups_size":0,"ondisk_log_size":7156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:45:03.473492+0000","scrub_duration":29,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":378,"num_read_kb":96012,"num_write":6474,"num_write_kb":975733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"27'2584","reported_seq":2628,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:05.001044+0000","last_change":"2024-04-25T00:01:05.001044+0000","last_active":"2024-04-25T00:01:05.001044+0000","last_peered":"2024-04-25T00:01:05.001044+0000","last_clean":"2024-04-25T00:01:05.001044+0000","last_became_active":"2024-04-24T23:34:54.383082+0000","last_became_peered":"2024-04-24T23:34:54.383082+0000","last_unstale":"2024-04-25T00:01:05.001044+0000","last_undegraded":"2024-04-25T00:01:05.001044+0000","last_fullsized":"2024-04-25T00:01:05.001044+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2584","last_scrub_stamp":"2024-04-25T00:01:05.000905+0000","last_deep_scrub":"27'2584","last_deep_scrub_stamp":"2024-04-25T00:01:05.000905+0000","last_clean_scrub_stamp":"2024-04-25T00:01:05.000905+0000","objects_scrubbed":0,"log_size":2584,"log_dups_size":0,"ondisk_log_size":2584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:31.816773+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"27'7103","reported_seq":7149,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:00.991975+0000","last_change":"2024-04-25T00:01:00.991975+0000","last_active":"2024-04-25T00:01:00.991975+0000","last_peered":"2024-04-25T00:01:00.991975+0000","last_clean":"2024-04-25T00:01:00.991975+0000","last_became_active":"2024-04-24T23:34:53.364259+0000","last_became_peered":"2024-04-24T23:34:53.364259+0000","last_unstale":"2024-04-25T00:01:00.991975+0000","last_undegraded":"2024-04-25T00:01:00.991975+0000","last_fullsized":"2024-04-25T00:01:00.991975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7103","last_scrub_stamp":"2024-04-25T00:01:00.991796+0000","last_deep_scrub":"27'7103","last_deep_scrub_stamp":"2024-04-25T00:01:00.991796+0000","last_clean_scrub_stamp":"2024-04-25T00:01:00.991796+0000","objects_scrubbed":2,"log_size":7103,"log_dups_size":0,"ondisk_log_size":7103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:45:00.753505+0000","scrub_duration":45,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":7102,"num_write_kb":53865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"27'9038","reported_seq":9145,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:09.256807+0000","last_change":"2024-04-25T00:01:09.256870+0000","last_active":"2024-04-25T00:01:09.256807+0000","last_peered":"2024-04-25T00:01:09.256807+0000","last_clean":"2024-04-25T00:01:09.256807+0000","last_became_active":"2024-04-24T23:34:58.405032+0000","last_became_peered":"2024-04-24T23:34:58.405032+0000","last_unstale":"2024-04-25T00:01:09.256807+0000","last_undegraded":"2024-04-25T00:01:09.256807+0000","last_fullsized":"2024-04-25T00:01:09.256807+0000","mapping_epoch":23,"log_start":"27'2712","ondisk_log_start":"27'2712","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9038","last_scrub_stamp":"2024-04-25T00:01:09.256679+0000","last_deep_scrub":"27'9038","last_deep_scrub_stamp":"2024-04-25T00:01:09.256679+0000","last_clean_scrub_stamp":"2024-04-25T00:01:09.256679+0000","objects_scrubbed":0,"log_size":6326,"log_dups_size":0,"ondisk_log_size":6326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:56.779919+0000","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":17526,"num_write":6404,"num_write_kb":965810,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"27'2558","reported_seq":2602,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:08.887064+0000","last_change":"2024-04-25T00:01:08.887064+0000","last_active":"2024-04-25T00:01:08.887064+0000","last_peered":"2024-04-25T00:01:08.887064+0000","last_clean":"2024-04-25T00:01:08.887064+0000","last_became_active":"2024-04-24T23:34:54.383354+0000","last_became_peered":"2024-04-24T23:34:54.383354+0000","last_unstale":"2024-04-25T00:01:08.887064+0000","last_undegraded":"2024-04-25T00:01:08.887064+0000","last_fullsized":"2024-04-25T00:01:08.887064+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2558","last_scrub_stamp":"2024-04-25T00:01:08.886937+0000","last_deep_scrub":"27'2558","last_deep_scrub_stamp":"2024-04-25T00:01:08.886937+0000","last_clean_scrub_stamp":"2024-04-25T00:01:08.886937+0000","objects_scrubbed":0,"log_size":2558,"log_dups_size":0,"ondisk_log_size":2558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:15:13.290766+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"27'6418","reported_seq":7310,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:57.253393+0000","last_change":"2024-04-25T00:00:57.253393+0000","last_active":"2024-04-25T00:00:57.253393+0000","last_peered":"2024-04-25T00:00:57.253393+0000","last_clean":"2024-04-25T00:00:57.253393+0000","last_became_active":"2024-04-24T23:34:53.362103+0000","last_became_peered":"2024-04-24T23:34:53.362103+0000","last_unstale":"2024-04-25T00:00:57.253393+0000","last_undegraded":"2024-04-25T00:00:57.253393+0000","last_fullsized":"2024-04-25T00:00:57.253393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6418","last_scrub_stamp":"2024-04-25T00:00:57.253246+0000","last_deep_scrub":"27'6418","last_deep_scrub_stamp":"2024-04-25T00:00:57.253246+0000","last_clean_scrub_stamp":"2024-04-25T00:00:57.253246+0000","objects_scrubbed":0,"log_size":6418,"log_dups_size":0,"ondisk_log_size":6418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:49:16.211875+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":847,"num_read_kb":4657,"num_write":6417,"num_write_kb":49337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"27'9416","reported_seq":9713,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:14.351499+0000","last_change":"2024-04-25T00:01:14.351523+0000","last_active":"2024-04-25T00:01:14.351499+0000","last_peered":"2024-04-25T00:01:14.351499+0000","last_clean":"2024-04-25T00:01:14.351499+0000","last_became_active":"2024-04-24T23:34:58.729217+0000","last_became_peered":"2024-04-24T23:34:58.729217+0000","last_unstale":"2024-04-25T00:01:14.351499+0000","last_undegraded":"2024-04-25T00:01:14.351499+0000","last_fullsized":"2024-04-25T00:01:14.351499+0000","mapping_epoch":23,"log_start":"27'3115","ondisk_log_start":"27'3115","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9416","last_scrub_stamp":"2024-04-25T00:01:14.351384+0000","last_deep_scrub":"27'9416","last_deep_scrub_stamp":"2024-04-25T00:01:14.351384+0000","last_clean_scrub_stamp":"2024-04-25T00:01:14.351384+0000","objects_scrubbed":0,"log_size":6301,"log_dups_size":0,"ondisk_log_size":6301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:05:00.012542+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":259,"num_read_kb":65786,"num_write":6628,"num_write_kb":1056336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"27'9147","reported_seq":9286,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:10.982447+0000","last_change":"2024-04-25T00:01:10.982484+0000","last_active":"2024-04-25T00:01:10.982447+0000","last_peered":"2024-04-25T00:01:10.982447+0000","last_clean":"2024-04-25T00:01:10.982447+0000","last_became_active":"2024-04-24T23:34:58.727914+0000","last_became_peered":"2024-04-24T23:34:58.727914+0000","last_unstale":"2024-04-25T00:01:10.982447+0000","last_undegraded":"2024-04-25T00:01:10.982447+0000","last_fullsized":"2024-04-25T00:01:10.982447+0000","mapping_epoch":23,"log_start":"27'1931","ondisk_log_start":"27'1931","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9147","last_scrub_stamp":"2024-04-25T00:01:10.982317+0000","last_deep_scrub":"27'9147","last_deep_scrub_stamp":"2024-04-25T00:01:10.982317+0000","last_clean_scrub_stamp":"2024-04-25T00:01:10.982317+0000","objects_scrubbed":0,"log_size":7216,"log_dups_size":0,"ondisk_log_size":7216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:19:22.482471+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":99,"num_read_kb":25146,"num_write":6551,"num_write_kb":920441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"27'2499","reported_seq":2542,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.125218+0000","last_change":"2024-04-25T00:00:55.125262+0000","last_active":"2024-04-25T00:00:55.125218+0000","last_peered":"2024-04-25T00:00:55.125218+0000","last_clean":"2024-04-25T00:00:55.125218+0000","last_became_active":"2024-04-24T23:34:54.363724+0000","last_became_peered":"2024-04-24T23:34:54.363724+0000","last_unstale":"2024-04-25T00:00:55.125218+0000","last_undegraded":"2024-04-25T00:00:55.125218+0000","last_fullsized":"2024-04-25T00:00:55.125218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2499","last_scrub_stamp":"2024-04-25T00:00:55.125079+0000","last_deep_scrub":"27'2499","last_deep_scrub_stamp":"2024-04-25T00:00:55.125079+0000","last_clean_scrub_stamp":"2024-04-25T00:00:55.125079+0000","objects_scrubbed":0,"log_size":2499,"log_dups_size":0,"ondisk_log_size":2499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:41.142244+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"27'9688","reported_seq":9734,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:58.982186+0000","last_change":"2024-04-25T00:00:58.982186+0000","last_active":"2024-04-25T00:00:58.982186+0000","last_peered":"2024-04-25T00:00:58.982186+0000","last_clean":"2024-04-25T00:00:58.982186+0000","last_became_active":"2024-04-24T23:34:53.359707+0000","last_became_peered":"2024-04-24T23:34:53.359707+0000","last_unstale":"2024-04-25T00:00:58.982186+0000","last_undegraded":"2024-04-25T00:00:58.982186+0000","last_fullsized":"2024-04-25T00:00:58.982186+0000","mapping_epoch":18,"log_start":"27'1900","ondisk_log_start":"27'1900","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9688","last_scrub_stamp":"2024-04-25T00:00:58.982041+0000","last_deep_scrub":"27'9688","last_deep_scrub_stamp":"2024-04-25T00:00:58.982041+0000","last_clean_scrub_stamp":"2024-04-25T00:00:58.982041+0000","objects_scrubbed":1,"log_size":7788,"log_dups_size":0,"ondisk_log_size":7788,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:59:04.463251+0000","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9677,"num_write_kb":67972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"27'9609","reported_seq":10709,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:11.120915+0000","last_change":"2024-04-25T00:01:11.120940+0000","last_active":"2024-04-25T00:01:11.120915+0000","last_peered":"2024-04-25T00:01:11.120915+0000","last_clean":"2024-04-25T00:01:11.120915+0000","last_became_active":"2024-04-24T23:35:43.834066+0000","last_became_peered":"2024-04-24T23:35:43.834066+0000","last_unstale":"2024-04-25T00:01:11.120915+0000","last_undegraded":"2024-04-25T00:01:11.120915+0000","last_fullsized":"2024-04-25T00:01:11.120915+0000","mapping_epoch":26,"log_start":"27'1708","ondisk_log_start":"27'1708","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9609","last_scrub_stamp":"2024-04-25T00:01:11.120817+0000","last_deep_scrub":"27'9609","last_deep_scrub_stamp":"2024-04-25T00:01:11.120817+0000","last_clean_scrub_stamp":"2024-04-25T00:01:11.120817+0000","objects_scrubbed":0,"log_size":7901,"log_dups_size":0,"ondisk_log_size":7901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:47.802601+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1054,"num_read_kb":267716,"num_write":6908,"num_write_kb":899799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2,0],"acting":[6,5,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"27'2603","reported_seq":2647,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:57.268434+0000","last_change":"2024-04-25T00:00:57.268434+0000","last_active":"2024-04-25T00:00:57.268434+0000","last_peered":"2024-04-25T00:00:57.268434+0000","last_clean":"2024-04-25T00:00:57.268434+0000","last_became_active":"2024-04-24T23:34:54.369155+0000","last_became_peered":"2024-04-24T23:34:54.369155+0000","last_unstale":"2024-04-25T00:00:57.268434+0000","last_undegraded":"2024-04-25T00:00:57.268434+0000","last_fullsized":"2024-04-25T00:00:57.268434+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2603","last_scrub_stamp":"2024-04-25T00:00:57.268303+0000","last_deep_scrub":"27'2603","last_deep_scrub_stamp":"2024-04-25T00:00:57.268303+0000","last_clean_scrub_stamp":"2024-04-25T00:00:57.268303+0000","objects_scrubbed":0,"log_size":2603,"log_dups_size":0,"ondisk_log_size":2603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:36:57.590443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"27'4846","reported_seq":4892,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:02.278048+0000","last_change":"2024-04-25T00:01:02.278048+0000","last_active":"2024-04-25T00:01:02.278048+0000","last_peered":"2024-04-25T00:01:02.278048+0000","last_clean":"2024-04-25T00:01:02.278048+0000","last_became_active":"2024-04-24T23:34:53.365029+0000","last_became_peered":"2024-04-24T23:34:53.365029+0000","last_unstale":"2024-04-25T00:01:02.278048+0000","last_undegraded":"2024-04-25T00:01:02.278048+0000","last_fullsized":"2024-04-25T00:01:02.278048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4846","last_scrub_stamp":"2024-04-25T00:01:02.277955+0000","last_deep_scrub":"27'4846","last_deep_scrub_stamp":"2024-04-25T00:01:02.277955+0000","last_clean_scrub_stamp":"2024-04-25T00:01:02.277955+0000","objects_scrubbed":2,"log_size":4846,"log_dups_size":0,"ondisk_log_size":4846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:17:52.774906+0000","scrub_duration":30,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4849,"num_write_kb":44132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"27'9229","reported_seq":9274,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:00.249412+0000","last_change":"2024-04-25T00:01:00.249459+0000","last_active":"2024-04-25T00:01:00.249412+0000","last_peered":"2024-04-25T00:01:00.249412+0000","last_clean":"2024-04-25T00:01:00.249412+0000","last_became_active":"2024-04-24T23:34:53.364821+0000","last_became_peered":"2024-04-24T23:34:53.364821+0000","last_unstale":"2024-04-25T00:01:00.249412+0000","last_undegraded":"2024-04-25T00:01:00.249412+0000","last_fullsized":"2024-04-25T00:01:00.249412+0000","mapping_epoch":18,"log_start":"27'1300","ondisk_log_start":"27'1300","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9229","last_scrub_stamp":"2024-04-25T00:01:00.249298+0000","last_deep_scrub":"27'9229","last_deep_scrub_stamp":"2024-04-25T00:01:00.249298+0000","last_clean_scrub_stamp":"2024-04-25T00:01:00.249298+0000","objects_scrubbed":1,"log_size":7929,"log_dups_size":0,"ondisk_log_size":7929,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:15.667297+0000","scrub_duration":29,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9220,"num_write_kb":71599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"27'2536","reported_seq":2579,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.053278+0000","last_change":"2024-04-25T00:00:55.053300+0000","last_active":"2024-04-25T00:00:55.053278+0000","last_peered":"2024-04-25T00:00:55.053278+0000","last_clean":"2024-04-25T00:00:55.053278+0000","last_became_active":"2024-04-24T23:34:54.375689+0000","last_became_peered":"2024-04-24T23:34:54.375689+0000","last_unstale":"2024-04-25T00:00:55.053278+0000","last_undegraded":"2024-04-25T00:00:55.053278+0000","last_fullsized":"2024-04-25T00:00:55.053278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2536","last_scrub_stamp":"2024-04-25T00:00:55.053203+0000","last_deep_scrub":"27'2536","last_deep_scrub_stamp":"2024-04-25T00:00:55.053203+0000","last_clean_scrub_stamp":"2024-04-25T00:00:55.053203+0000","objects_scrubbed":0,"log_size":2536,"log_dups_size":0,"ondisk_log_size":2536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:49:14.011832+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"27'9729","reported_seq":9962,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:04.048851+0000","last_change":"2024-04-25T00:01:04.048875+0000","last_active":"2024-04-25T00:01:04.048851+0000","last_peered":"2024-04-25T00:01:04.048851+0000","last_clean":"2024-04-25T00:01:04.048851+0000","last_became_active":"2024-04-24T23:35:43.834061+0000","last_became_peered":"2024-04-24T23:35:43.834061+0000","last_unstale":"2024-04-25T00:01:04.048851+0000","last_undegraded":"2024-04-25T00:01:04.048851+0000","last_fullsized":"2024-04-25T00:01:04.048851+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9729","last_scrub_stamp":"2024-04-25T00:01:04.048769+0000","last_deep_scrub":"27'9729","last_deep_scrub_stamp":"2024-04-25T00:01:04.048769+0000","last_clean_scrub_stamp":"2024-04-25T00:01:04.048769+0000","objects_scrubbed":0,"log_size":9729,"log_dups_size":0,"ondisk_log_size":9729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:15:02.110418+0000","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":189,"num_read_kb":48006,"num_write":6998,"num_write_kb":973944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3,5],"acting":[2,0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"27'7147","reported_seq":7192,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:01.276351+0000","last_change":"2024-04-25T00:01:01.276399+0000","last_active":"2024-04-25T00:01:01.276351+0000","last_peered":"2024-04-25T00:01:01.276351+0000","last_clean":"2024-04-25T00:01:01.276351+0000","last_became_active":"2024-04-24T23:34:53.362548+0000","last_became_peered":"2024-04-24T23:34:53.362548+0000","last_unstale":"2024-04-25T00:01:01.276351+0000","last_undegraded":"2024-04-25T00:01:01.276351+0000","last_fullsized":"2024-04-25T00:01:01.276351+0000","mapping_epoch":18,"log_start":"27'800","ondisk_log_start":"27'800","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7147","last_scrub_stamp":"2024-04-25T00:01:01.276272+0000","last_deep_scrub":"27'7147","last_deep_scrub_stamp":"2024-04-25T00:01:01.276272+0000","last_clean_scrub_stamp":"2024-04-25T00:01:01.276272+0000","objects_scrubbed":0,"log_size":6347,"log_dups_size":0,"ondisk_log_size":6347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:49:20.637238+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7134,"num_write_kb":66117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"27'2573","reported_seq":2616,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:58.745714+0000","last_change":"2024-04-25T00:00:58.745751+0000","last_active":"2024-04-25T00:00:58.745714+0000","last_peered":"2024-04-25T00:00:58.745714+0000","last_clean":"2024-04-25T00:00:58.745714+0000","last_became_active":"2024-04-24T23:34:54.364814+0000","last_became_peered":"2024-04-24T23:34:54.364814+0000","last_unstale":"2024-04-25T00:00:58.745714+0000","last_undegraded":"2024-04-25T00:00:58.745714+0000","last_fullsized":"2024-04-25T00:00:58.745714+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2573","last_scrub_stamp":"2024-04-25T00:00:58.745637+0000","last_deep_scrub":"27'2573","last_deep_scrub_stamp":"2024-04-25T00:00:58.745637+0000","last_clean_scrub_stamp":"2024-04-25T00:00:58.745637+0000","objects_scrubbed":0,"log_size":2573,"log_dups_size":0,"ondisk_log_size":2573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:38:11.990248+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"27'9398","reported_seq":9613,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:10.294046+0000","last_change":"2024-04-25T00:01:10.294070+0000","last_active":"2024-04-25T00:01:10.294046+0000","last_peered":"2024-04-25T00:01:10.294046+0000","last_clean":"2024-04-25T00:01:10.294046+0000","last_became_active":"2024-04-24T23:34:58.406811+0000","last_became_peered":"2024-04-24T23:34:58.406811+0000","last_unstale":"2024-04-25T00:01:10.294046+0000","last_undegraded":"2024-04-25T00:01:10.294046+0000","last_fullsized":"2024-04-25T00:01:10.294046+0000","mapping_epoch":23,"log_start":"27'3112","ondisk_log_start":"27'3112","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9398","last_scrub_stamp":"2024-04-25T00:01:10.293962+0000","last_deep_scrub":"27'9398","last_deep_scrub_stamp":"2024-04-25T00:01:10.293962+0000","last_clean_scrub_stamp":"2024-04-25T00:01:10.293962+0000","objects_scrubbed":0,"log_size":6286,"log_dups_size":0,"ondisk_log_size":6286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:53.391877+0000","scrub_duration":41,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":178,"num_read_kb":45212,"num_write":6641,"num_write_kb":1045709,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"27'2471","reported_seq":2516,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:03.159347+0000","last_change":"2024-04-25T00:01:03.159369+0000","last_active":"2024-04-25T00:01:03.159347+0000","last_peered":"2024-04-25T00:01:03.159347+0000","last_clean":"2024-04-25T00:01:03.159347+0000","last_became_active":"2024-04-24T23:34:54.374844+0000","last_became_peered":"2024-04-24T23:34:54.374844+0000","last_unstale":"2024-04-25T00:01:03.159347+0000","last_undegraded":"2024-04-25T00:01:03.159347+0000","last_fullsized":"2024-04-25T00:01:03.159347+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2471","last_scrub_stamp":"2024-04-25T00:01:03.159211+0000","last_deep_scrub":"27'2471","last_deep_scrub_stamp":"2024-04-25T00:01:03.159211+0000","last_clean_scrub_stamp":"2024-04-25T00:01:03.159211+0000","objects_scrubbed":0,"log_size":2471,"log_dups_size":0,"ondisk_log_size":2471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:08:10.995716+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"27'2599","reported_seq":2644,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:00.987482+0000","last_change":"2024-04-25T00:01:00.987505+0000","last_active":"2024-04-25T00:01:00.987482+0000","last_peered":"2024-04-25T00:01:00.987482+0000","last_clean":"2024-04-25T00:01:00.987482+0000","last_became_active":"2024-04-24T23:34:54.383253+0000","last_became_peered":"2024-04-24T23:34:54.383253+0000","last_unstale":"2024-04-25T00:01:00.987482+0000","last_undegraded":"2024-04-25T00:01:00.987482+0000","last_fullsized":"2024-04-25T00:01:00.987482+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2599","last_scrub_stamp":"2024-04-25T00:01:00.987360+0000","last_deep_scrub":"27'2599","last_deep_scrub_stamp":"2024-04-25T00:01:00.987360+0000","last_clean_scrub_stamp":"2024-04-25T00:01:00.987360+0000","objects_scrubbed":0,"log_size":2599,"log_dups_size":0,"ondisk_log_size":2599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:44:48.841627+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"27'2500","reported_seq":2531,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:03.000975+0000","last_change":"2024-04-25T00:01:03.000997+0000","last_active":"2024-04-25T00:01:03.000975+0000","last_peered":"2024-04-25T00:01:03.000975+0000","last_clean":"2024-04-25T00:01:03.000975+0000","last_became_active":"2024-04-24T23:35:43.525929+0000","last_became_peered":"2024-04-24T23:35:43.525929+0000","last_unstale":"2024-04-25T00:01:03.000975+0000","last_undegraded":"2024-04-25T00:01:03.000975+0000","last_fullsized":"2024-04-25T00:01:03.000975+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2500","last_scrub_stamp":"2024-04-25T00:01:03.000914+0000","last_deep_scrub":"27'2500","last_deep_scrub_stamp":"2024-04-25T00:01:03.000914+0000","last_clean_scrub_stamp":"2024-04-25T00:01:03.000914+0000","objects_scrubbed":0,"log_size":2500,"log_dups_size":0,"ondisk_log_size":2500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:32:27.297166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"27'2582","reported_seq":2627,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:59.059928+0000","last_change":"2024-04-25T00:00:59.060129+0000","last_active":"2024-04-25T00:00:59.059928+0000","last_peered":"2024-04-25T00:00:59.059928+0000","last_clean":"2024-04-25T00:00:59.059928+0000","last_became_active":"2024-04-24T23:34:54.366005+0000","last_became_peered":"2024-04-24T23:34:54.366005+0000","last_unstale":"2024-04-25T00:00:59.059928+0000","last_undegraded":"2024-04-25T00:00:59.059928+0000","last_fullsized":"2024-04-25T00:00:59.059928+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2582","last_scrub_stamp":"2024-04-25T00:00:59.059802+0000","last_deep_scrub":"27'2582","last_deep_scrub_stamp":"2024-04-25T00:00:59.059802+0000","last_clean_scrub_stamp":"2024-04-25T00:00:59.059802+0000","objects_scrubbed":0,"log_size":2582,"log_dups_size":0,"ondisk_log_size":2582,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:59:33.893854+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"27'2585","reported_seq":2630,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:03.237339+0000","last_change":"2024-04-25T00:01:03.237381+0000","last_active":"2024-04-25T00:01:03.237339+0000","last_peered":"2024-04-25T00:01:03.237339+0000","last_clean":"2024-04-25T00:01:03.237339+0000","last_became_active":"2024-04-24T23:34:54.360266+0000","last_became_peered":"2024-04-24T23:34:54.360266+0000","last_unstale":"2024-04-25T00:01:03.237339+0000","last_undegraded":"2024-04-25T00:01:03.237339+0000","last_fullsized":"2024-04-25T00:01:03.237339+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2585","last_scrub_stamp":"2024-04-25T00:01:03.237248+0000","last_deep_scrub":"27'2585","last_deep_scrub_stamp":"2024-04-25T00:01:03.237248+0000","last_clean_scrub_stamp":"2024-04-25T00:01:03.237248+0000","objects_scrubbed":0,"log_size":2585,"log_dups_size":0,"ondisk_log_size":2585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:30:45.427659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"27'2453","reported_seq":2498,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:05.107266+0000","last_change":"2024-04-25T00:01:05.107305+0000","last_active":"2024-04-25T00:01:05.107266+0000","last_peered":"2024-04-25T00:01:05.107266+0000","last_clean":"2024-04-25T00:01:05.107266+0000","last_became_active":"2024-04-24T23:34:54.365493+0000","last_became_peered":"2024-04-24T23:34:54.365493+0000","last_unstale":"2024-04-25T00:01:05.107266+0000","last_undegraded":"2024-04-25T00:01:05.107266+0000","last_fullsized":"2024-04-25T00:01:05.107266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2453","last_scrub_stamp":"2024-04-25T00:01:05.107173+0000","last_deep_scrub":"27'2453","last_deep_scrub_stamp":"2024-04-25T00:01:05.107173+0000","last_clean_scrub_stamp":"2024-04-25T00:01:05.107173+0000","objects_scrubbed":0,"log_size":2453,"log_dups_size":0,"ondisk_log_size":2453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:12:53.127579+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"27'2590","reported_seq":2635,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:06.108968+0000","last_change":"2024-04-25T00:01:06.109006+0000","last_active":"2024-04-25T00:01:06.108968+0000","last_peered":"2024-04-25T00:01:06.108968+0000","last_clean":"2024-04-25T00:01:06.108968+0000","last_became_active":"2024-04-24T23:34:54.378568+0000","last_became_peered":"2024-04-24T23:34:54.378568+0000","last_unstale":"2024-04-25T00:01:06.108968+0000","last_undegraded":"2024-04-25T00:01:06.108968+0000","last_fullsized":"2024-04-25T00:01:06.108968+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2590","last_scrub_stamp":"2024-04-25T00:01:06.108860+0000","last_deep_scrub":"27'2590","last_deep_scrub_stamp":"2024-04-25T00:01:06.108860+0000","last_clean_scrub_stamp":"2024-04-25T00:01:06.108860+0000","objects_scrubbed":0,"log_size":2590,"log_dups_size":0,"ondisk_log_size":2590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:06:37.123991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"27'2557","reported_seq":2602,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:04.256443+0000","last_change":"2024-04-25T00:01:04.256479+0000","last_active":"2024-04-25T00:01:04.256443+0000","last_peered":"2024-04-25T00:01:04.256443+0000","last_clean":"2024-04-25T00:01:04.256443+0000","last_became_active":"2024-04-24T23:34:54.363439+0000","last_became_peered":"2024-04-24T23:34:54.363439+0000","last_unstale":"2024-04-25T00:01:04.256443+0000","last_undegraded":"2024-04-25T00:01:04.256443+0000","last_fullsized":"2024-04-25T00:01:04.256443+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2557","last_scrub_stamp":"2024-04-25T00:01:04.256331+0000","last_deep_scrub":"27'2557","last_deep_scrub_stamp":"2024-04-25T00:01:04.256331+0000","last_clean_scrub_stamp":"2024-04-25T00:01:04.256331+0000","objects_scrubbed":0,"log_size":2557,"log_dups_size":0,"ondisk_log_size":2557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:32:13.247335+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"27'2540","reported_seq":2585,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:04.914408+0000","last_change":"2024-04-25T00:01:04.914408+0000","last_active":"2024-04-25T00:01:04.914408+0000","last_peered":"2024-04-25T00:01:04.914408+0000","last_clean":"2024-04-25T00:01:04.914408+0000","last_became_active":"2024-04-24T23:34:54.375123+0000","last_became_peered":"2024-04-24T23:34:54.375123+0000","last_unstale":"2024-04-25T00:01:04.914408+0000","last_undegraded":"2024-04-25T00:01:04.914408+0000","last_fullsized":"2024-04-25T00:01:04.914408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2540","last_scrub_stamp":"2024-04-25T00:01:04.914323+0000","last_deep_scrub":"27'2540","last_deep_scrub_stamp":"2024-04-25T00:01:04.914323+0000","last_clean_scrub_stamp":"2024-04-25T00:01:04.914323+0000","objects_scrubbed":0,"log_size":2540,"log_dups_size":0,"ondisk_log_size":2540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:12:52.934728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"27'2526","reported_seq":2570,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:59.305901+0000","last_change":"2024-04-25T00:00:59.305924+0000","last_active":"2024-04-25T00:00:59.305901+0000","last_peered":"2024-04-25T00:00:59.305901+0000","last_clean":"2024-04-25T00:00:59.305901+0000","last_became_active":"2024-04-24T23:34:54.369219+0000","last_became_peered":"2024-04-24T23:34:54.369219+0000","last_unstale":"2024-04-25T00:00:59.305901+0000","last_undegraded":"2024-04-25T00:00:59.305901+0000","last_fullsized":"2024-04-25T00:00:59.305901+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2526","last_scrub_stamp":"2024-04-25T00:00:59.305832+0000","last_deep_scrub":"27'2526","last_deep_scrub_stamp":"2024-04-25T00:00:59.305832+0000","last_clean_scrub_stamp":"2024-04-25T00:00:59.305832+0000","objects_scrubbed":0,"log_size":2526,"log_dups_size":0,"ondisk_log_size":2526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:01.562527+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"27'2564","reported_seq":2594,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:04.973041+0000","last_change":"2024-04-25T00:01:04.973082+0000","last_active":"2024-04-25T00:01:04.973041+0000","last_peered":"2024-04-25T00:01:04.973041+0000","last_clean":"2024-04-25T00:01:04.973041+0000","last_became_active":"2024-04-24T23:35:43.933686+0000","last_became_peered":"2024-04-24T23:35:43.933686+0000","last_unstale":"2024-04-25T00:01:04.973041+0000","last_undegraded":"2024-04-25T00:01:04.973041+0000","last_fullsized":"2024-04-25T00:01:04.973041+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2564","last_scrub_stamp":"2024-04-25T00:01:04.972966+0000","last_deep_scrub":"27'2564","last_deep_scrub_stamp":"2024-04-25T00:01:04.972966+0000","last_clean_scrub_stamp":"2024-04-25T00:01:04.972966+0000","objects_scrubbed":0,"log_size":2564,"log_dups_size":0,"ondisk_log_size":2564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:02:51.830227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2b","version":"27'2598","reported_seq":2642,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:02.014639+0000","last_change":"2024-04-25T00:01:02.014662+0000","last_active":"2024-04-25T00:01:02.014639+0000","last_peered":"2024-04-25T00:01:02.014639+0000","last_clean":"2024-04-25T00:01:02.014639+0000","last_became_active":"2024-04-24T23:34:54.370793+0000","last_became_peered":"2024-04-24T23:34:54.370793+0000","last_unstale":"2024-04-25T00:01:02.014639+0000","last_undegraded":"2024-04-25T00:01:02.014639+0000","last_fullsized":"2024-04-25T00:01:02.014639+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2598","last_scrub_stamp":"2024-04-25T00:01:02.014571+0000","last_deep_scrub":"27'2598","last_deep_scrub_stamp":"2024-04-25T00:01:02.014571+0000","last_clean_scrub_stamp":"2024-04-25T00:01:02.014571+0000","objects_scrubbed":0,"log_size":2598,"log_dups_size":0,"ondisk_log_size":2598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:55:58.335701+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"27'2595","reported_seq":2639,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:04.137947+0000","last_change":"2024-04-25T00:01:04.137986+0000","last_active":"2024-04-25T00:01:04.137947+0000","last_peered":"2024-04-25T00:01:04.137947+0000","last_clean":"2024-04-25T00:01:04.137947+0000","last_became_active":"2024-04-24T23:34:54.376940+0000","last_became_peered":"2024-04-24T23:34:54.376940+0000","last_unstale":"2024-04-25T00:01:04.137947+0000","last_undegraded":"2024-04-25T00:01:04.137947+0000","last_fullsized":"2024-04-25T00:01:04.137947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2595","last_scrub_stamp":"2024-04-25T00:01:04.137832+0000","last_deep_scrub":"27'2595","last_deep_scrub_stamp":"2024-04-25T00:01:04.137832+0000","last_clean_scrub_stamp":"2024-04-25T00:01:04.137832+0000","objects_scrubbed":0,"log_size":2595,"log_dups_size":0,"ondisk_log_size":2595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:45:03.899541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"27'2588","reported_seq":2631,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:58.053198+0000","last_change":"2024-04-25T00:00:58.053220+0000","last_active":"2024-04-25T00:00:58.053198+0000","last_peered":"2024-04-25T00:00:58.053198+0000","last_clean":"2024-04-25T00:00:58.053198+0000","last_became_active":"2024-04-24T23:34:54.368228+0000","last_became_peered":"2024-04-24T23:34:54.368228+0000","last_unstale":"2024-04-25T00:00:58.053198+0000","last_undegraded":"2024-04-25T00:00:58.053198+0000","last_fullsized":"2024-04-25T00:00:58.053198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2588","last_scrub_stamp":"2024-04-25T00:00:58.053136+0000","last_deep_scrub":"27'2588","last_deep_scrub_stamp":"2024-04-25T00:00:58.053136+0000","last_clean_scrub_stamp":"2024-04-25T00:00:58.053136+0000","objects_scrubbed":0,"log_size":2588,"log_dups_size":0,"ondisk_log_size":2588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:44.070301+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"27'2532","reported_seq":2576,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:59.982893+0000","last_change":"2024-04-25T00:00:59.982913+0000","last_active":"2024-04-25T00:00:59.982893+0000","last_peered":"2024-04-25T00:00:59.982893+0000","last_clean":"2024-04-25T00:00:59.982893+0000","last_became_active":"2024-04-24T23:34:54.372766+0000","last_became_peered":"2024-04-24T23:34:54.372766+0000","last_unstale":"2024-04-25T00:00:59.982893+0000","last_undegraded":"2024-04-25T00:00:59.982893+0000","last_fullsized":"2024-04-25T00:00:59.982893+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2532","last_scrub_stamp":"2024-04-25T00:00:59.982822+0000","last_deep_scrub":"27'2532","last_deep_scrub_stamp":"2024-04-25T00:00:59.982822+0000","last_clean_scrub_stamp":"2024-04-25T00:00:59.982822+0000","objects_scrubbed":0,"log_size":2532,"log_dups_size":0,"ondisk_log_size":2532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:11:44.041900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"27'2601","reported_seq":2645,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:09.896631+0000","last_change":"2024-04-25T00:01:09.896670+0000","last_active":"2024-04-25T00:01:09.896631+0000","last_peered":"2024-04-25T00:01:09.896631+0000","last_clean":"2024-04-25T00:01:09.896631+0000","last_became_active":"2024-04-24T23:34:54.383463+0000","last_became_peered":"2024-04-24T23:34:54.383463+0000","last_unstale":"2024-04-25T00:01:09.896631+0000","last_undegraded":"2024-04-25T00:01:09.896631+0000","last_fullsized":"2024-04-25T00:01:09.896631+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2601","last_scrub_stamp":"2024-04-25T00:01:09.896541+0000","last_deep_scrub":"27'2601","last_deep_scrub_stamp":"2024-04-25T00:01:09.896541+0000","last_clean_scrub_stamp":"2024-04-25T00:01:09.896541+0000","objects_scrubbed":0,"log_size":2601,"log_dups_size":0,"ondisk_log_size":2601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:21.944500+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"27'2529","reported_seq":2573,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:07.890936+0000","last_change":"2024-04-25T00:01:07.890974+0000","last_active":"2024-04-25T00:01:07.890936+0000","last_peered":"2024-04-25T00:01:07.890936+0000","last_clean":"2024-04-25T00:01:07.890936+0000","last_became_active":"2024-04-24T23:34:54.380702+0000","last_became_peered":"2024-04-24T23:34:54.380702+0000","last_unstale":"2024-04-25T00:01:07.890936+0000","last_undegraded":"2024-04-25T00:01:07.890936+0000","last_fullsized":"2024-04-25T00:01:07.890936+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2529","last_scrub_stamp":"2024-04-25T00:01:07.890855+0000","last_deep_scrub":"27'2529","last_deep_scrub_stamp":"2024-04-25T00:01:07.890855+0000","last_clean_scrub_stamp":"2024-04-25T00:01:07.890855+0000","objects_scrubbed":0,"log_size":2529,"log_dups_size":0,"ondisk_log_size":2529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:13:25.437443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"27'2650","reported_seq":2694,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:56.127927+0000","last_change":"2024-04-25T00:00:56.127955+0000","last_active":"2024-04-25T00:00:56.127927+0000","last_peered":"2024-04-25T00:00:56.127927+0000","last_clean":"2024-04-25T00:00:56.127927+0000","last_became_active":"2024-04-24T23:34:54.358203+0000","last_became_peered":"2024-04-24T23:34:54.358203+0000","last_unstale":"2024-04-25T00:00:56.127927+0000","last_undegraded":"2024-04-25T00:00:56.127927+0000","last_fullsized":"2024-04-25T00:00:56.127927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2650","last_scrub_stamp":"2024-04-25T00:00:56.127849+0000","last_deep_scrub":"27'2650","last_deep_scrub_stamp":"2024-04-25T00:00:56.127849+0000","last_clean_scrub_stamp":"2024-04-25T00:00:56.127849+0000","objects_scrubbed":0,"log_size":2650,"log_dups_size":0,"ondisk_log_size":2650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:38:09.372461+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"27'2497","reported_seq":2541,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:00.049532+0000","last_change":"2024-04-25T00:01:00.049568+0000","last_active":"2024-04-25T00:01:00.049532+0000","last_peered":"2024-04-25T00:01:00.049532+0000","last_clean":"2024-04-25T00:01:00.049532+0000","last_became_active":"2024-04-24T23:34:54.367595+0000","last_became_peered":"2024-04-24T23:34:54.367595+0000","last_unstale":"2024-04-25T00:01:00.049532+0000","last_undegraded":"2024-04-25T00:01:00.049532+0000","last_fullsized":"2024-04-25T00:01:00.049532+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2497","last_scrub_stamp":"2024-04-25T00:01:00.049447+0000","last_deep_scrub":"27'2497","last_deep_scrub_stamp":"2024-04-25T00:01:00.049447+0000","last_clean_scrub_stamp":"2024-04-25T00:01:00.049447+0000","objects_scrubbed":0,"log_size":2497,"log_dups_size":0,"ondisk_log_size":2497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:18:48.938901+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"27'2502","reported_seq":2531,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:12.172793+0000","last_change":"2024-04-25T00:01:12.172815+0000","last_active":"2024-04-25T00:01:12.172793+0000","last_peered":"2024-04-25T00:01:12.172793+0000","last_clean":"2024-04-25T00:01:12.172793+0000","last_became_active":"2024-04-24T23:35:43.529631+0000","last_became_peered":"2024-04-24T23:35:43.529631+0000","last_unstale":"2024-04-25T00:01:12.172793+0000","last_undegraded":"2024-04-25T00:01:12.172793+0000","last_fullsized":"2024-04-25T00:01:12.172793+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2502","last_scrub_stamp":"2024-04-25T00:01:12.172724+0000","last_deep_scrub":"27'2502","last_deep_scrub_stamp":"2024-04-25T00:01:12.172724+0000","last_clean_scrub_stamp":"2024-04-25T00:01:12.172724+0000","objects_scrubbed":0,"log_size":2502,"log_dups_size":0,"ondisk_log_size":2502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:17:58.506910+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.34","version":"27'2450","reported_seq":2494,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:01.625856+0000","last_change":"2024-04-25T00:01:01.625856+0000","last_active":"2024-04-25T00:01:01.625856+0000","last_peered":"2024-04-25T00:01:01.625856+0000","last_clean":"2024-04-25T00:01:01.625856+0000","last_became_active":"2024-04-24T23:34:54.377277+0000","last_became_peered":"2024-04-24T23:34:54.377277+0000","last_unstale":"2024-04-25T00:01:01.625856+0000","last_undegraded":"2024-04-25T00:01:01.625856+0000","last_fullsized":"2024-04-25T00:01:01.625856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2450","last_scrub_stamp":"2024-04-25T00:01:01.625754+0000","last_deep_scrub":"27'2450","last_deep_scrub_stamp":"2024-04-25T00:01:01.625754+0000","last_clean_scrub_stamp":"2024-04-25T00:01:01.625754+0000","objects_scrubbed":0,"log_size":2450,"log_dups_size":0,"ondisk_log_size":2450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:18:50.515208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"27'2496","reported_seq":2540,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:07.109233+0000","last_change":"2024-04-25T00:01:07.109233+0000","last_active":"2024-04-25T00:01:07.109233+0000","last_peered":"2024-04-25T00:01:07.109233+0000","last_clean":"2024-04-25T00:01:07.109233+0000","last_became_active":"2024-04-24T23:34:54.387781+0000","last_became_peered":"2024-04-24T23:34:54.387781+0000","last_unstale":"2024-04-25T00:01:07.109233+0000","last_undegraded":"2024-04-25T00:01:07.109233+0000","last_fullsized":"2024-04-25T00:01:07.109233+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2496","last_scrub_stamp":"2024-04-25T00:01:07.109132+0000","last_deep_scrub":"27'2496","last_deep_scrub_stamp":"2024-04-25T00:01:07.109132+0000","last_clean_scrub_stamp":"2024-04-25T00:01:07.109132+0000","objects_scrubbed":0,"log_size":2496,"log_dups_size":0,"ondisk_log_size":2496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:07:54.441793+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"27'2613","reported_seq":2656,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:08.106753+0000","last_change":"2024-04-25T00:01:08.106791+0000","last_active":"2024-04-25T00:01:08.106753+0000","last_peered":"2024-04-25T00:01:08.106753+0000","last_clean":"2024-04-25T00:01:08.106753+0000","last_became_active":"2024-04-24T23:34:54.387913+0000","last_became_peered":"2024-04-24T23:34:54.387913+0000","last_unstale":"2024-04-25T00:01:08.106753+0000","last_undegraded":"2024-04-25T00:01:08.106753+0000","last_fullsized":"2024-04-25T00:01:08.106753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2613","last_scrub_stamp":"2024-04-25T00:01:08.106638+0000","last_deep_scrub":"27'2613","last_deep_scrub_stamp":"2024-04-25T00:01:08.106638+0000","last_clean_scrub_stamp":"2024-04-25T00:01:08.106638+0000","objects_scrubbed":0,"log_size":2613,"log_dups_size":0,"ondisk_log_size":2613,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:29:25.481550+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"27'2583","reported_seq":2625,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:13.852424+0000","last_change":"2024-04-25T00:01:13.852447+0000","last_active":"2024-04-25T00:01:13.852424+0000","last_peered":"2024-04-25T00:01:13.852424+0000","last_clean":"2024-04-25T00:01:13.852424+0000","last_became_active":"2024-04-24T23:34:54.383024+0000","last_became_peered":"2024-04-24T23:34:54.383024+0000","last_unstale":"2024-04-25T00:01:13.852424+0000","last_undegraded":"2024-04-25T00:01:13.852424+0000","last_fullsized":"2024-04-25T00:01:13.852424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2583","last_scrub_stamp":"2024-04-25T00:01:13.852347+0000","last_deep_scrub":"27'2583","last_deep_scrub_stamp":"2024-04-25T00:01:13.852347+0000","last_clean_scrub_stamp":"2024-04-25T00:01:13.852347+0000","objects_scrubbed":0,"log_size":2583,"log_dups_size":0,"ondisk_log_size":2583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:04:38.109888+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"27'2526","reported_seq":2569,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:58.299580+0000","last_change":"2024-04-25T00:00:58.299619+0000","last_active":"2024-04-25T00:00:58.299580+0000","last_peered":"2024-04-25T00:00:58.299580+0000","last_clean":"2024-04-25T00:00:58.299580+0000","last_became_active":"2024-04-24T23:34:54.382482+0000","last_became_peered":"2024-04-24T23:34:54.382482+0000","last_unstale":"2024-04-25T00:00:58.299580+0000","last_undegraded":"2024-04-25T00:00:58.299580+0000","last_fullsized":"2024-04-25T00:00:58.299580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2526","last_scrub_stamp":"2024-04-25T00:00:58.299502+0000","last_deep_scrub":"27'2526","last_deep_scrub_stamp":"2024-04-25T00:00:58.299502+0000","last_clean_scrub_stamp":"2024-04-25T00:00:58.299502+0000","objects_scrubbed":0,"log_size":2526,"log_dups_size":0,"ondisk_log_size":2526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:10.303461+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"27'2536","reported_seq":2578,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:07.847203+0000","last_change":"2024-04-25T00:01:07.847223+0000","last_active":"2024-04-25T00:01:07.847203+0000","last_peered":"2024-04-25T00:01:07.847203+0000","last_clean":"2024-04-25T00:01:07.847203+0000","last_became_active":"2024-04-24T23:34:54.368663+0000","last_became_peered":"2024-04-24T23:34:54.368663+0000","last_unstale":"2024-04-25T00:01:07.847203+0000","last_undegraded":"2024-04-25T00:01:07.847203+0000","last_fullsized":"2024-04-25T00:01:07.847203+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2536","last_scrub_stamp":"2024-04-25T00:01:07.847132+0000","last_deep_scrub":"27'2536","last_deep_scrub_stamp":"2024-04-25T00:01:07.847132+0000","last_clean_scrub_stamp":"2024-04-25T00:01:07.847132+0000","objects_scrubbed":0,"log_size":2536,"log_dups_size":0,"ondisk_log_size":2536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:34.663000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"27'2479","reported_seq":2522,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:06.868019+0000","last_change":"2024-04-25T00:01:06.868066+0000","last_active":"2024-04-25T00:01:06.868019+0000","last_peered":"2024-04-25T00:01:06.868019+0000","last_clean":"2024-04-25T00:01:06.868019+0000","last_became_active":"2024-04-24T23:34:54.380052+0000","last_became_peered":"2024-04-24T23:34:54.380052+0000","last_unstale":"2024-04-25T00:01:06.868019+0000","last_undegraded":"2024-04-25T00:01:06.868019+0000","last_fullsized":"2024-04-25T00:01:06.868019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2479","last_scrub_stamp":"2024-04-25T00:01:06.867940+0000","last_deep_scrub":"27'2479","last_deep_scrub_stamp":"2024-04-25T00:01:06.867940+0000","last_clean_scrub_stamp":"2024-04-25T00:01:06.867940+0000","objects_scrubbed":0,"log_size":2479,"log_dups_size":0,"ondisk_log_size":2479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:48.539103+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"27'2516","reported_seq":2558,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:06.916177+0000","last_change":"2024-04-25T00:01:06.916216+0000","last_active":"2024-04-25T00:01:06.916177+0000","last_peered":"2024-04-25T00:01:06.916177+0000","last_clean":"2024-04-25T00:01:06.916177+0000","last_became_active":"2024-04-24T23:34:54.364605+0000","last_became_peered":"2024-04-24T23:34:54.364605+0000","last_unstale":"2024-04-25T00:01:06.916177+0000","last_undegraded":"2024-04-25T00:01:06.916177+0000","last_fullsized":"2024-04-25T00:01:06.916177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2516","last_scrub_stamp":"2024-04-25T00:01:06.916086+0000","last_deep_scrub":"27'2516","last_deep_scrub_stamp":"2024-04-25T00:01:06.916086+0000","last_clean_scrub_stamp":"2024-04-25T00:01:06.916086+0000","objects_scrubbed":0,"log_size":2516,"log_dups_size":0,"ondisk_log_size":2516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:29:07.082246+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"27'2568","reported_seq":2599,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:02.018530+0000","last_change":"2024-04-25T00:01:02.018551+0000","last_active":"2024-04-25T00:01:02.018530+0000","last_peered":"2024-04-25T00:01:02.018530+0000","last_clean":"2024-04-25T00:01:02.018530+0000","last_became_active":"2024-04-24T23:35:43.525191+0000","last_became_peered":"2024-04-24T23:35:43.525191+0000","last_unstale":"2024-04-25T00:01:02.018530+0000","last_undegraded":"2024-04-25T00:01:02.018530+0000","last_fullsized":"2024-04-25T00:01:02.018530+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2568","last_scrub_stamp":"2024-04-25T00:01:02.018471+0000","last_deep_scrub":"27'2568","last_deep_scrub_stamp":"2024-04-25T00:01:02.018471+0000","last_clean_scrub_stamp":"2024-04-25T00:01:02.018471+0000","objects_scrubbed":0,"log_size":2568,"log_dups_size":0,"ondisk_log_size":2568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:38:01.492823+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"27'2637","reported_seq":2682,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:00:58.081112+0000","last_change":"2024-04-25T00:00:58.081137+0000","last_active":"2024-04-25T00:00:58.081112+0000","last_peered":"2024-04-25T00:00:58.081112+0000","last_clean":"2024-04-25T00:00:58.081112+0000","last_became_active":"2024-04-24T23:34:54.365814+0000","last_became_peered":"2024-04-24T23:34:54.365814+0000","last_unstale":"2024-04-25T00:00:58.081112+0000","last_undegraded":"2024-04-25T00:00:58.081112+0000","last_fullsized":"2024-04-25T00:00:58.081112+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2637","last_scrub_stamp":"2024-04-25T00:00:58.081028+0000","last_deep_scrub":"27'2637","last_deep_scrub_stamp":"2024-04-25T00:00:58.081028+0000","last_clean_scrub_stamp":"2024-04-25T00:00:58.081028+0000","objects_scrubbed":0,"log_size":2637,"log_dups_size":0,"ondisk_log_size":2637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:19:32.412193+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"27'2476","reported_seq":2521,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:06.101132+0000","last_change":"2024-04-25T00:01:06.101155+0000","last_active":"2024-04-25T00:01:06.101132+0000","last_peered":"2024-04-25T00:01:06.101132+0000","last_clean":"2024-04-25T00:01:06.101132+0000","last_became_active":"2024-04-24T23:34:54.368663+0000","last_became_peered":"2024-04-24T23:34:54.368663+0000","last_unstale":"2024-04-25T00:01:06.101132+0000","last_undegraded":"2024-04-25T00:01:06.101132+0000","last_fullsized":"2024-04-25T00:01:06.101132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2476","last_scrub_stamp":"2024-04-25T00:01:06.101057+0000","last_deep_scrub":"27'2476","last_deep_scrub_stamp":"2024-04-25T00:01:06.101057+0000","last_clean_scrub_stamp":"2024-04-25T00:01:06.101057+0000","objects_scrubbed":0,"log_size":2476,"log_dups_size":0,"ondisk_log_size":2476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:25.585996+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"27'2458","reported_seq":2502,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T00:01:08.893917+0000","last_change":"2024-04-25T00:01:08.893937+0000","last_active":"2024-04-25T00:01:08.893917+0000","last_peered":"2024-04-25T00:01:08.893917+0000","last_clean":"2024-04-25T00:01:08.893917+0000","last_became_active":"2024-04-24T23:34:54.373531+0000","last_became_peered":"2024-04-24T23:34:54.373531+0000","last_unstale":"2024-04-25T00:01:08.893917+0000","last_undegraded":"2024-04-25T00:01:08.893917+0000","last_fullsized":"2024-04-25T00:01:08.893917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2458","last_scrub_stamp":"2024-04-25T00:01:08.893844+0000","last_deep_scrub":"27'2458","last_deep_scrub_stamp":"2024-04-25T00:01:08.893844+0000","last_clean_scrub_stamp":"2024-04-25T00:01:08.893844+0000","objects_scrubbed":0,"log_size":2458,"log_dups_size":0,"ondisk_log_size":2458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:29:26.268756+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12310,"num_read_kb":3126740,"num_write":212524,"num_write_kb":30664179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":230531,"ondisk_log_size":230531,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1120,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":162975,"ondisk_log_size":162975,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":236421422,"num_objects":79,"num_object_clones":0,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":0,"num_read":3152,"num_read_kb":19253,"num_write":210114,"num_write_kb":1682543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5807,"num_omap_keys":12,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":287891456,"data_stored":472842844,"data_compressed":55745062,"data_compressed_allocated":81436672,"data_compressed_original":266452992,"omap_allocated":0,"internal_metadata":0},"log_size":195731,"ondisk_log_size":195731,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65248,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542467,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":961884,"kb_used_data":28436,"kb_used_omap":1,"kb_used_meta":933438,"kb_avail":93409956,"statfs":{"total":96636764160,"available":95651794944,"internally_reserved":0,"allocated":29118464,"data_stored":47674208,"data_compressed":5564865,"data_compressed_allocated":8151040,"data_compressed_original":26886144,"omap_allocated":1590,"internal_metadata":955840970},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542466,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1126144,"kb_used_data":40056,"kb_used_omap":1,"kb_used_meta":1086078,"kb_avail":93245696,"statfs":{"total":96636764160,"available":95483592704,"internally_reserved":0,"allocated":41017344,"data_stored":68055450,"data_compressed":8055094,"data_compressed_allocated":11661312,"data_compressed_original":38879232,"omap_allocated":1594,"internal_metadata":1112144326},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542466,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":980308,"kb_used_data":24716,"kb_used_omap":1,"kb_used_meta":955582,"kb_avail":93391532,"statfs":{"total":96636764160,"available":95632928768,"internally_reserved":0,"allocated":25309184,"data_stored":42060410,"data_compressed":5058379,"data_compressed_allocated":7299072,"data_compressed_original":24223744,"omap_allocated":1595,"internal_metadata":978516421},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542466,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":924500,"kb_used_data":40972,"kb_used_omap":1,"kb_used_meta":883518,"kb_avail":93447340,"statfs":{"total":96636764160,"available":95690076160,"internally_reserved":0,"allocated":41955328,"data_stored":67226270,"data_compressed":7785480,"data_compressed_allocated":11411456,"data_compressed_original":36864000,"omap_allocated":1595,"internal_metadata":904722885},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542467,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":660908,"kb_used_data":34148,"kb_used_omap":1,"kb_used_meta":626750,"kb_avail":93710932,"statfs":{"total":96636764160,"available":95959994368,"internally_reserved":0,"allocated":34967552,"data_stored":54643232,"data_compressed":6190216,"data_compressed_allocated":9224192,"data_compressed_original":29069312,"omap_allocated":1595,"internal_metadata":641792453},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542467,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1239756,"kb_used_data":39748,"kb_used_omap":1,"kb_used_meta":1199998,"kb_avail":93132084,"statfs":{"total":96636764160,"available":95367254016,"internally_reserved":0,"allocated":40701952,"data_stored":63742294,"data_compressed":7360432,"data_compressed_allocated":10985472,"data_compressed_original":34205696,"omap_allocated":1595,"internal_metadata":1228798405},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542467,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":827048,"kb_used_data":33824,"kb_used_omap":1,"kb_used_meta":793214,"kb_avail":93544792,"statfs":{"total":96636764160,"available":95789867008,"internally_reserved":0,"allocated":34635776,"data_stored":60138836,"data_compressed":7346778,"data_compressed_allocated":10571776,"data_compressed_original":36261888,"omap_allocated":1595,"internal_metadata":812251589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575172,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1145812,"kb_used_data":41612,"kb_used_omap":1,"kb_used_meta":1104190,"kb_avail":93226028,"statfs":{"total":96636764160,"available":95463452672,"internally_reserved":0,"allocated":42610688,"data_stored":71421120,"data_compressed":8449066,"data_compressed_allocated":12222464,"data_compressed_original":41209856,"omap_allocated":1593,"internal_metadata":1130691015},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32624,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32624,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":34283520,"data_stored":59431188,"data_compressed":7314154,"data_compressed_allocated":10526720,"data_compressed_original":35688448,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40415232,"data_stored":63625014,"data_compressed":7360432,"data_compressed_allocated":10985472,"data_compressed_original":34205696,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":34680832,"data_stored":54525952,"data_compressed":6190216,"data_compressed_allocated":9224192,"data_compressed_original":29069312,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":42323968,"data_stored":71303840,"data_compressed":8449066,"data_compressed_allocated":12222464,"data_compressed_original":41209856,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":41668608,"data_stored":67108990,"data_compressed":7785480,"data_compressed_allocated":11411456,"data_compressed_original":36864000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":25022464,"data_stored":41943130,"data_compressed":5058379,"data_compressed_allocated":7299072,"data_compressed_original":24223744,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40730624,"data_stored":67938170,"data_compressed":8055094,"data_compressed_allocated":11661312,"data_compressed_original":38879232,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28766208,"data_stored":46966560,"data_compressed":5532241,"data_compressed_allocated":8105984,"data_compressed_original":26312704,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:01:38.328 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-25T00:01:38.462 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.456+0000 7f9b5e561640 1 Processor -- start 2024-04-25T00:01:38.462 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.456+0000 7f9b5e561640 1 -- start start 2024-04-25T00:01:38.462 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.456+0000 7f9b5e561640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9b58073170 0x7f9b580c32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:38.462 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.456+0000 7f9b5e561640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b580c51c0 0x7f9b580c3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:38.462 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.457+0000 7f9b5e561640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b580c1940 con 0x7f9b580c5780 2024-04-25T00:01:38.463 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.457+0000 7f9b5e561640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f9b5808b1c0 con 0x7f9b58073170 2024-04-25T00:01:38.463 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.457+0000 7f9b5e561640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9b580c6650 con 0x7f9b580c51c0 2024-04-25T00:01:38.463 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.457+0000 7f9b57fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9b58073170 0x7f9b580c32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:38.463 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.457+0000 7f9b57fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9b58073170 0x7f9b580c32e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44168/0 (socket says 172.21.15.157:44168) 2024-04-25T00:01:38.463 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.457+0000 7f9b57fff640 1 -- 172.21.15.157:0/3044298567 learned_addr learned my addr 172.21.15.157:0/3044298567 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:01:38.463 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.457+0000 7f9b577fe640 1 --2- 172.21.15.157:0/3044298567 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b580c51c0 0x7f9b580c3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:38.463 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.457+0000 7f9b56ffd640 1 -- 172.21.15.157:0/3044298567 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3776092012 0 0) 0x7f9b580c1940 con 0x7f9b580c5780 2024-04-25T00:01:38.463 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.461+0000 7f9b56ffd640 1 -- 172.21.15.157:0/3044298567 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b40003500 con 0x7f9b580c5780 2024-04-25T00:01:38.463 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.461+0000 7f9b57fff640 1 -- 172.21.15.157:0/3044298567 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b580c51c0 msgr2=0x7f9b580c3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:38.463 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.461+0000 7f9b56ffd640 1 -- 172.21.15.157:0/3044298567 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1922035290 0 0) 0x7f9b40003500 con 0x7f9b580c5780 2024-04-25T00:01:38.463 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.461+0000 7f9b57fff640 1 --2- 172.21.15.157:0/3044298567 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b580c51c0 0x7f9b580c3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:38.463 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.461+0000 7f9b57fff640 1 -- 172.21.15.157:0/3044298567 >> v1:172.21.15.169:6789/0 conn(0x7f9b580c5780 legacy=0x7f9b5806d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:38.463 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.461+0000 7f9b57fff640 1 -- 172.21.15.157:0/3044298567 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b5808b3f0 con 0x7f9b58073170 2024-04-25T00:01:38.467 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.462+0000 7f9b57fff640 1 --2- 172.21.15.157:0/3044298567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b58073170 0x7f9b580c32e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9b4800b920 tx=0x7f9b4802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52ffa00d4ccc8823 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:38.467 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.462+0000 7f9b56ffd640 1 -- 172.21.15.157:0/3044298567 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b50001a10 con 0x7f9b58073170 2024-04-25T00:01:38.467 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.462+0000 7f9b56ffd640 1 -- 172.21.15.157:0/3044298567 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b4802f510 con 0x7f9b58073170 2024-04-25T00:01:38.467 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.462+0000 7f9b56ffd640 1 -- 172.21.15.157:0/3044298567 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b4802f8c0 con 0x7f9b58073170 2024-04-25T00:01:38.467 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.463+0000 7f9b5e561640 1 -- 172.21.15.157:0/3044298567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b58073170 msgr2=0x7f9b580c32e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:38.467 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.463+0000 7f9b5e561640 1 --2- 172.21.15.157:0/3044298567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b58073170 0x7f9b580c32e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9b4800b920 tx=0x7f9b4802a5f0 comp rx=0 tx=0).stop 2024-04-25T00:01:38.467 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.463+0000 7f9b5e561640 1 -- 172.21.15.157:0/3044298567 shutdown_connections 2024-04-25T00:01:38.467 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.463+0000 7f9b5e561640 1 --2- 172.21.15.157:0/3044298567 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b580c51c0 0x7f9b580c3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:38.467 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.463+0000 7f9b5e561640 1 --2- 172.21.15.157:0/3044298567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b58073170 0x7f9b580c32e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:38.467 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.463+0000 7f9b5e561640 1 -- 172.21.15.157:0/3044298567 >> 172.21.15.157:0/3044298567 conn(0x7f9b580c68e0 msgr2=0x7f9b580c6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:38.467 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.463+0000 7f9b5e561640 1 -- 172.21.15.157:0/3044298567 shutdown_connections 2024-04-25T00:01:38.467 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.463+0000 7f9b5e561640 1 -- 172.21.15.157:0/3044298567 wait complete. 2024-04-25T00:01:38.468 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.464+0000 7f9b5e561640 1 Processor -- start 2024-04-25T00:01:38.468 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.464+0000 7f9b5e561640 1 -- start start 2024-04-25T00:01:38.468 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.465+0000 7f9b5e561640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b58073170 0x7f9b58165ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:38.468 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.465+0000 7f9b5e561640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b580c51c0 0x7f9b581f8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:38.468 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.465+0000 7f9b5e561640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b580c5780 0x7f9b581f9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:38.468 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.465+0000 7f9b5e561640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9b580a6340 con 0x7f9b580c51c0 2024-04-25T00:01:38.468 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.465+0000 7f9b5e561640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9b580c6370 con 0x7f9b580c5780 2024-04-25T00:01:38.468 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.465+0000 7f9b5e561640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9b580c1e20 con 0x7f9b58073170 2024-04-25T00:01:38.468 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.465+0000 7f9b577fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b580c51c0 0x7f9b581f8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:38.468 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.465+0000 7f9b577fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b580c51c0 0x7f9b581f8c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44174/0 (socket says 172.21.15.157:44174) 2024-04-25T00:01:38.468 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.465+0000 7f9b577fe640 1 -- 172.21.15.157:0/3021309755 learned_addr learned my addr 172.21.15.157:0/3021309755 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:01:38.468 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.465+0000 7f9b57fff640 1 --2- 172.21.15.157:0/3021309755 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b58073170 0x7f9b58165ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:38.468 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.465+0000 7f9b5cafe640 1 --2- 172.21.15.157:0/3021309755 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b580c5780 0x7f9b581f9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:38.469 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.465+0000 7f9b577fe640 1 -- 172.21.15.157:0/3021309755 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b58073170 msgr2=0x7f9b58165ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:38.469 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.465+0000 7f9b577fe640 1 --2- 172.21.15.157:0/3021309755 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b58073170 0x7f9b58165ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:38.469 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.465+0000 7f9b577fe640 1 -- 172.21.15.157:0/3021309755 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b580c5780 msgr2=0x7f9b581f9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:38.469 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.465+0000 7f9b577fe640 1 --2- 172.21.15.157:0/3021309755 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b580c5780 0x7f9b581f9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:38.469 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.465+0000 7f9b577fe640 1 -- 172.21.15.157:0/3021309755 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b581fb990 con 0x7f9b580c51c0 2024-04-25T00:01:38.469 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.465+0000 7f9b577fe640 1 --2- 172.21.15.157:0/3021309755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b580c51c0 0x7f9b581f8c30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c00ecb0 tx=0x7f9b4c00c5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:38.469 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.465+0000 7f9b54ff9640 1 -- 172.21.15.157:0/3021309755 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b4c00b040 con 0x7f9b580c51c0 2024-04-25T00:01:38.469 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.466+0000 7f9b54ff9640 1 -- 172.21.15.157:0/3021309755 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b4c00c040 con 0x7f9b580c51c0 2024-04-25T00:01:38.469 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.466+0000 7f9b5e561640 1 -- 172.21.15.157:0/3021309755 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b581ecaf0 con 0x7f9b580c51c0 2024-04-25T00:01:38.469 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.466+0000 7f9b54ff9640 1 -- 172.21.15.157:0/3021309755 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b4c01a610 con 0x7f9b580c51c0 2024-04-25T00:01:38.469 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.466+0000 7f9b5e561640 1 -- 172.21.15.157:0/3021309755 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b581ebda0 con 0x7f9b580c51c0 2024-04-25T00:01:38.470 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.467+0000 7f9b5e561640 1 -- 172.21.15.157:0/3021309755 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b580681c0 con 0x7f9b580c51c0 2024-04-25T00:01:38.470 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.467+0000 7f9b54ff9640 1 -- 172.21.15.157:0/3021309755 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9b4c0113f0 con 0x7f9b580c51c0 2024-04-25T00:01:38.470 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.467+0000 7f9b54ff9640 1 --2- 172.21.15.157:0/3021309755 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f9b4406be60 0x7f9b4406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:38.470 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.468+0000 7f9b54ff9640 1 -- 172.21.15.157:0/3021309755 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f9b4c08b550 con 0x7f9b580c51c0 2024-04-25T00:01:38.470 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.468+0000 7f9b57fff640 1 --2- 172.21.15.157:0/3021309755 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f9b4406be60 0x7f9b4406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:38.471 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.468+0000 7f9b57fff640 1 --2- 172.21.15.157:0/3021309755 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f9b4406be60 0x7f9b4406e320 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9b48034000 tx=0x7f9b4802f000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:38.472 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.472+0000 7f9b54ff9640 1 -- 172.21.15.157:0/3021309755 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9b4c016070 con 0x7f9b580c51c0 2024-04-25T00:01:38.647 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.647+0000 7f9b5e561640 1 -- 172.21.15.157:0/3021309755 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f9b580c41a0 con 0x7f9b580c51c0 2024-04-25T00:01:38.650 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.650+0000 7f9b54ff9640 1 -- 172.21.15.157:0/3021309755 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v10)=0 v10) ==== 123+0+0 (secure 0 0 0) 0x7f9b4c059420 con 0x7f9b580c51c0 2024-04-25T00:01:38.651 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.650+0000 7f9b54ff9640 1 -- 172.21.15.157:0/3021309755 <== mon.0 v2:172.21.15.157:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f9b4c05d720 con 0x7f9b580c51c0 2024-04-25T00:01:38.653 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.652+0000 7f9b5e561640 1 -- 172.21.15.157:0/3021309755 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f9b4406be60 msgr2=0x7f9b4406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:38.653 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.652+0000 7f9b5e561640 1 --2- 172.21.15.157:0/3021309755 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f9b4406be60 0x7f9b4406e320 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9b48034000 tx=0x7f9b4802f000 comp rx=0 tx=0).stop 2024-04-25T00:01:38.653 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.652+0000 7f9b5e561640 1 -- 172.21.15.157:0/3021309755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b580c51c0 msgr2=0x7f9b581f8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:38.653 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.652+0000 7f9b5e561640 1 --2- 172.21.15.157:0/3021309755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b580c51c0 0x7f9b581f8c30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c00ecb0 tx=0x7f9b4c00c5c0 comp rx=0 tx=0).stop 2024-04-25T00:01:38.653 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.652+0000 7f9b5e561640 1 -- 172.21.15.157:0/3021309755 shutdown_connections 2024-04-25T00:01:38.654 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.652+0000 7f9b5e561640 1 --2- 172.21.15.157:0/3021309755 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f9b4406be60 0x7f9b4406e320 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:38.654 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.652+0000 7f9b5e561640 1 --2- 172.21.15.157:0/3021309755 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b580c5780 0x7f9b581f9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:38.654 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.652+0000 7f9b5e561640 1 --2- 172.21.15.157:0/3021309755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b580c51c0 0x7f9b581f8c30 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:38.654 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.652+0000 7f9b5e561640 1 --2- 172.21.15.157:0/3021309755 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b58073170 0x7f9b58165ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:38.654 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.652+0000 7f9b5e561640 1 -- 172.21.15.157:0/3021309755 >> 172.21.15.157:0/3021309755 conn(0x7f9b580c68e0 msgr2=0x7f9b580ab790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:38.654 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.653+0000 7f9b5e561640 1 -- 172.21.15.157:0/3021309755 shutdown_connections 2024-04-25T00:01:38.654 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.653+0000 7f9b5e561640 1 -- 172.21.15.157:0/3021309755 wait complete. 2024-04-25T00:01:38.664 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T00:01:38.664 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:01:38.761 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.760+0000 7fa27d812640 1 Processor -- start 2024-04-25T00:01:38.761 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.760+0000 7fa27d812640 1 -- start start 2024-04-25T00:01:38.763 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.761+0000 7fa27d812640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa278155a60 0x7fa278160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:38.763 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.761+0000 7fa27d812640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2780c8bb0 con 0x7fa278152b40 2024-04-25T00:01:38.763 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.761+0000 7fa27d812640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa278159480 con 0x7fa2780be360 2024-04-25T00:01:38.763 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.761+0000 7fa27d812640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fa2780c8e90 con 0x7fa278155a60 2024-04-25T00:01:38.763 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.761+0000 7fa276ffd640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fa278152b40 0x7fa2780c5630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:50258/0 (socket says 172.21.15.157:50258) 2024-04-25T00:01:38.763 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.761+0000 7fa276ffd640 1 -- 172.21.15.157:0/2290174603 learned_addr learned my addr 172.21.15.157:0/2290174603 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-25T00:01:38.763 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.761+0000 7fa2777fe640 1 --2- 172.21.15.157:0/2290174603 >> v2:172.21.15.169:3300/0 conn(0x7fa278155a60 0x7fa278160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:38.763 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.762+0000 7fa2767fc640 1 -- 172.21.15.157:0/2290174603 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 690857564 0 0) 0x7fa2780c8bb0 con 0x7fa278152b40 2024-04-25T00:01:38.764 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.762+0000 7fa2767fc640 1 -- 172.21.15.157:0/2290174603 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa264003520 con 0x7fa278152b40 2024-04-25T00:01:38.764 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.762+0000 7fa2777fe640 1 -- 172.21.15.157:0/2290174603 >> v1:172.21.15.169:6789/0 conn(0x7fa2780be360 legacy=0x7fa278155350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:38.764 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.762+0000 7fa2777fe640 1 -- 172.21.15.157:0/2290174603 >> v1:172.21.15.157:6789/0 conn(0x7fa278152b40 legacy=0x7fa2780c5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:38.764 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.762+0000 7fa2777fe640 1 -- 172.21.15.157:0/2290174603 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa278159760 con 0x7fa278155a60 2024-04-25T00:01:38.764 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.762+0000 7fa2777fe640 1 --2- 172.21.15.157:0/2290174603 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa278155a60 0x7fa278160ce0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa270009d90 tx=0x7fa27002a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=880ebb0eddffe8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:38.764 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.763+0000 7fa2767fc640 1 -- 172.21.15.157:0/2290174603 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa270031600 con 0x7fa278155a60 2024-04-25T00:01:38.764 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.763+0000 7fa2767fc640 1 -- 172.21.15.157:0/2290174603 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fa2700317a0 con 0x7fa278155a60 2024-04-25T00:01:38.764 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.763+0000 7fa2767fc640 1 -- 172.21.15.157:0/2290174603 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa270031b90 con 0x7fa278155a60 2024-04-25T00:01:38.769 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.763+0000 7fa27d812640 1 -- 172.21.15.157:0/2290174603 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa278155a60 msgr2=0x7fa278160ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:38.769 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.763+0000 7fa27d812640 1 --2- 172.21.15.157:0/2290174603 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa278155a60 0x7fa278160ce0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa270009d90 tx=0x7fa27002a5d0 comp rx=0 tx=0).stop 2024-04-25T00:01:38.769 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.763+0000 7fa27d812640 1 -- 172.21.15.157:0/2290174603 shutdown_connections 2024-04-25T00:01:38.769 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.763+0000 7fa27d812640 1 --2- 172.21.15.157:0/2290174603 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa278155a60 0x7fa278160ce0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:38.769 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.763+0000 7fa27d812640 1 -- 172.21.15.157:0/2290174603 >> 172.21.15.157:0/2290174603 conn(0x7fa2780672c0 msgr2=0x7fa2780676d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:38.770 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.763+0000 7fa27d812640 1 -- 172.21.15.157:0/2290174603 shutdown_connections 2024-04-25T00:01:38.770 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.763+0000 7fa27d812640 1 -- 172.21.15.157:0/2290174603 wait complete. 2024-04-25T00:01:38.770 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.764+0000 7fa27d812640 1 Processor -- start 2024-04-25T00:01:38.770 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.764+0000 7fa27d812640 1 -- start start 2024-04-25T00:01:38.770 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.765+0000 7fa27d812640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2780be360 0x7fa2780ac860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:38.770 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.765+0000 7fa27d812640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa278152b40 0x7fa2781ee280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:38.770 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.765+0000 7fa27d812640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa278155a60 0x7fa2780be930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:38.770 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.765+0000 7fa27d812640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa2780caa00 con 0x7fa2780be360 2024-04-25T00:01:38.770 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.765+0000 7fa27d812640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa2780cb210 con 0x7fa278152b40 2024-04-25T00:01:38.770 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.765+0000 7fa27d812640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa2780cba40 con 0x7fa278155a60 2024-04-25T00:01:38.770 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.765+0000 7fa276ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2780be360 0x7fa2780ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:38.770 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.765+0000 7fa276ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2780be360 0x7fa2780ac860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44176/0 (socket says 172.21.15.157:44176) 2024-04-25T00:01:38.771 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.765+0000 7fa276ffd640 1 -- 172.21.15.157:0/1107846755 learned_addr learned my addr 172.21.15.157:0/1107846755 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:01:38.771 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.765+0000 7fa26ffff640 1 --2- 172.21.15.157:0/1107846755 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa278152b40 0x7fa2781ee280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:38.771 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.765+0000 7fa2777fe640 1 --2- 172.21.15.157:0/1107846755 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa278155a60 0x7fa2780be930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:38.771 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.765+0000 7fa276ffd640 1 -- 172.21.15.157:0/1107846755 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa278155a60 msgr2=0x7fa2780be930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:38.771 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.765+0000 7fa276ffd640 1 --2- 172.21.15.157:0/1107846755 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa278155a60 0x7fa2780be930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:38.771 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.765+0000 7fa276ffd640 1 -- 172.21.15.157:0/1107846755 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa278152b40 msgr2=0x7fa2781ee280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:38.771 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.765+0000 7fa276ffd640 1 --2- 172.21.15.157:0/1107846755 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa278152b40 0x7fa2781ee280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:38.771 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.766+0000 7fa276ffd640 1 -- 172.21.15.157:0/1107846755 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2781eeac0 con 0x7fa2780be360 2024-04-25T00:01:38.771 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.766+0000 7fa26ffff640 1 --2- 172.21.15.157:0/1107846755 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa278152b40 0x7fa2781ee280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:01:38.771 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.766+0000 7fa276ffd640 1 --2- 172.21.15.157:0/1107846755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2780be360 0x7fa2780ac860 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa26800c500 tx=0x7fa26800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:38.771 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.766+0000 7fa274ff9640 1 -- 172.21.15.157:0/1107846755 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa268017070 con 0x7fa2780be360 2024-04-25T00:01:38.771 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.766+0000 7fa274ff9640 1 -- 172.21.15.157:0/1107846755 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fa268005ce0 con 0x7fa2780be360 2024-04-25T00:01:38.772 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.766+0000 7fa27d812640 1 -- 172.21.15.157:0/1107846755 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2781eeda0 con 0x7fa2780be360 2024-04-25T00:01:38.772 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.766+0000 7fa274ff9640 1 -- 172.21.15.157:0/1107846755 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa268007350 con 0x7fa2780be360 2024-04-25T00:01:38.772 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.766+0000 7fa27d812640 1 -- 172.21.15.157:0/1107846755 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2780ad600 con 0x7fa2780be360 2024-04-25T00:01:38.772 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.767+0000 7fa27d812640 1 -- 172.21.15.157:0/1107846755 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2780ad110 con 0x7fa2780be360 2024-04-25T00:01:38.772 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.767+0000 7fa274ff9640 1 -- 172.21.15.157:0/1107846755 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa26801a070 con 0x7fa2780be360 2024-04-25T00:01:38.772 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.768+0000 7fa274ff9640 1 --2- 172.21.15.157:0/1107846755 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fa25806be30 0x7fa25806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:38.772 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.768+0000 7fa274ff9640 1 -- 172.21.15.157:0/1107846755 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fa26808a690 con 0x7fa2780be360 2024-04-25T00:01:38.772 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.768+0000 7fa26ffff640 1 --2- 172.21.15.157:0/1107846755 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fa25806be30 0x7fa25806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:38.772 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.768+0000 7fa26ffff640 1 --2- 172.21.15.157:0/1107846755 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fa25806be30 0x7fa25806e2f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa278155760 tx=0x7fa260009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:38.773 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.772+0000 7fa274ff9640 1 -- 172.21.15.157:0/1107846755 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa2680586c0 con 0x7fa2780be360 2024-04-25T00:01:38.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.983+0000 7fa27d812640 1 -- 172.21.15.157:0/1107846755 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa2780681c0 con 0x7fa2780be360 2024-04-25T00:01:38.985 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.984+0000 7fa274ff9640 1 -- 172.21.15.157:0/1107846755 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5189 (secure 0 0 0) 0x7fa26805c9c0 con 0x7fa2780be360 2024-04-25T00:01:38.985 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-25T00:01:38.985 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1276438765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1276438765},{"type":"v1","addr":"172.21.15.169:6837","nonce":1276438765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1398599086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":1398599086},{"type":"v1","addr":"172.21.15.157:6835","nonce":1398599086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4258,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3795840339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3795840339},{"type":"v1","addr":"172.21.15.157:6834","nonce":3795840339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:34:53.701085+0000","modified":"2024-04-24T23:35:15.789166+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6836/3782746080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6835","nonce":3782746080},{"type":"v1","addr":"172.21.15.169:6836","nonce":3782746080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4270,"qdb_cluster":[4270]},"id":1}]} 2024-04-25T00:01:38.988 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 9 2024-04-25T00:01:38.988 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.986+0000 7fa27d812640 1 -- 172.21.15.157:0/1107846755 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fa25806be30 msgr2=0x7fa25806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:38.988 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.986+0000 7fa27d812640 1 --2- 172.21.15.157:0/1107846755 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fa25806be30 0x7fa25806e2f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa278155760 tx=0x7fa260009040 comp rx=0 tx=0).stop 2024-04-25T00:01:38.988 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.986+0000 7fa27d812640 1 -- 172.21.15.157:0/1107846755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2780be360 msgr2=0x7fa2780ac860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:38.988 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.986+0000 7fa27d812640 1 --2- 172.21.15.157:0/1107846755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2780be360 0x7fa2780ac860 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa26800c500 tx=0x7fa26800c9c0 comp rx=0 tx=0).stop 2024-04-25T00:01:38.989 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.986+0000 7fa27d812640 1 -- 172.21.15.157:0/1107846755 shutdown_connections 2024-04-25T00:01:38.989 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.986+0000 7fa27d812640 1 --2- 172.21.15.157:0/1107846755 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fa25806be30 0x7fa25806e2f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:38.989 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.986+0000 7fa27d812640 1 --2- 172.21.15.157:0/1107846755 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa278155a60 0x7fa2780be930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:38.989 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.986+0000 7fa27d812640 1 --2- 172.21.15.157:0/1107846755 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa278152b40 0x7fa2781ee280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:38.989 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.986+0000 7fa27d812640 1 --2- 172.21.15.157:0/1107846755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2780be360 0x7fa2780ac860 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:38.989 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.986+0000 7fa27d812640 1 -- 172.21.15.157:0/1107846755 >> 172.21.15.157:0/1107846755 conn(0x7fa2780672c0 msgr2=0x7fa27814d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:38.990 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.986+0000 7fa27d812640 1 -- 172.21.15.157:0/1107846755 shutdown_connections 2024-04-25T00:01:38.990 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:38.986+0000 7fa27d812640 1 -- 172.21.15.157:0/1107846755 wait complete. 2024-04-25T00:01:38.998 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:01:39.096 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.096+0000 7fb6654b3640 1 Processor -- start 2024-04-25T00:01:39.096 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.096+0000 7fb6654b3640 1 -- start start 2024-04-25T00:01:39.100 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.097+0000 7fb6654b3640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb660073130 0x7fb6600c2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:39.100 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.097+0000 7fb6654b3640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb6600c4b20 0x7fb66006ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:39.100 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.097+0000 7fb6654b3640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb66014b720 0x7fb66014bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:39.100 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.097+0000 7fb6654b3640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fb66006c400 con 0x7fb6600c4b20 2024-04-25T00:01:39.100 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.097+0000 7fb6654b3640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fb660073510 con 0x7fb66014b720 2024-04-25T00:01:39.100 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.097+0000 7fb6654b3640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb66006c6a0 con 0x7fb660073130 2024-04-25T00:01:39.101 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.097+0000 7fb65e7fc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb6600c4b20 0x7fb66006ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:39.102 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.097+0000 7fb65e7fc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb6600c4b20 0x7fb66006ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44192/0 (socket says 172.21.15.157:44192) 2024-04-25T00:01:39.102 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.097+0000 7fb65e7fc640 1 -- 172.21.15.157:0/3021458017 learned_addr learned my addr 172.21.15.157:0/3021458017 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:01:39.102 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.097+0000 7fb65effd640 1 --2- 172.21.15.157:0/3021458017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb660073130 0x7fb6600c2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:39.102 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.097+0000 7fb65f7fe640 1 --2- 172.21.15.157:0/3021458017 >> v2:172.21.15.169:3300/0 conn(0x7fb66014b720 0x7fb66014bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:39.102 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.097+0000 7fb65e7fc640 1 -- 172.21.15.157:0/3021458017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb660073130 msgr2=0x7fb6600c2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:39.102 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.097+0000 7fb65e7fc640 1 --2- 172.21.15.157:0/3021458017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb660073130 0x7fb6600c2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.102 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.097+0000 7fb65e7fc640 1 -- 172.21.15.157:0/3021458017 >> v2:172.21.15.169:3300/0 conn(0x7fb66014b720 msgr2=0x7fb66014bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:39.102 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.097+0000 7fb65e7fc640 1 --2- 172.21.15.157:0/3021458017 >> v2:172.21.15.169:3300/0 conn(0x7fb66014b720 0x7fb66014bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.102 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.097+0000 7fb65e7fc640 1 -- 172.21.15.157:0/3021458017 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6600c1eb0 con 0x7fb6600c4b20 2024-04-25T00:01:39.102 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.097+0000 7fb65f7fe640 1 --2- 172.21.15.157:0/3021458017 >> v2:172.21.15.169:3300/0 conn(0x7fb66014b720 0x7fb66014bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:39.102 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.097+0000 7fb65effd640 1 --2- 172.21.15.157:0/3021458017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb660073130 0x7fb6600c2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:39.102 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.098+0000 7fb65e7fc640 1 --2- 172.21.15.157:0/3021458017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6600c4b20 0x7fb66006ccb0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb6600708f0 tx=0x7fb65402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9b2b4639ffb232c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:39.103 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.098+0000 7fb65dffb640 1 -- 172.21.15.157:0/3021458017 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb654037070 con 0x7fb6600c4b20 2024-04-25T00:01:39.103 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.098+0000 7fb65dffb640 1 -- 172.21.15.157:0/3021458017 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fb654032440 con 0x7fb6600c4b20 2024-04-25T00:01:39.103 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.098+0000 7fb65dffb640 1 -- 172.21.15.157:0/3021458017 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb654032830 con 0x7fb6600c4b20 2024-04-25T00:01:39.103 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.099+0000 7fb6654b3640 1 -- 172.21.15.157:0/3021458017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6600c4b20 msgr2=0x7fb66006ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:39.103 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.099+0000 7fb6654b3640 1 --2- 172.21.15.157:0/3021458017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6600c4b20 0x7fb66006ccb0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb6600708f0 tx=0x7fb65402a140 comp rx=0 tx=0).stop 2024-04-25T00:01:39.103 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.099+0000 7fb6654b3640 1 -- 172.21.15.157:0/3021458017 shutdown_connections 2024-04-25T00:01:39.103 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.099+0000 7fb6654b3640 1 --2- 172.21.15.157:0/3021458017 >> v2:172.21.15.169:3300/0 conn(0x7fb66014b720 0x7fb66014bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.103 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.099+0000 7fb6654b3640 1 --2- 172.21.15.157:0/3021458017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6600c4b20 0x7fb66006ccb0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.103 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.099+0000 7fb6654b3640 1 --2- 172.21.15.157:0/3021458017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb660073130 0x7fb6600c2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.103 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.099+0000 7fb6654b3640 1 -- 172.21.15.157:0/3021458017 >> 172.21.15.157:0/3021458017 conn(0x7fb6600a8720 msgr2=0x7fb6600a8b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:39.103 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.099+0000 7fb6654b3640 1 -- 172.21.15.157:0/3021458017 shutdown_connections 2024-04-25T00:01:39.103 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.099+0000 7fb6654b3640 1 -- 172.21.15.157:0/3021458017 wait complete. 2024-04-25T00:01:39.106 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.101+0000 7fb6654b3640 1 Processor -- start 2024-04-25T00:01:39.107 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.101+0000 7fb6654b3640 1 -- start start 2024-04-25T00:01:39.107 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.102+0000 7fb6654b3640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb660073130 0x7fb6601486f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:39.107 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.102+0000 7fb6654b3640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6600c4b20 0x7fb6601516f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:39.107 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.102+0000 7fb6654b3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb66014b720 0x7fb660071a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:39.107 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.102+0000 7fb6654b3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb66006d1f0 con 0x7fb66014b720 2024-04-25T00:01:39.107 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.102+0000 7fb6654b3640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb6600c2470 con 0x7fb660073130 2024-04-25T00:01:39.107 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.102+0000 7fb6654b3640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb66006d370 con 0x7fb6600c4b20 2024-04-25T00:01:39.107 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.102+0000 7fb65f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb66014b720 0x7fb660071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:39.110 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.102+0000 7fb65f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb66014b720 0x7fb660071a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44194/0 (socket says 172.21.15.157:44194) 2024-04-25T00:01:39.110 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.102+0000 7fb65effd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb660073130 0x7fb6601486f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:39.111 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.102+0000 7fb65f7fe640 1 -- 172.21.15.157:0/4175933062 learned_addr learned my addr 172.21.15.157:0/4175933062 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:01:39.111 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.102+0000 7fb65e7fc640 1 --2- 172.21.15.157:0/4175933062 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6600c4b20 0x7fb6601516f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:39.111 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.102+0000 7fb65f7fe640 1 -- 172.21.15.157:0/4175933062 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6600c4b20 msgr2=0x7fb6601516f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:39.111 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.102+0000 7fb65f7fe640 1 --2- 172.21.15.157:0/4175933062 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6600c4b20 0x7fb6601516f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.111 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.102+0000 7fb65f7fe640 1 -- 172.21.15.157:0/4175933062 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb660073130 msgr2=0x7fb6601486f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:39.111 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.102+0000 7fb65f7fe640 1 --2- 172.21.15.157:0/4175933062 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb660073130 0x7fb6601486f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.112 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.102+0000 7fb65f7fe640 1 -- 172.21.15.157:0/4175933062 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6601ebe80 con 0x7fb66014b720 2024-04-25T00:01:39.112 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.102+0000 7fb65effd640 1 --2- 172.21.15.157:0/4175933062 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb660073130 0x7fb6601486f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:01:39.112 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.102+0000 7fb65e7fc640 1 --2- 172.21.15.157:0/4175933062 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6600c4b20 0x7fb6601516f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:01:39.112 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.103+0000 7fb65f7fe640 1 --2- 172.21.15.157:0/4175933062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb66014b720 0x7fb660071a40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb658009fd0 tx=0x7fb65800f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:39.112 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.103+0000 7fb6477fe640 1 -- 172.21.15.157:0/4175933062 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb658003b90 con 0x7fb66014b720 2024-04-25T00:01:39.112 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.103+0000 7fb6477fe640 1 -- 172.21.15.157:0/4175933062 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fb658003d50 con 0x7fb66014b720 2024-04-25T00:01:39.113 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.103+0000 7fb6477fe640 1 -- 172.21.15.157:0/4175933062 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6580128d0 con 0x7fb66014b720 2024-04-25T00:01:39.113 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.103+0000 7fb6654b3640 1 -- 172.21.15.157:0/4175933062 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb660148f10 con 0x7fb66014b720 2024-04-25T00:01:39.113 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.103+0000 7fb6654b3640 1 -- 172.21.15.157:0/4175933062 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6601ec100 con 0x7fb66014b720 2024-04-25T00:01:39.113 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.104+0000 7fb6654b3640 1 -- 172.21.15.157:0/4175933062 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6600681c0 con 0x7fb66014b720 2024-04-25T00:01:39.113 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.104+0000 7fb6477fe640 1 -- 172.21.15.157:0/4175933062 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb658012a70 con 0x7fb66014b720 2024-04-25T00:01:39.113 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.105+0000 7fb6477fe640 1 --2- 172.21.15.157:0/4175933062 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fb64c06be30 0x7fb64c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:39.114 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.105+0000 7fb6477fe640 1 -- 172.21.15.157:0/4175933062 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fb65808dee0 con 0x7fb66014b720 2024-04-25T00:01:39.114 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.105+0000 7fb65effd640 1 --2- 172.21.15.157:0/4175933062 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fb64c06be30 0x7fb64c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:39.114 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.105+0000 7fb65effd640 1 --2- 172.21.15.157:0/4175933062 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fb64c06be30 0x7fb64c06e2f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb6600c62a0 tx=0x7fb650008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:39.115 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.109+0000 7fb6477fe640 1 -- 172.21.15.157:0/4175933062 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb6601ebe80 con 0x7fb66014b720 2024-04-25T00:01:39.321 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.320+0000 7fb6654b3640 1 -- 172.21.15.157:0/4175933062 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb66006dda0 con 0x7fb66014b720 2024-04-25T00:01:39.322 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.321+0000 7fb6477fe640 1 -- 172.21.15.157:0/4175933062 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5189 (secure 0 0 0) 0x7fb658018020 con 0x7fb66014b720 2024-04-25T00:01:39.322 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-25T00:01:39.322 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1276438765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1276438765},{"type":"v1","addr":"172.21.15.169:6837","nonce":1276438765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1398599086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":1398599086},{"type":"v1","addr":"172.21.15.157:6835","nonce":1398599086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4258,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3795840339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3795840339},{"type":"v1","addr":"172.21.15.157:6834","nonce":3795840339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:34:53.701085+0000","modified":"2024-04-24T23:35:15.789166+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6836/3782746080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6835","nonce":3782746080},{"type":"v1","addr":"172.21.15.169:6836","nonce":3782746080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4270,"qdb_cluster":[4270]},"id":1}]} 2024-04-25T00:01:39.322 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 9 2024-04-25T00:01:39.324 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.323+0000 7fb6654b3640 1 -- 172.21.15.157:0/4175933062 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fb64c06be30 msgr2=0x7fb64c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:39.324 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.323+0000 7fb6654b3640 1 --2- 172.21.15.157:0/4175933062 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fb64c06be30 0x7fb64c06e2f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb6600c62a0 tx=0x7fb650008040 comp rx=0 tx=0).stop 2024-04-25T00:01:39.324 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.323+0000 7fb6654b3640 1 -- 172.21.15.157:0/4175933062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb66014b720 msgr2=0x7fb660071a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:39.324 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.323+0000 7fb6654b3640 1 --2- 172.21.15.157:0/4175933062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb66014b720 0x7fb660071a40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb658009fd0 tx=0x7fb65800f770 comp rx=0 tx=0).stop 2024-04-25T00:01:39.325 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.323+0000 7fb6654b3640 1 -- 172.21.15.157:0/4175933062 shutdown_connections 2024-04-25T00:01:39.325 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.323+0000 7fb6654b3640 1 --2- 172.21.15.157:0/4175933062 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fb64c06be30 0x7fb64c06e2f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.325 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.323+0000 7fb6654b3640 1 --2- 172.21.15.157:0/4175933062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb66014b720 0x7fb660071a40 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.325 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.323+0000 7fb6654b3640 1 --2- 172.21.15.157:0/4175933062 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6600c4b20 0x7fb6601516f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.325 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.323+0000 7fb6654b3640 1 --2- 172.21.15.157:0/4175933062 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb660073130 0x7fb6601486f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.325 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.323+0000 7fb6654b3640 1 -- 172.21.15.157:0/4175933062 >> 172.21.15.157:0/4175933062 conn(0x7fb6600a8720 msgr2=0x7fb6600a7510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:39.325 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.323+0000 7fb6654b3640 1 -- 172.21.15.157:0/4175933062 shutdown_connections 2024-04-25T00:01:39.325 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.323+0000 7fb6654b3640 1 -- 172.21.15.157:0/4175933062 wait complete. 2024-04-25T00:01:39.335 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:01:39.433 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.432+0000 7f0540310640 1 Processor -- start 2024-04-25T00:01:39.433 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.432+0000 7f0540310640 1 -- start start 2024-04-25T00:01:39.435 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.433+0000 7f0540310640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0538152b40 0x7f05380c5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:39.436 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.433+0000 7f0540310640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05380be360 0x7f0538155350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:39.436 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.433+0000 7f0540310640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05380c8bb0 con 0x7f05380c5b70 2024-04-25T00:01:39.436 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.433+0000 7f0540310640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f05380c8e90 con 0x7f0538152b40 2024-04-25T00:01:39.436 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.433+0000 7f0540310640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0538159480 con 0x7f05380be360 2024-04-25T00:01:39.436 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.433+0000 7f053e8ad640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f05380c5b70 0x7f0538155890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:50262/0 (socket says 172.21.15.157:50262) 2024-04-25T00:01:39.436 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.433+0000 7f053e8ad640 1 -- 172.21.15.157:0/2388077082 learned_addr learned my addr 172.21.15.157:0/2388077082 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-25T00:01:39.436 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.433+0000 7f053d8ab640 1 --2- 172.21.15.157:0/2388077082 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05380be360 0x7f0538155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:39.436 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.433+0000 7f053e0ac640 1 --2- 172.21.15.157:0/2388077082 >> v2:172.21.15.169:3300/0 conn(0x7f0538152b40 0x7f05380c5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:39.436 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.433+0000 7f053d0aa640 1 -- 172.21.15.157:0/2388077082 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 982547516 0 0) 0x7f05380c8bb0 con 0x7f05380c5b70 2024-04-25T00:01:39.436 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.433+0000 7f053d0aa640 1 -- 172.21.15.157:0/2388077082 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0524003520 con 0x7f05380c5b70 2024-04-25T00:01:39.436 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.433+0000 7f053e0ac640 1 -- 172.21.15.157:0/2388077082 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05380be360 msgr2=0x7f0538155350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:39.436 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.434+0000 7f053e0ac640 1 --2- 172.21.15.157:0/2388077082 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05380be360 0x7f0538155350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.436 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.434+0000 7f053e0ac640 1 -- 172.21.15.157:0/2388077082 >> v1:172.21.15.157:6789/0 conn(0x7f05380c5b70 legacy=0x7f0538155890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:39.437 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.434+0000 7f053e0ac640 1 -- 172.21.15.157:0/2388077082 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0538159760 con 0x7f0538152b40 2024-04-25T00:01:39.437 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.434+0000 7f053d8ab640 1 --2- 172.21.15.157:0/2388077082 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05380be360 0x7f0538155350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:01:39.437 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.434+0000 7f053e0ac640 1 --2- 172.21.15.157:0/2388077082 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0538152b40 0x7f05380c5630 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f05280097a0 tx=0x7f052802a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a703f6bd4e6ac223 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:39.437 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.434+0000 7f053d0aa640 1 -- 172.21.15.157:0/2388077082 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0528037070 con 0x7f0538152b40 2024-04-25T00:01:39.437 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.434+0000 7f053d0aa640 1 -- 172.21.15.157:0/2388077082 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f052802adb0 con 0x7f0538152b40 2024-04-25T00:01:39.437 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.434+0000 7f053d0aa640 1 -- 172.21.15.157:0/2388077082 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0528032640 con 0x7f0538152b40 2024-04-25T00:01:39.437 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.435+0000 7f0540310640 1 -- 172.21.15.157:0/2388077082 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0538152b40 msgr2=0x7f05380c5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:39.437 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.435+0000 7f0540310640 1 --2- 172.21.15.157:0/2388077082 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0538152b40 0x7f05380c5630 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f05280097a0 tx=0x7f052802a060 comp rx=0 tx=0).stop 2024-04-25T00:01:39.437 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.435+0000 7f0540310640 1 -- 172.21.15.157:0/2388077082 shutdown_connections 2024-04-25T00:01:39.437 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.435+0000 7f0540310640 1 --2- 172.21.15.157:0/2388077082 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05380be360 0x7f0538155350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.437 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.435+0000 7f0540310640 1 --2- 172.21.15.157:0/2388077082 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0538152b40 0x7f05380c5630 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.437 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.435+0000 7f0540310640 1 -- 172.21.15.157:0/2388077082 >> 172.21.15.157:0/2388077082 conn(0x7f05380672c0 msgr2=0x7f05380676d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:39.437 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.435+0000 7f0540310640 1 -- 172.21.15.157:0/2388077082 shutdown_connections 2024-04-25T00:01:39.437 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.435+0000 7f0540310640 1 -- 172.21.15.157:0/2388077082 wait complete. 2024-04-25T00:01:39.440 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.436+0000 7f0540310640 1 Processor -- start 2024-04-25T00:01:39.440 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.436+0000 7f0540310640 1 -- start start 2024-04-25T00:01:39.440 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.437+0000 7f0540310640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05380be360 0x7f05380be7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:39.440 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.437+0000 7f0540310640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05380c5b70 0x7f0538157900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:39.440 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.437+0000 7f0540310640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0538152b40 0x7f05380bf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:39.440 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.437+0000 7f0540310640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f05380caa00 con 0x7f0538152b40 2024-04-25T00:01:39.440 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.437+0000 7f0540310640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f05380cb210 con 0x7f05380be360 2024-04-25T00:01:39.440 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.437+0000 7f0540310640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f05380cba40 con 0x7f05380c5b70 2024-04-25T00:01:39.440 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.437+0000 7f053e8ad640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0538152b40 0x7f05380bf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:39.440 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.437+0000 7f053e8ad640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0538152b40 0x7f05380bf4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44210/0 (socket says 172.21.15.157:44210) 2024-04-25T00:01:39.440 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.437+0000 7f053e8ad640 1 -- 172.21.15.157:0/939637597 learned_addr learned my addr 172.21.15.157:0/939637597 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:01:39.440 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.437+0000 7f053e0ac640 1 --2- 172.21.15.157:0/939637597 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05380be360 0x7f05380be7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:39.440 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.437+0000 7f053d8ab640 1 --2- 172.21.15.157:0/939637597 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05380c5b70 0x7f0538157900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:39.440 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.437+0000 7f053e8ad640 1 -- 172.21.15.157:0/939637597 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05380c5b70 msgr2=0x7f0538157900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:39.441 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.437+0000 7f053e8ad640 1 --2- 172.21.15.157:0/939637597 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05380c5b70 0x7f0538157900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.441 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.437+0000 7f053e8ad640 1 -- 172.21.15.157:0/939637597 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05380be360 msgr2=0x7f05380be7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:39.441 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.437+0000 7f053e8ad640 1 --2- 172.21.15.157:0/939637597 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05380be360 0x7f05380be7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.441 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.437+0000 7f053e8ad640 1 -- 172.21.15.157:0/939637597 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05380c13b0 con 0x7f0538152b40 2024-04-25T00:01:39.441 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.437+0000 7f053e0ac640 1 --2- 172.21.15.157:0/939637597 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05380be360 0x7f05380be7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:01:39.441 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.437+0000 7f053e8ad640 1 --2- 172.21.15.157:0/939637597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0538152b40 0x7f05380bf4f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f053000cb70 tx=0x7f053000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:39.441 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.438+0000 7f052effd640 1 -- 172.21.15.157:0/939637597 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0530017070 con 0x7f0538152b40 2024-04-25T00:01:39.441 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.438+0000 7f052effd640 1 -- 172.21.15.157:0/939637597 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f053000ad40 con 0x7f0538152b40 2024-04-25T00:01:39.441 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.438+0000 7f0540310640 1 -- 172.21.15.157:0/939637597 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05380c1690 con 0x7f0538152b40 2024-04-25T00:01:39.441 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.438+0000 7f052effd640 1 -- 172.21.15.157:0/939637597 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0530012640 con 0x7f0538152b40 2024-04-25T00:01:39.441 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.438+0000 7f0540310640 1 -- 172.21.15.157:0/939637597 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0538157e40 con 0x7f0538152b40 2024-04-25T00:01:39.441 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.439+0000 7f0540310640 1 -- 172.21.15.157:0/939637597 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0538069260 con 0x7f0538152b40 2024-04-25T00:01:39.441 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.439+0000 7f052effd640 1 -- 172.21.15.157:0/939637597 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f053001a070 con 0x7f0538152b40 2024-04-25T00:01:39.442 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.440+0000 7f052effd640 1 --2- 172.21.15.157:0/939637597 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f051c06be60 0x7f051c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:39.442 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.440+0000 7f052effd640 1 -- 172.21.15.157:0/939637597 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f053008b060 con 0x7f0538152b40 2024-04-25T00:01:39.442 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.440+0000 7f053e0ac640 1 --2- 172.21.15.157:0/939637597 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f051c06be60 0x7f051c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:39.442 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.440+0000 7f053e0ac640 1 --2- 172.21.15.157:0/939637597 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f051c06be60 0x7f051c06e320 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0528034000 tx=0x7f052802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:39.444 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.444+0000 7f052effd640 1 -- 172.21.15.157:0/939637597 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0538069260 con 0x7f0538152b40 2024-04-25T00:01:39.620 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.619+0000 7f0540310640 1 -- 172.21.15.157:0/939637597 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f05380c1860 con 0x7f0538152b40 2024-04-25T00:01:39.621 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.620+0000 7f052effd640 1 -- 172.21.15.157:0/939637597 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18928 (secure 0 0 0) 0x7f0530058f30 con 0x7f0538152b40 2024-04-25T00:01:39.621 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-25T00:01:39.621 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":27,"fsid":"223685fb-5b64-46a5-9857-3791c9c07df2","created":"2024-04-24T23:34:37.504202+0000","modified":"2024-04-24T23:35:43.517316+0000","last_up_change":"2024-04-24T23:34:47.584963+0000","last_in_change":"2024-04-24T23:34:39.632954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:34:47.796655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:34:52.177208+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:34:52.667022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:34:56.395517+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"95d14ac8-fccf-4261-be8e-edafa746f629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6809","nonce":4003822160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6811","nonce":4003822160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6815","nonce":4003822160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":4003822160},{"type":"v1","addr":"172.21.15.157:6813","nonce":4003822160}]},"public_addr":"172.21.15.157:6809/4003822160","cluster_addr":"172.21.15.157:6811/4003822160","heartbeat_back_addr":"172.21.15.157:6815/4003822160","heartbeat_front_addr":"172.21.15.157:6813/4003822160","state":["exists","up"]},{"osd":1,"uuid":"7be79da9-8eec-4990-9178-71d602e39d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6817","nonce":1557575576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6819","nonce":1557575576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6823","nonce":1557575576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1557575576},{"type":"v1","addr":"172.21.15.157:6821","nonce":1557575576}]},"public_addr":"172.21.15.157:6817/1557575576","cluster_addr":"172.21.15.157:6819/1557575576","heartbeat_back_addr":"172.21.15.157:6823/1557575576","heartbeat_front_addr":"172.21.15.157:6821/1557575576","state":["exists","up"]},{"osd":2,"uuid":"0127ed00-b7c3-466d-bfb0-01b0fd315f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6825","nonce":2793948092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6827","nonce":2793948092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6831","nonce":2793948092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2793948092},{"type":"v1","addr":"172.21.15.157:6829","nonce":2793948092}]},"public_addr":"172.21.15.157:6825/2793948092","cluster_addr":"172.21.15.157:6827/2793948092","heartbeat_back_addr":"172.21.15.157:6831/2793948092","heartbeat_front_addr":"172.21.15.157:6829/2793948092","state":["exists","up"]},{"osd":3,"uuid":"f2331acc-aa37-433e-868e-3561b6039562","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6801","nonce":422758549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6803","nonce":422758549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6807","nonce":422758549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":422758549},{"type":"v1","addr":"172.21.15.157:6805","nonce":422758549}]},"public_addr":"172.21.15.157:6801/422758549","cluster_addr":"172.21.15.157:6803/422758549","heartbeat_back_addr":"172.21.15.157:6807/422758549","heartbeat_front_addr":"172.21.15.157:6805/422758549","state":["exists","up"]},{"osd":4,"uuid":"1998854b-7398-43f5-ab7d-79fa0debcf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6801","nonce":321910099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6803","nonce":321910099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6807","nonce":321910099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":321910099},{"type":"v1","addr":"172.21.15.169:6805","nonce":321910099}]},"public_addr":"172.21.15.169:6801/321910099","cluster_addr":"172.21.15.169:6803/321910099","heartbeat_back_addr":"172.21.15.169:6807/321910099","heartbeat_front_addr":"172.21.15.169:6805/321910099","state":["exists","up"]},{"osd":5,"uuid":"c37153ff-9b2b-48b9-a684-2c82b6f39f15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6825","nonce":243321711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6827","nonce":243321711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6831","nonce":243321711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":243321711},{"type":"v1","addr":"172.21.15.169:6829","nonce":243321711}]},"public_addr":"172.21.15.169:6825/243321711","cluster_addr":"172.21.15.169:6827/243321711","heartbeat_back_addr":"172.21.15.169:6831/243321711","heartbeat_front_addr":"172.21.15.169:6829/243321711","state":["exists","up"]},{"osd":6,"uuid":"8db79e1c-f36b-49cb-8d8a-a20c2aec57c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6809","nonce":3441420785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6811","nonce":3441420785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6815","nonce":3441420785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3441420785},{"type":"v1","addr":"172.21.15.169:6813","nonce":3441420785}]},"public_addr":"172.21.15.169:6809/3441420785","cluster_addr":"172.21.15.169:6811/3441420785","heartbeat_back_addr":"172.21.15.169:6815/3441420785","heartbeat_front_addr":"172.21.15.169:6813/3441420785","state":["exists","up"]},{"osd":7,"uuid":"928586e9-296c-4b8b-9b27-42f9f155dbdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6817","nonce":1663179020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6819","nonce":1663179020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6823","nonce":1663179020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1663179020},{"type":"v1","addr":"172.21.15.169:6821","nonce":1663179020}]},"public_addr":"172.21.15.169:6817/1663179020","cluster_addr":"172.21.15.169:6819/1663179020","heartbeat_back_addr":"172.21.15.169:6823/1663179020","heartbeat_front_addr":"172.21.15.169:6821/1663179020","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.848039+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.826489+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.801120+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.727706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.824343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.791915+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.878073+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:34:44.810831+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.33","mappings":[{"from":3,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:01:39.623 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.622+0000 7f0540310640 1 -- 172.21.15.157:0/939637597 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f051c06be60 msgr2=0x7f051c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:39.623 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.622+0000 7f0540310640 1 --2- 172.21.15.157:0/939637597 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f051c06be60 0x7f051c06e320 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0528034000 tx=0x7f052802e000 comp rx=0 tx=0).stop 2024-04-25T00:01:39.624 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.622+0000 7f0540310640 1 -- 172.21.15.157:0/939637597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0538152b40 msgr2=0x7f05380bf4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:39.624 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.622+0000 7f0540310640 1 --2- 172.21.15.157:0/939637597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0538152b40 0x7f05380bf4f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f053000cb70 tx=0x7f053000a430 comp rx=0 tx=0).stop 2024-04-25T00:01:39.624 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.622+0000 7f0540310640 1 -- 172.21.15.157:0/939637597 shutdown_connections 2024-04-25T00:01:39.624 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.622+0000 7f0540310640 1 --2- 172.21.15.157:0/939637597 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f051c06be60 0x7f051c06e320 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.624 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.622+0000 7f0540310640 1 --2- 172.21.15.157:0/939637597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0538152b40 0x7f05380bf4f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.624 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.622+0000 7f0540310640 1 --2- 172.21.15.157:0/939637597 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05380c5b70 0x7f0538157900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.624 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.622+0000 7f0540310640 1 --2- 172.21.15.157:0/939637597 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05380be360 0x7f05380be7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.624 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.622+0000 7f0540310640 1 -- 172.21.15.157:0/939637597 >> 172.21.15.157:0/939637597 conn(0x7f05380672c0 msgr2=0x7f053814afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:39.624 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.623+0000 7f0540310640 1 -- 172.21.15.157:0/939637597 shutdown_connections 2024-04-25T00:01:39.624 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.623+0000 7f0540310640 1 -- 172.21.15.157:0/939637597 wait complete. 2024-04-25T00:01:39.634 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T00:01:39.737 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.733+0000 7f18c3914640 1 Processor -- start 2024-04-25T00:01:39.737 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.733+0000 7f18c3914640 1 -- start start 2024-04-25T00:01:39.738 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.734+0000 7f18c3914640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f18bc0c68d0 0x7f18bc0c6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:39.738 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.734+0000 7f18c3914640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18bc0c58a0 0x7f18bc0c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:39.738 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.734+0000 7f18c3914640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18bc06bd80 con 0x7f18bc0c71f0 2024-04-25T00:01:39.738 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.734+0000 7f18c3914640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f18bc0c6730 con 0x7f18bc0c68d0 2024-04-25T00:01:39.738 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.734+0000 7f18c3914640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f18bc1561a0 con 0x7f18bc0c58a0 2024-04-25T00:01:39.739 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.734+0000 7f18c16b0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f18bc0c68d0 0x7f18bc0c6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:39.739 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.734+0000 7f18c0eaf640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f18bc0c71f0 0x7f18bc0c8ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:50270/0 (socket says 172.21.15.157:50270) 2024-04-25T00:01:39.739 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.734+0000 7f18c16b0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f18bc0c68d0 0x7f18bc0c6cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44220/0 (socket says 172.21.15.157:44220) 2024-04-25T00:01:39.739 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.734+0000 7f18c0eaf640 1 -- 172.21.15.157:0/2363438964 learned_addr learned my addr 172.21.15.157:0/2363438964 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-25T00:01:39.739 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.734+0000 7f18c1eb1640 1 --2- 172.21.15.157:0/2363438964 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18bc0c58a0 0x7f18bc0c92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:39.739 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.734+0000 7f18c16b0640 1 -- 172.21.15.157:0/2363438964 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18bc0c58a0 msgr2=0x7f18bc0c92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:39.739 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.734+0000 7f18c16b0640 1 --2- 172.21.15.157:0/2363438964 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18bc0c58a0 0x7f18bc0c92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.739 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.734+0000 7f18c16b0640 1 -- 172.21.15.157:0/2363438964 >> v1:172.21.15.157:6789/0 conn(0x7f18bc0c71f0 legacy=0x7f18bc0c8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:39.739 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.734+0000 7f18b3fff640 1 -- 172.21.15.157:0/2363438964 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2516998894 0 0) 0x7f18bc06bd80 con 0x7f18bc0c71f0 2024-04-25T00:01:39.739 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.734+0000 7f18c16b0640 1 -- 172.21.15.157:0/2363438964 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18bc0c6370 con 0x7f18bc0c68d0 2024-04-25T00:01:39.739 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.735+0000 7f18c16b0640 1 --2- 172.21.15.157:0/2363438964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18bc0c68d0 0x7f18bc0c6cb0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f18ac0097a0 tx=0x7f18ac02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=202da5b0a8ff305 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:39.739 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.736+0000 7f18b3fff640 1 -- 172.21.15.157:0/2363438964 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ac037070 con 0x7f18bc0c68d0 2024-04-25T00:01:39.739 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.736+0000 7f18b3fff640 1 -- 172.21.15.157:0/2363438964 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f18ac032440 con 0x7f18bc0c68d0 2024-04-25T00:01:39.740 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.736+0000 7f18b3fff640 1 -- 172.21.15.157:0/2363438964 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ac032830 con 0x7f18bc0c68d0 2024-04-25T00:01:39.740 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.736+0000 7f18c3914640 1 -- 172.21.15.157:0/2363438964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18bc0c68d0 msgr2=0x7f18bc0c6cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:39.740 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.736+0000 7f18c3914640 1 --2- 172.21.15.157:0/2363438964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18bc0c68d0 0x7f18bc0c6cb0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f18ac0097a0 tx=0x7f18ac02a210 comp rx=0 tx=0).stop 2024-04-25T00:01:39.740 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.736+0000 7f18c3914640 1 -- 172.21.15.157:0/2363438964 shutdown_connections 2024-04-25T00:01:39.740 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.736+0000 7f18c3914640 1 --2- 172.21.15.157:0/2363438964 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18bc0c58a0 0x7f18bc0c92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.740 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.736+0000 7f18c3914640 1 --2- 172.21.15.157:0/2363438964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18bc0c68d0 0x7f18bc0c6cb0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.740 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.736+0000 7f18c3914640 1 -- 172.21.15.157:0/2363438964 >> 172.21.15.157:0/2363438964 conn(0x7f18bc0caca0 msgr2=0x7f18bc0cb0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:39.740 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.736+0000 7f18c3914640 1 -- 172.21.15.157:0/2363438964 shutdown_connections 2024-04-25T00:01:39.740 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.736+0000 7f18c3914640 1 -- 172.21.15.157:0/2363438964 wait complete. 2024-04-25T00:01:39.743 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.737+0000 7f18c3914640 1 Processor -- start 2024-04-25T00:01:39.743 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.737+0000 7f18c3914640 1 -- start start 2024-04-25T00:01:39.743 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.738+0000 7f18c3914640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18bc0c58a0 0x7f18bc154c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:39.743 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.738+0000 7f18c3914640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18bc0c68d0 0x7f18bc144530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:39.743 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.738+0000 7f18c3914640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18bc0c71f0 0x7f18bc144a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:39.743 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.738+0000 7f18c3914640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f18bc152ed0 con 0x7f18bc0c58a0 2024-04-25T00:01:39.743 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.738+0000 7f18c3914640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f18bc06b830 con 0x7f18bc0c71f0 2024-04-25T00:01:39.743 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.738+0000 7f18c3914640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f18bc153050 con 0x7f18bc0c68d0 2024-04-25T00:01:39.743 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.738+0000 7f18c16b0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18bc0c58a0 0x7f18bc154c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:39.743 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.738+0000 7f18c0eaf640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18bc0c68d0 0x7f18bc144530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:39.743 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.738+0000 7f18c16b0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18bc0c58a0 0x7f18bc154c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44226/0 (socket says 172.21.15.157:44226) 2024-04-25T00:01:39.743 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.738+0000 7f18c16b0640 1 -- 172.21.15.157:0/3323092879 learned_addr learned my addr 172.21.15.157:0/3323092879 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:01:39.743 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.738+0000 7f18c1eb1640 1 --2- 172.21.15.157:0/3323092879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18bc0c71f0 0x7f18bc144a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:39.744 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.738+0000 7f18c16b0640 1 -- 172.21.15.157:0/3323092879 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18bc0c68d0 msgr2=0x7f18bc144530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:39.744 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.738+0000 7f18c16b0640 1 --2- 172.21.15.157:0/3323092879 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18bc0c68d0 0x7f18bc144530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.744 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.738+0000 7f18c16b0640 1 -- 172.21.15.157:0/3323092879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18bc0c71f0 msgr2=0x7f18bc144a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:39.744 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.738+0000 7f18c16b0640 1 --2- 172.21.15.157:0/3323092879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18bc0c71f0 0x7f18bc144a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.744 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.738+0000 7f18c16b0640 1 -- 172.21.15.157:0/3323092879 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18bc14e6b0 con 0x7f18bc0c58a0 2024-04-25T00:01:39.744 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.738+0000 7f18c1eb1640 1 --2- 172.21.15.157:0/3323092879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18bc0c71f0 0x7f18bc144a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:01:39.744 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.739+0000 7f18c16b0640 1 --2- 172.21.15.157:0/3323092879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18bc0c58a0 0x7f18bc154c70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f18ac0098c0 tx=0x7f18ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:39.744 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.739+0000 7f18b1ffb640 1 -- 172.21.15.157:0/3323092879 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ac037040 con 0x7f18bc0c58a0 2024-04-25T00:01:39.744 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.739+0000 7f18b1ffb640 1 -- 172.21.15.157:0/3323092879 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f18ac042070 con 0x7f18bc0c58a0 2024-04-25T00:01:39.744 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.739+0000 7f18b1ffb640 1 -- 172.21.15.157:0/3323092879 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ac0309a0 con 0x7f18bc0c58a0 2024-04-25T00:01:39.744 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.739+0000 7f18c3914640 1 -- 172.21.15.157:0/3323092879 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18bc1ebff0 con 0x7f18bc0c58a0 2024-04-25T00:01:39.744 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.739+0000 7f18c3914640 1 -- 172.21.15.157:0/3323092879 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18bc1eb750 con 0x7f18bc0c58a0 2024-04-25T00:01:39.744 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.740+0000 7f18c3914640 1 -- 172.21.15.157:0/3323092879 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18bc0681c0 con 0x7f18bc0c58a0 2024-04-25T00:01:39.745 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.740+0000 7f18b1ffb640 1 -- 172.21.15.157:0/3323092879 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f18ac030b40 con 0x7f18bc0c58a0 2024-04-25T00:01:39.745 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.741+0000 7f18b1ffb640 1 --2- 172.21.15.157:0/3323092879 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f18a006be30 0x7f18a006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:39.745 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.741+0000 7f18b1ffb640 1 -- 172.21.15.157:0/3323092879 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f18ac0abbb0 con 0x7f18bc0c58a0 2024-04-25T00:01:39.745 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.741+0000 7f18c0eaf640 1 --2- 172.21.15.157:0/3323092879 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f18a006be30 0x7f18a006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:39.745 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.741+0000 7f18c0eaf640 1 --2- 172.21.15.157:0/3323092879 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f18a006be30 0x7f18a006e2f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f18bc155590 tx=0x7f18b8007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:39.745 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.745+0000 7f18b1ffb640 1 -- 172.21.15.157:0/3323092879 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f18bc14e6b0 con 0x7f18bc0c58a0 2024-04-25T00:01:39.960 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.959+0000 7f18c3914640 1 -- 172.21.15.157:0/3323092879 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f18bc1504d0 con 0x7f18bc0c58a0 2024-04-25T00:01:39.961 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.961+0000 7f18b1ffb640 1 -- 172.21.15.157:0/3323092879 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) ==== 161+0+0 (secure 0 0 0) 0x7f18ac079ba0 con 0x7f18bc0c58a0 2024-04-25T00:01:39.961 INFO:teuthology.orchestra.run.smithi157.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T00:01:39.965 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.963+0000 7f18c3914640 1 -- 172.21.15.157:0/3323092879 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f18a006be30 msgr2=0x7f18a006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:39.965 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.963+0000 7f18c3914640 1 --2- 172.21.15.157:0/3323092879 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f18a006be30 0x7f18a006e2f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f18bc155590 tx=0x7f18b8007d50 comp rx=0 tx=0).stop 2024-04-25T00:01:39.965 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.963+0000 7f18c3914640 1 -- 172.21.15.157:0/3323092879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18bc0c58a0 msgr2=0x7f18bc154c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:39.965 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.963+0000 7f18c3914640 1 --2- 172.21.15.157:0/3323092879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18bc0c58a0 0x7f18bc154c70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f18ac0098c0 tx=0x7f18ac0046c0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.965 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.964+0000 7f18c3914640 1 -- 172.21.15.157:0/3323092879 shutdown_connections 2024-04-25T00:01:39.965 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.964+0000 7f18c3914640 1 --2- 172.21.15.157:0/3323092879 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f18a006be30 0x7f18a006e2f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.965 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.964+0000 7f18c3914640 1 --2- 172.21.15.157:0/3323092879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18bc0c71f0 0x7f18bc144a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.965 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.964+0000 7f18c3914640 1 --2- 172.21.15.157:0/3323092879 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18bc0c68d0 0x7f18bc144530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.965 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.964+0000 7f18c3914640 1 --2- 172.21.15.157:0/3323092879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18bc0c58a0 0x7f18bc154c70 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.965 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.964+0000 7f18c3914640 1 -- 172.21.15.157:0/3323092879 >> 172.21.15.157:0/3323092879 conn(0x7f18bc0caca0 msgr2=0x7f18bc0aa770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:39.966 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.966+0000 7f18c3914640 1 -- 172.21.15.157:0/3323092879 shutdown_connections 2024-04-25T00:01:39.967 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:39.966+0000 7f18c3914640 1 -- 172.21.15.157:0/3323092879 wait complete. 2024-04-25T00:01:39.978 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T00:01:40.078 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.077+0000 7fa529932640 1 Processor -- start 2024-04-25T00:01:40.078 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.077+0000 7fa529932640 1 -- start start 2024-04-25T00:01:40.080 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.078+0000 7fa529932640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa5240c3370 0x7fa524156390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:40.080 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.078+0000 7fa529932640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa524152cc0 0x7fa5240c3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:40.080 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.078+0000 7fa529932640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5240736b0 con 0x7fa5240a5c50 2024-04-25T00:01:40.080 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.078+0000 7fa529932640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fa524153130 con 0x7fa5240c3370 2024-04-25T00:01:40.080 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.078+0000 7fa529932640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa524155f00 con 0x7fa524152cc0 2024-04-25T00:01:40.080 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.078+0000 7fa51bfff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fa5240a5c50 0x7fa5240c12e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:50286/0 (socket says 172.21.15.157:50286) 2024-04-25T00:01:40.080 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.078+0000 7fa51bfff640 1 -- 172.21.15.157:0/517131289 learned_addr learned my addr 172.21.15.157:0/517131289 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-25T00:01:40.080 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.078+0000 7fa5237fe640 1 --2- 172.21.15.157:0/517131289 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa524152cc0 0x7fa5240c3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:40.080 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.078+0000 7fa522ffd640 1 --2- 172.21.15.157:0/517131289 >> v2:172.21.15.169:3300/0 conn(0x7fa5240c3370 0x7fa524156390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:40.081 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.078+0000 7fa5227fc640 1 -- 172.21.15.157:0/517131289 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1686243173 0 0) 0x7fa5240736b0 con 0x7fa5240a5c50 2024-04-25T00:01:40.081 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.078+0000 7fa5227fc640 1 -- 172.21.15.157:0/517131289 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa510003520 con 0x7fa5240a5c50 2024-04-25T00:01:40.081 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.078+0000 7fa5237fe640 1 -- 172.21.15.157:0/517131289 >> v2:172.21.15.169:3300/0 conn(0x7fa5240c3370 msgr2=0x7fa524156390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:40.081 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.078+0000 7fa5237fe640 1 --2- 172.21.15.157:0/517131289 >> v2:172.21.15.169:3300/0 conn(0x7fa5240c3370 0x7fa524156390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:40.081 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.078+0000 7fa5237fe640 1 -- 172.21.15.157:0/517131289 >> v1:172.21.15.157:6789/0 conn(0x7fa5240a5c50 legacy=0x7fa5240c12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:40.081 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.078+0000 7fa5237fe640 1 -- 172.21.15.157:0/517131289 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5240a60e0 con 0x7fa524152cc0 2024-04-25T00:01:40.081 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.078+0000 7fa522ffd640 1 --2- 172.21.15.157:0/517131289 >> v2:172.21.15.169:3300/0 conn(0x7fa5240c3370 0x7fa524156390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:01:40.081 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.079+0000 7fa5237fe640 1 --2- 172.21.15.157:0/517131289 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa524152cc0 0x7fa5240c3750 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa51c009d90 tx=0x7fa51c02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5726e64999178dbd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:40.081 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.079+0000 7fa5227fc640 1 -- 172.21.15.157:0/517131289 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51c007740 con 0x7fa524152cc0 2024-04-25T00:01:40.081 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.079+0000 7fa5227fc640 1 -- 172.21.15.157:0/517131289 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fa51c0078e0 con 0x7fa524152cc0 2024-04-25T00:01:40.081 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.079+0000 7fa5227fc640 1 -- 172.21.15.157:0/517131289 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51c007cd0 con 0x7fa524152cc0 2024-04-25T00:01:40.084 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.080+0000 7fa529932640 1 -- 172.21.15.157:0/517131289 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa524152cc0 msgr2=0x7fa5240c3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:40.084 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.080+0000 7fa529932640 1 --2- 172.21.15.157:0/517131289 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa524152cc0 0x7fa5240c3750 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa51c009d90 tx=0x7fa51c02a660 comp rx=0 tx=0).stop 2024-04-25T00:01:40.084 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.080+0000 7fa529932640 1 -- 172.21.15.157:0/517131289 shutdown_connections 2024-04-25T00:01:40.084 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.080+0000 7fa529932640 1 --2- 172.21.15.157:0/517131289 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa524152cc0 0x7fa5240c3750 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:40.085 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.080+0000 7fa529932640 1 --2- 172.21.15.157:0/517131289 >> v2:172.21.15.169:3300/0 conn(0x7fa5240c3370 0x7fa524156390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:40.085 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.080+0000 7fa529932640 1 -- 172.21.15.157:0/517131289 >> 172.21.15.157:0/517131289 conn(0x7fa5240c68e0 msgr2=0x7fa5240c6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:40.085 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.080+0000 7fa529932640 1 -- 172.21.15.157:0/517131289 shutdown_connections 2024-04-25T00:01:40.085 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.080+0000 7fa529932640 1 -- 172.21.15.157:0/517131289 wait complete. 2024-04-25T00:01:40.085 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.081+0000 7fa529932640 1 Processor -- start 2024-04-25T00:01:40.085 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.081+0000 7fa529932640 1 -- start start 2024-04-25T00:01:40.085 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.081+0000 7fa529932640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5240c3370 0x7fa5241e6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:40.085 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.081+0000 7fa529932640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa524152cc0 0x7fa524151030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:40.085 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.081+0000 7fa529932640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa524148080 0x7fa52414db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:40.085 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.081+0000 7fa529932640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa524155b90 con 0x7fa524148080 2024-04-25T00:01:40.085 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.081+0000 7fa529932640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa5241558b0 con 0x7fa524152cc0 2024-04-25T00:01:40.085 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.081+0000 7fa529932640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa5240c19f0 con 0x7fa5240c3370 2024-04-25T00:01:40.085 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.082+0000 7fa522ffd640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5240c3370 0x7fa5241e6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:40.086 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.082+0000 7fa51bfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa524152cc0 0x7fa524151030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:40.086 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.082+0000 7fa5237fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa524148080 0x7fa52414db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:40.086 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.082+0000 7fa522ffd640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5240c3370 0x7fa5241e6a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.157:51502/0 (socket says 172.21.15.157:51502) 2024-04-25T00:01:40.086 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.082+0000 7fa522ffd640 1 -- 172.21.15.157:0/3716794265 learned_addr learned my addr 172.21.15.157:0/3716794265 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:01:40.086 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.082+0000 7fa51bfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa524152cc0 0x7fa524151030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.157:43866/0 (socket says 172.21.15.157:43866) 2024-04-25T00:01:40.086 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.082+0000 7fa522ffd640 1 -- 172.21.15.157:0/3716794265 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa524152cc0 msgr2=0x7fa524151030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:40.086 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.082+0000 7fa522ffd640 1 --2- 172.21.15.157:0/3716794265 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa524152cc0 0x7fa524151030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:40.086 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.082+0000 7fa522ffd640 1 -- 172.21.15.157:0/3716794265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa524148080 msgr2=0x7fa52414db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:40.086 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.082+0000 7fa522ffd640 1 --2- 172.21.15.157:0/3716794265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa524148080 0x7fa52414db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:40.086 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.082+0000 7fa522ffd640 1 -- 172.21.15.157:0/3716794265 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5240bc190 con 0x7fa5240c3370 2024-04-25T00:01:40.086 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.082+0000 7fa522ffd640 1 --2- 172.21.15.157:0/3716794265 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5240c3370 0x7fa5241e6a30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa51400ebe0 tx=0x7fa51400c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:40.086 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.082+0000 7fa520ff9640 1 -- 172.21.15.157:0/3716794265 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa514017070 con 0x7fa5240c3370 2024-04-25T00:01:40.086 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.082+0000 7fa520ff9640 1 -- 172.21.15.157:0/3716794265 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fa51400cbd0 con 0x7fa5240c3370 2024-04-25T00:01:40.087 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.082+0000 7fa529932640 1 -- 172.21.15.157:0/3716794265 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5241eab50 con 0x7fa5240c3370 2024-04-25T00:01:40.087 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.082+0000 7fa520ff9640 1 -- 172.21.15.157:0/3716794265 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa514013470 con 0x7fa5240c3370 2024-04-25T00:01:40.087 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.082+0000 7fa529932640 1 -- 172.21.15.157:0/3716794265 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5241eae30 con 0x7fa5240c3370 2024-04-25T00:01:40.087 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.083+0000 7fa529932640 1 -- 172.21.15.157:0/3716794265 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5240681c0 con 0x7fa5240c3370 2024-04-25T00:01:40.087 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.084+0000 7fa520ff9640 1 -- 172.21.15.157:0/3716794265 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa51401a070 con 0x7fa5240c3370 2024-04-25T00:01:40.088 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.084+0000 7fa520ff9640 1 --2- 172.21.15.157:0/3716794265 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fa50406be30 0x7fa50406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:40.088 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.084+0000 7fa520ff9640 1 -- 172.21.15.157:0/3716794265 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(28..28 src has 1..28) ==== 6821+0+0 (secure 0 0 0) 0x7fa51408b170 con 0x7fa5240c3370 2024-04-25T00:01:40.088 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.084+0000 7fa51bfff640 1 --2- 172.21.15.157:0/3716794265 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fa50406be30 0x7fa50406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:40.088 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.085+0000 7fa51bfff640 1 --2- 172.21.15.157:0/3716794265 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fa50406be30 0x7fa50406e2f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa5240ac870 tx=0x7fa50c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:40.089 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.089+0000 7fa520ff9640 1 -- 172.21.15.157:0/3716794265 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa514059050 con 0x7fa5240c3370 2024-04-25T00:01:40.129 INFO:tasks.ceph.mds.b.smithi169.stdout:starting mds.b at 2024-04-25T00:01:40.302 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.301+0000 7fa529932640 1 -- 172.21.15.157:0/3716794265 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fa5240c4240 con 0x7fa5240c3370 2024-04-25T00:01:40.966 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.966+0000 7fa520ff9640 1 -- 172.21.15.157:0/3716794265 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7fa51405d350 con 0x7fa5240c3370 2024-04-25T00:01:40.969 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.968+0000 7fa529932640 1 -- 172.21.15.157:0/3716794265 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fa50406be30 msgr2=0x7fa50406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:40.969 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.968+0000 7fa529932640 1 --2- 172.21.15.157:0/3716794265 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fa50406be30 0x7fa50406e2f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa5240ac870 tx=0x7fa50c009040 comp rx=0 tx=0).stop 2024-04-25T00:01:40.969 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.968+0000 7fa529932640 1 -- 172.21.15.157:0/3716794265 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5240c3370 msgr2=0x7fa5241e6a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:40.969 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.968+0000 7fa529932640 1 --2- 172.21.15.157:0/3716794265 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5240c3370 0x7fa5241e6a30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa51400ebe0 tx=0x7fa51400c520 comp rx=0 tx=0).stop 2024-04-25T00:01:40.969 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.968+0000 7fa529932640 1 -- 172.21.15.157:0/3716794265 shutdown_connections 2024-04-25T00:01:40.969 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.968+0000 7fa529932640 1 --2- 172.21.15.157:0/3716794265 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fa50406be30 0x7fa50406e2f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:40.970 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.968+0000 7fa529932640 1 --2- 172.21.15.157:0/3716794265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa524148080 0x7fa52414db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:40.970 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.968+0000 7fa529932640 1 --2- 172.21.15.157:0/3716794265 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa524152cc0 0x7fa524151030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:40.970 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.968+0000 7fa529932640 1 --2- 172.21.15.157:0/3716794265 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5240c3370 0x7fa5241e6a30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:40.970 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.968+0000 7fa529932640 1 -- 172.21.15.157:0/3716794265 >> 172.21.15.157:0/3716794265 conn(0x7fa5240c68e0 msgr2=0x7fa524152550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:40.970 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.968+0000 7fa529932640 1 -- 172.21.15.157:0/3716794265 shutdown_connections 2024-04-25T00:01:40.970 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:40.968+0000 7fa529932640 1 -- 172.21.15.157:0/3716794265 wait complete. 2024-04-25T00:01:40.980 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T00:01:41.081 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.080+0000 7f2bc3dc8640 1 Processor -- start 2024-04-25T00:01:41.081 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.080+0000 7f2bc3dc8640 1 -- start start 2024-04-25T00:01:41.082 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.081+0000 7f2bc3dc8640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2bbc14f3a0 0x7f2bbc06d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:41.083 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.081+0000 7f2bc3dc8640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2bbc0b0870 0x7f2bbc0c7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:41.083 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.081+0000 7f2bc3dc8640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bbc0c75f0 con 0x7f2bbc0afd00 2024-04-25T00:01:41.083 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.081+0000 7f2bc3dc8640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f2bbc155ef0 con 0x7f2bbc0b0870 2024-04-25T00:01:41.083 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.081+0000 7f2bc3dc8640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2bbc1561d0 con 0x7f2bbc14f3a0 2024-04-25T00:01:41.083 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.081+0000 7f2bc2365640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2bbc0b0870 0x7f2bbc0c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:41.083 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.081+0000 7f2bc1b64640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2bbc14f3a0 0x7f2bbc06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:41.083 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.081+0000 7f2bc2365640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2bbc0b0870 0x7f2bbc0c7860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.157:43880/0 (socket says 172.21.15.157:43880) 2024-04-25T00:01:41.083 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.081+0000 7f2bc1363640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f2bbc0afd00 0x7f2bbc0b0160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.157:37282/0 (socket says 172.21.15.157:37282) 2024-04-25T00:01:41.083 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.081+0000 7f2bc2365640 1 -- 172.21.15.157:0/2019884781 learned_addr learned my addr 172.21.15.157:0/2019884781 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:01:41.083 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.082+0000 7f2bc2365640 1 -- 172.21.15.157:0/2019884781 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2bbc14f3a0 msgr2=0x7f2bbc06d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:41.083 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.082+0000 7f2bc2365640 1 --2- 172.21.15.157:0/2019884781 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2bbc14f3a0 0x7f2bbc06d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:41.083 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.082+0000 7f2bc2365640 1 -- 172.21.15.157:0/2019884781 >> v1:172.21.15.169:6789/0 conn(0x7f2bbc0afd00 legacy=0x7f2bbc0b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:41.083 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.082+0000 7f2bc2365640 1 -- 172.21.15.157:0/2019884781 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bbc0c57d0 con 0x7f2bbc0b0870 2024-04-25T00:01:41.084 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.082+0000 7f2bc0b62640 1 -- 172.21.15.157:0/2019884781 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2782649649 0 0) 0x7f2bbc0c75f0 con 0x7f2bbc0afd00 2024-04-25T00:01:41.085 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.082+0000 7f2bc2365640 1 --2- 172.21.15.157:0/2019884781 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bbc0b0870 0x7f2bbc0c7860 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2bb4009e50 tx=0x7f2bb402a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62c0d30637babf46 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:41.085 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.083+0000 7f2bc0b62640 1 -- 172.21.15.157:0/2019884781 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb4036070 con 0x7f2bbc0b0870 2024-04-25T00:01:41.085 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.083+0000 7f2bc0b62640 1 -- 172.21.15.157:0/2019884781 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f2bb4031600 con 0x7f2bbc0b0870 2024-04-25T00:01:41.085 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.083+0000 7f2bc0b62640 1 -- 172.21.15.157:0/2019884781 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb40319f0 con 0x7f2bbc0b0870 2024-04-25T00:01:41.085 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.083+0000 7f2bc3dc8640 1 -- 172.21.15.157:0/2019884781 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bbc0b0870 msgr2=0x7f2bbc0c7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:41.085 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.083+0000 7f2bc3dc8640 1 --2- 172.21.15.157:0/2019884781 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bbc0b0870 0x7f2bbc0c7860 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2bb4009e50 tx=0x7f2bb402a710 comp rx=0 tx=0).stop 2024-04-25T00:01:41.086 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.083+0000 7f2bc3dc8640 1 -- 172.21.15.157:0/2019884781 shutdown_connections 2024-04-25T00:01:41.086 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.083+0000 7f2bc3dc8640 1 --2- 172.21.15.157:0/2019884781 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bbc0b0870 0x7f2bbc0c7860 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:41.086 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.083+0000 7f2bc3dc8640 1 --2- 172.21.15.157:0/2019884781 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2bbc14f3a0 0x7f2bbc06d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:41.086 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.083+0000 7f2bc3dc8640 1 -- 172.21.15.157:0/2019884781 >> 172.21.15.157:0/2019884781 conn(0x7f2bbc066ea0 msgr2=0x7f2bbc0672b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:41.086 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.083+0000 7f2bc3dc8640 1 -- 172.21.15.157:0/2019884781 shutdown_connections 2024-04-25T00:01:41.086 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.083+0000 7f2bc3dc8640 1 -- 172.21.15.157:0/2019884781 wait complete. 2024-04-25T00:01:41.086 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.084+0000 7f2bc3dc8640 1 Processor -- start 2024-04-25T00:01:41.086 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.084+0000 7f2bc3dc8640 1 -- start start 2024-04-25T00:01:41.088 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.085+0000 7f2bc3dc8640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2bbc0afd00 0x7f2bbc0c24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:41.088 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.085+0000 7f2bc3dc8640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bbc14f3a0 0x7f2bbc1e8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:41.088 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.085+0000 7f2bc3dc8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bbc0acff0 0x7f2bbc1eb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:41.088 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.085+0000 7f2bc3dc8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2bbc14f780 con 0x7f2bbc0acff0 2024-04-25T00:01:41.088 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.085+0000 7f2bc3dc8640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2bbc0c5d90 con 0x7f2bbc14f3a0 2024-04-25T00:01:41.089 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.085+0000 7f2bc3dc8640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2bbc0c7da0 con 0x7f2bbc0afd00 2024-04-25T00:01:41.089 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.085+0000 7f2bc2365640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bbc0acff0 0x7f2bbc1eb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:41.089 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.085+0000 7f2bc1b64640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2bbc0afd00 0x7f2bbc0c24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:41.089 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.085+0000 7f2bc1363640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bbc14f3a0 0x7f2bbc1e8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:41.089 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.085+0000 7f2bc1b64640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2bbc0afd00 0x7f2bbc0c24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.157:51518/0 (socket says 172.21.15.157:51518) 2024-04-25T00:01:41.089 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.085+0000 7f2bc2365640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bbc0acff0 0x7f2bbc1eb3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44238/0 (socket says 172.21.15.157:44238) 2024-04-25T00:01:41.089 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.085+0000 7f2bc1b64640 1 -- 172.21.15.157:0/3124828957 learned_addr learned my addr 172.21.15.157:0/3124828957 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:01:41.089 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.086+0000 7f2bc1b64640 1 -- 172.21.15.157:0/3124828957 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bbc14f3a0 msgr2=0x7f2bbc1e8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:41.089 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.086+0000 7f2bc1b64640 1 --2- 172.21.15.157:0/3124828957 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bbc14f3a0 0x7f2bbc1e8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:41.089 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.086+0000 7f2bc1b64640 1 -- 172.21.15.157:0/3124828957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bbc0acff0 msgr2=0x7f2bbc1eb3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:41.089 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.086+0000 7f2bc1b64640 1 --2- 172.21.15.157:0/3124828957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bbc0acff0 0x7f2bbc1eb3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:41.089 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.086+0000 7f2bc1b64640 1 -- 172.21.15.157:0/3124828957 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bbc1eabf0 con 0x7f2bbc0afd00 2024-04-25T00:01:41.089 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.086+0000 7f2bc1363640 1 --2- 172.21.15.157:0/3124828957 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bbc14f3a0 0x7f2bbc1e8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:01:41.089 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.086+0000 7f2bc2365640 1 --2- 172.21.15.157:0/3124828957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bbc0acff0 0x7f2bbc1eb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:01:41.090 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.086+0000 7f2bc1b64640 1 --2- 172.21.15.157:0/3124828957 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2bbc0afd00 0x7f2bbc0c24e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2bac00e8a0 tx=0x7f2bac00ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:41.090 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.086+0000 7f2bb27fc640 1 -- 172.21.15.157:0/3124828957 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bac017070 con 0x7f2bbc0afd00 2024-04-25T00:01:41.090 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.086+0000 7f2bb27fc640 1 -- 172.21.15.157:0/3124828957 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f2bac00c940 con 0x7f2bbc0afd00 2024-04-25T00:01:41.090 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.086+0000 7f2bc3dc8640 1 -- 172.21.15.157:0/3124828957 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bbc0c1f90 con 0x7f2bbc0afd00 2024-04-25T00:01:41.090 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.086+0000 7f2bb27fc640 1 -- 172.21.15.157:0/3124828957 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bac00cd50 con 0x7f2bbc0afd00 2024-04-25T00:01:41.090 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.086+0000 7f2bc3dc8640 1 -- 172.21.15.157:0/3124828957 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bbc0c4bb0 con 0x7f2bbc0afd00 2024-04-25T00:01:41.090 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.087+0000 7f2bc3dc8640 1 -- 172.21.15.157:0/3124828957 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bbc0681c0 con 0x7f2bbc0afd00 2024-04-25T00:01:41.090 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.087+0000 7f2bb27fc640 1 -- 172.21.15.157:0/3124828957 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2bac01a070 con 0x7f2bbc0afd00 2024-04-25T00:01:41.090 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.088+0000 7f2bb27fc640 1 --2- 172.21.15.157:0/3124828957 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f2ba006be30 0x7f2ba006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:41.090 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.088+0000 7f2bb27fc640 1 -- 172.21.15.157:0/3124828957 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(28..28 src has 1..28) ==== 6821+0+0 (secure 0 0 0) 0x7f2bac08b450 con 0x7f2bbc0afd00 2024-04-25T00:01:41.090 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.088+0000 7f2bc1363640 1 --2- 172.21.15.157:0/3124828957 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f2ba006be30 0x7f2ba006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:41.091 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.088+0000 7f2bc1363640 1 --2- 172.21.15.157:0/3124828957 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f2ba006be30 0x7f2ba006e2f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc0c1db0 tx=0x7f2bb8007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:41.092 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.092+0000 7f2bb27fc640 1 -- 172.21.15.157:0/3124828957 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2bac059340 con 0x7f2bbc0afd00 2024-04-25T00:01:41.271 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.271+0000 7f2bc3dc8640 1 -- 172.21.15.157:0/3124828957 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2bbc06d310 con 0x7f2bbc0afd00 2024-04-25T00:01:41.978 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.977+0000 7f2bb27fc640 1 -- 172.21.15.157:0/3124828957 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v29) ==== 181+0+0 (secure 0 0 0) 0x7f2bac05d640 con 0x7f2bbc0afd00 2024-04-25T00:01:41.979 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_metadata' removed 2024-04-25T00:01:41.983 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.982+0000 7f2bc3dc8640 1 -- 172.21.15.157:0/3124828957 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f2ba006be30 msgr2=0x7f2ba006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:41.983 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.982+0000 7f2bc3dc8640 1 --2- 172.21.15.157:0/3124828957 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f2ba006be30 0x7f2ba006e2f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc0c1db0 tx=0x7f2bb8007d50 comp rx=0 tx=0).stop 2024-04-25T00:01:41.983 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.982+0000 7f2bc3dc8640 1 -- 172.21.15.157:0/3124828957 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2bbc0afd00 msgr2=0x7f2bbc0c24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:41.983 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.982+0000 7f2bc3dc8640 1 --2- 172.21.15.157:0/3124828957 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2bbc0afd00 0x7f2bbc0c24e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2bac00e8a0 tx=0x7f2bac00ed60 comp rx=0 tx=0).stop 2024-04-25T00:01:41.983 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.983+0000 7f2bc3dc8640 1 -- 172.21.15.157:0/3124828957 shutdown_connections 2024-04-25T00:01:41.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.983+0000 7f2bc3dc8640 1 --2- 172.21.15.157:0/3124828957 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f2ba006be30 0x7f2ba006e2f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:41.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.983+0000 7f2bc3dc8640 1 --2- 172.21.15.157:0/3124828957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bbc0acff0 0x7f2bbc1eb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:41.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.983+0000 7f2bc3dc8640 1 --2- 172.21.15.157:0/3124828957 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bbc14f3a0 0x7f2bbc1e8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:41.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.983+0000 7f2bc3dc8640 1 --2- 172.21.15.157:0/3124828957 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2bbc0afd00 0x7f2bbc0c24e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:41.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.983+0000 7f2bc3dc8640 1 -- 172.21.15.157:0/3124828957 >> 172.21.15.157:0/3124828957 conn(0x7f2bbc066ea0 msgr2=0x7f2bbc153b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:41.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.983+0000 7f2bc3dc8640 1 -- 172.21.15.157:0/3124828957 shutdown_connections 2024-04-25T00:01:41.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:41.983+0000 7f2bc3dc8640 1 -- 172.21.15.157:0/3124828957 wait complete. 2024-04-25T00:01:42.002 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T00:01:42.136 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.135+0000 7fc762295640 1 Processor -- start 2024-04-25T00:01:42.136 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.135+0000 7fc762295640 1 -- start start 2024-04-25T00:01:42.137 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.136+0000 7fc762295640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc75c1508c0 0x7fc75c0abb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:42.137 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.136+0000 7fc762295640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc75c0c2dc0 0x7fc75c1530d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:42.138 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.136+0000 7fc762295640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc75c06ea60 con 0x7fc75c0ac090 2024-04-25T00:01:42.138 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.136+0000 7fc762295640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fc75c0c64f0 con 0x7fc75c0c2dc0 2024-04-25T00:01:42.138 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.136+0000 7fc762295640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fc75c157090 con 0x7fc75c1508c0 2024-04-25T00:01:42.138 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.136+0000 7fc75b7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc75c0c2dc0 0x7fc75c1530d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:42.138 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.136+0000 7fc760832640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fc75c0ac090 0x7fc75c153610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:50298/0 (socket says 172.21.15.157:50298) 2024-04-25T00:01:42.138 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.136+0000 7fc760832640 1 -- 172.21.15.157:0/1807285344 learned_addr learned my addr 172.21.15.157:0/1807285344 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-25T00:01:42.138 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.136+0000 7fc75bfff640 1 --2- 172.21.15.157:0/1807285344 >> v2:172.21.15.169:3300/0 conn(0x7fc75c1508c0 0x7fc75c0abb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:42.138 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.136+0000 7fc75b7fe640 1 -- 172.21.15.157:0/1807285344 >> v2:172.21.15.169:3300/0 conn(0x7fc75c1508c0 msgr2=0x7fc75c0abb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:42.138 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.136+0000 7fc75b7fe640 1 --2- 172.21.15.157:0/1807285344 >> v2:172.21.15.169:3300/0 conn(0x7fc75c1508c0 0x7fc75c0abb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:42.138 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.136+0000 7fc75b7fe640 1 -- 172.21.15.157:0/1807285344 >> v1:172.21.15.157:6789/0 conn(0x7fc75c0ac090 legacy=0x7fc75c153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:42.138 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.136+0000 7fc75b7fe640 1 -- 172.21.15.157:0/1807285344 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc75c156760 con 0x7fc75c0c2dc0 2024-04-25T00:01:42.138 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.136+0000 7fc75affd640 1 -- 172.21.15.157:0/1807285344 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 77094236 0 0) 0x7fc75c06ea60 con 0x7fc75c0ac090 2024-04-25T00:01:42.139 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.137+0000 7fc75b7fe640 1 --2- 172.21.15.157:0/1807285344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc75c0c2dc0 0x7fc75c1530d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc75c0c41a0 tx=0x7fc74c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b7499479a2484a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:42.139 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.137+0000 7fc75affd640 1 -- 172.21.15.157:0/1807285344 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74c037070 con 0x7fc75c0c2dc0 2024-04-25T00:01:42.139 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.137+0000 7fc75affd640 1 -- 172.21.15.157:0/1807285344 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fc74c032440 con 0x7fc75c0c2dc0 2024-04-25T00:01:42.139 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.137+0000 7fc75affd640 1 -- 172.21.15.157:0/1807285344 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74c02ae60 con 0x7fc75c0c2dc0 2024-04-25T00:01:42.142 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.137+0000 7fc762295640 1 -- 172.21.15.157:0/1807285344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc75c0c2dc0 msgr2=0x7fc75c1530d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:42.142 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.137+0000 7fc762295640 1 --2- 172.21.15.157:0/1807285344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc75c0c2dc0 0x7fc75c1530d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc75c0c41a0 tx=0x7fc74c02a110 comp rx=0 tx=0).stop 2024-04-25T00:01:42.142 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.138+0000 7fc762295640 1 -- 172.21.15.157:0/1807285344 shutdown_connections 2024-04-25T00:01:42.142 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.138+0000 7fc762295640 1 --2- 172.21.15.157:0/1807285344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc75c0c2dc0 0x7fc75c1530d0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:42.142 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.138+0000 7fc762295640 1 --2- 172.21.15.157:0/1807285344 >> v2:172.21.15.169:3300/0 conn(0x7fc75c1508c0 0x7fc75c0abb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:42.142 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.138+0000 7fc762295640 1 -- 172.21.15.157:0/1807285344 >> 172.21.15.157:0/1807285344 conn(0x7fc75c0c3290 msgr2=0x7fc75c0c3670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:42.142 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.138+0000 7fc762295640 1 -- 172.21.15.157:0/1807285344 shutdown_connections 2024-04-25T00:01:42.142 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.138+0000 7fc762295640 1 -- 172.21.15.157:0/1807285344 wait complete. 2024-04-25T00:01:42.142 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.139+0000 7fc762295640 1 Processor -- start 2024-04-25T00:01:42.143 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.139+0000 7fc762295640 1 -- start start 2024-04-25T00:01:42.143 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.139+0000 7fc762295640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc75c0ac090 0x7fc75c073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:42.143 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.139+0000 7fc762295640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc75c0c2dc0 0x7fc75c071560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:42.143 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.140+0000 7fc762295640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc75c1508c0 0x7fc75c071aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:42.143 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.140+0000 7fc762295640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc75c157520 con 0x7fc75c0ac090 2024-04-25T00:01:42.143 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.140+0000 7fc762295640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc75c156d20 con 0x7fc75c0c2dc0 2024-04-25T00:01:42.143 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.140+0000 7fc762295640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc75c06e640 con 0x7fc75c1508c0 2024-04-25T00:01:42.143 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.140+0000 7fc75bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc75c0ac090 0x7fc75c073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:42.143 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.140+0000 7fc75bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc75c0ac090 0x7fc75c073040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44256/0 (socket says 172.21.15.157:44256) 2024-04-25T00:01:42.143 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.140+0000 7fc75bfff640 1 -- 172.21.15.157:0/1874425629 learned_addr learned my addr 172.21.15.157:0/1874425629 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:01:42.143 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.140+0000 7fc75b7fe640 1 --2- 172.21.15.157:0/1874425629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc75c0c2dc0 0x7fc75c071560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:42.143 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.140+0000 7fc760832640 1 --2- 172.21.15.157:0/1874425629 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc75c1508c0 0x7fc75c071aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:42.144 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.140+0000 7fc75bfff640 1 -- 172.21.15.157:0/1874425629 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc75c1508c0 msgr2=0x7fc75c071aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:42.144 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.140+0000 7fc75bfff640 1 --2- 172.21.15.157:0/1874425629 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc75c1508c0 0x7fc75c071aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:42.144 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.140+0000 7fc75bfff640 1 -- 172.21.15.157:0/1874425629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc75c0c2dc0 msgr2=0x7fc75c071560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:42.144 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.140+0000 7fc75bfff640 1 --2- 172.21.15.157:0/1874425629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc75c0c2dc0 0x7fc75c071560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:42.144 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.140+0000 7fc75bfff640 1 -- 172.21.15.157:0/1874425629 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc75c0707a0 con 0x7fc75c0ac090 2024-04-25T00:01:42.144 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.140+0000 7fc75bfff640 1 --2- 172.21.15.157:0/1874425629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc75c0ac090 0x7fc75c073040 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc75000e870 tx=0x7fc75000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:42.144 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.140+0000 7fc758ff9640 1 -- 172.21.15.157:0/1874425629 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc750017070 con 0x7fc75c0ac090 2024-04-25T00:01:42.144 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.140+0000 7fc758ff9640 1 -- 172.21.15.157:0/1874425629 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fc75000c940 con 0x7fc75c0ac090 2024-04-25T00:01:42.144 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.140+0000 7fc762295640 1 -- 172.21.15.157:0/1874425629 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc75c070a80 con 0x7fc75c0ac090 2024-04-25T00:01:42.144 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.140+0000 7fc758ff9640 1 -- 172.21.15.157:0/1874425629 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75000cd50 con 0x7fc75c0ac090 2024-04-25T00:01:42.144 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.140+0000 7fc762295640 1 -- 172.21.15.157:0/1874425629 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc75c073580 con 0x7fc75c0ac090 2024-04-25T00:01:42.144 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.141+0000 7fc762295640 1 -- 172.21.15.157:0/1874425629 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc75c067c70 con 0x7fc75c0ac090 2024-04-25T00:01:42.145 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.142+0000 7fc758ff9640 1 -- 172.21.15.157:0/1874425629 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc75001a070 con 0x7fc75c0ac090 2024-04-25T00:01:42.145 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.142+0000 7fc758ff9640 1 --2- 172.21.15.157:0/1874425629 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fc74806be30 0x7fc74806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:42.145 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.142+0000 7fc758ff9640 1 -- 172.21.15.157:0/1874425629 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6383+0+0 (secure 0 0 0) 0x7fc75008aeb0 con 0x7fc75c0ac090 2024-04-25T00:01:42.146 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.142+0000 7fc75b7fe640 1 --2- 172.21.15.157:0/1874425629 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fc74806be30 0x7fc74806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:42.146 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.143+0000 7fc75b7fe640 1 --2- 172.21.15.157:0/1874425629 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fc74806be30 0x7fc74806e2f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc74c034000 tx=0x7fc74c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:42.147 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.146+0000 7fc758ff9640 1 -- 172.21.15.157:0/1874425629 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc750059000 con 0x7fc75c0ac090 2024-04-25T00:01:42.343 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.342+0000 7fc762295640 1 -- 172.21.15.157:0/1874425629 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc75c0681c0 con 0x7fc75c0ac090 2024-04-25T00:01:42.974 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.973+0000 7fc758ff9640 1 -- 172.21.15.157:0/1874425629 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v30) ==== 169+0+0 (secure 0 0 0) 0x7fc75005d300 con 0x7fc75c0ac090 2024-04-25T00:01:42.974 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_data' removed 2024-04-25T00:01:42.979 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.978+0000 7fc762295640 1 -- 172.21.15.157:0/1874425629 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fc74806be30 msgr2=0x7fc74806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:42.979 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.978+0000 7fc762295640 1 --2- 172.21.15.157:0/1874425629 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fc74806be30 0x7fc74806e2f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc74c034000 tx=0x7fc74c02e000 comp rx=0 tx=0).stop 2024-04-25T00:01:42.979 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.978+0000 7fc762295640 1 -- 172.21.15.157:0/1874425629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc75c0ac090 msgr2=0x7fc75c073040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:42.979 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.978+0000 7fc762295640 1 --2- 172.21.15.157:0/1874425629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc75c0ac090 0x7fc75c073040 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc75000e870 tx=0x7fc75000ed30 comp rx=0 tx=0).stop 2024-04-25T00:01:42.979 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.979+0000 7fc762295640 1 -- 172.21.15.157:0/1874425629 shutdown_connections 2024-04-25T00:01:42.979 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.979+0000 7fc762295640 1 --2- 172.21.15.157:0/1874425629 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7fc74806be30 0x7fc74806e2f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:42.980 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.979+0000 7fc762295640 1 --2- 172.21.15.157:0/1874425629 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc75c1508c0 0x7fc75c071aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:42.980 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.979+0000 7fc762295640 1 --2- 172.21.15.157:0/1874425629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc75c0c2dc0 0x7fc75c071560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:42.980 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.979+0000 7fc762295640 1 --2- 172.21.15.157:0/1874425629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc75c0ac090 0x7fc75c073040 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:42.980 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.979+0000 7fc762295640 1 -- 172.21.15.157:0/1874425629 >> 172.21.15.157:0/1874425629 conn(0x7fc75c0c3290 msgr2=0x7fc75c0c43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:42.980 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.979+0000 7fc762295640 1 -- 172.21.15.157:0/1874425629 shutdown_connections 2024-04-25T00:01:42.980 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:42.979+0000 7fc762295640 1 -- 172.21.15.157:0/1874425629 wait complete. 2024-04-25T00:01:42.997 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-25T00:01:43.120 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.120+0000 7f1050bfb640 1 Processor -- start 2024-04-25T00:01:43.120 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.120+0000 7f1050bfb640 1 -- start start 2024-04-25T00:01:43.122 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.120+0000 7f1050bfb640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f104c14f3a0 0x7f104c06d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:43.122 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.120+0000 7f1050bfb640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f104c0b0870 0x7f104c0c7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:43.122 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.120+0000 7f1050bfb640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f104c0c75f0 con 0x7f104c0afd00 2024-04-25T00:01:43.122 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.120+0000 7f1050bfb640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f104c155ef0 con 0x7f104c0b0870 2024-04-25T00:01:43.122 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.120+0000 7f1050bfb640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f104c1561d0 con 0x7f104c14f3a0 2024-04-25T00:01:43.122 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.121+0000 7f1049d9b640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f104c0afd00 0x7f104c0b0160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:50306/0 (socket says 172.21.15.157:50306) 2024-04-25T00:01:43.122 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.121+0000 7f1049d9b640 1 -- 172.21.15.157:0/363882570 learned_addr learned my addr 172.21.15.157:0/363882570 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-25T00:01:43.122 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.121+0000 7f104a59c640 1 --2- 172.21.15.157:0/363882570 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f104c14f3a0 0x7f104c06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:43.122 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.121+0000 7f104ad9d640 1 --2- 172.21.15.157:0/363882570 >> v2:172.21.15.169:3300/0 conn(0x7f104c0b0870 0x7f104c0c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:43.122 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.121+0000 7f104959a640 1 -- 172.21.15.157:0/363882570 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1572588840 0 0) 0x7f104c0c75f0 con 0x7f104c0afd00 2024-04-25T00:01:43.123 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.121+0000 7f104959a640 1 -- 172.21.15.157:0/363882570 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1038003520 con 0x7f104c0afd00 2024-04-25T00:01:43.123 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.121+0000 7f104ad9d640 1 -- 172.21.15.157:0/363882570 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f104c14f3a0 msgr2=0x7f104c06d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:43.123 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.121+0000 7f104ad9d640 1 --2- 172.21.15.157:0/363882570 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f104c14f3a0 0x7f104c06d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:43.123 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.121+0000 7f104ad9d640 1 -- 172.21.15.157:0/363882570 >> v1:172.21.15.157:6789/0 conn(0x7f104c0afd00 legacy=0x7f104c0b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:43.123 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.121+0000 7f104ad9d640 1 -- 172.21.15.157:0/363882570 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f104c0c57d0 con 0x7f104c0b0870 2024-04-25T00:01:43.123 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.121+0000 7f104a59c640 1 --2- 172.21.15.157:0/363882570 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f104c14f3a0 0x7f104c06d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:01:43.124 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.122+0000 7f104ad9d640 1 --2- 172.21.15.157:0/363882570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f104c0b0870 0x7f104c0c7860 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1044009d90 tx=0x7f104402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44add39c4d894c23 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:43.124 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.122+0000 7f104959a640 1 -- 172.21.15.157:0/363882570 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1044007740 con 0x7f104c0b0870 2024-04-25T00:01:43.124 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.122+0000 7f104959a640 1 -- 172.21.15.157:0/363882570 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f10440078e0 con 0x7f104c0b0870 2024-04-25T00:01:43.124 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.122+0000 7f104959a640 1 -- 172.21.15.157:0/363882570 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1044007cd0 con 0x7f104c0b0870 2024-04-25T00:01:43.125 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.122+0000 7f1050bfb640 1 -- 172.21.15.157:0/363882570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f104c0b0870 msgr2=0x7f104c0c7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:43.125 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.122+0000 7f1050bfb640 1 --2- 172.21.15.157:0/363882570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f104c0b0870 0x7f104c0c7860 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1044009d90 tx=0x7f104402a660 comp rx=0 tx=0).stop 2024-04-25T00:01:43.125 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.123+0000 7f1050bfb640 1 -- 172.21.15.157:0/363882570 shutdown_connections 2024-04-25T00:01:43.125 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.123+0000 7f1050bfb640 1 --2- 172.21.15.157:0/363882570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f104c0b0870 0x7f104c0c7860 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:43.125 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.123+0000 7f1050bfb640 1 --2- 172.21.15.157:0/363882570 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f104c14f3a0 0x7f104c06d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:43.125 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.123+0000 7f1050bfb640 1 -- 172.21.15.157:0/363882570 >> 172.21.15.157:0/363882570 conn(0x7f104c066ea0 msgr2=0x7f104c0672b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:43.125 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.123+0000 7f1050bfb640 1 -- 172.21.15.157:0/363882570 shutdown_connections 2024-04-25T00:01:43.125 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.123+0000 7f1050bfb640 1 -- 172.21.15.157:0/363882570 wait complete. 2024-04-25T00:01:43.125 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.124+0000 7f1050bfb640 1 Processor -- start 2024-04-25T00:01:43.125 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.124+0000 7f1050bfb640 1 -- start start 2024-04-25T00:01:43.128 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.125+0000 7f1050bfb640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f104c0afd00 0x7f104c0c4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:43.128 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.125+0000 7f1050bfb640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f104c14f3a0 0x7f104c1ee6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:43.128 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.125+0000 7f1050bfb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f104c0c4950 0x7f104c14ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:43.128 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.125+0000 7f1050bfb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f104c14f780 con 0x7f104c0c4950 2024-04-25T00:01:43.128 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.125+0000 7f1050bfb640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f104c0c5d90 con 0x7f104c0afd00 2024-04-25T00:01:43.128 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.125+0000 7f1050bfb640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f104c0c7da0 con 0x7f104c14f3a0 2024-04-25T00:01:43.128 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.125+0000 7f104ad9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f104c0c4950 0x7f104c14ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:43.128 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.125+0000 7f104ad9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f104c0c4950 0x7f104c14ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44262/0 (socket says 172.21.15.157:44262) 2024-04-25T00:01:43.128 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.125+0000 7f104ad9d640 1 -- 172.21.15.157:0/1064734876 learned_addr learned my addr 172.21.15.157:0/1064734876 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:01:43.129 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.125+0000 7f104a59c640 1 --2- 172.21.15.157:0/1064734876 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f104c0afd00 0x7f104c0c4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:43.129 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.125+0000 7f1049d9b640 1 --2- 172.21.15.157:0/1064734876 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f104c14f3a0 0x7f104c1ee6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:43.129 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.125+0000 7f104ad9d640 1 -- 172.21.15.157:0/1064734876 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f104c14f3a0 msgr2=0x7f104c1ee6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:43.129 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.125+0000 7f104ad9d640 1 --2- 172.21.15.157:0/1064734876 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f104c14f3a0 0x7f104c1ee6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:43.129 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.125+0000 7f104ad9d640 1 -- 172.21.15.157:0/1064734876 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f104c0afd00 msgr2=0x7f104c0c4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:43.129 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.125+0000 7f104ad9d640 1 --2- 172.21.15.157:0/1064734876 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f104c0afd00 0x7f104c0c4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:43.129 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.125+0000 7f104ad9d640 1 -- 172.21.15.157:0/1064734876 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f104c1ef150 con 0x7f104c0c4950 2024-04-25T00:01:43.129 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.125+0000 7f104ad9d640 1 --2- 172.21.15.157:0/1064734876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f104c0c4950 0x7f104c14ac10 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1044009d90 tx=0x7f1044004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:43.129 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.125+0000 7f1042ffd640 1 -- 172.21.15.157:0/1064734876 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1044040070 con 0x7f104c0c4950 2024-04-25T00:01:43.129 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.125+0000 7f1042ffd640 1 -- 172.21.15.157:0/1064734876 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f1044031bc0 con 0x7f104c0c4950 2024-04-25T00:01:43.129 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.125+0000 7f1050bfb640 1 -- 172.21.15.157:0/1064734876 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f104c1ef430 con 0x7f104c0c4950 2024-04-25T00:01:43.129 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.125+0000 7f1042ffd640 1 -- 172.21.15.157:0/1064734876 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1044009450 con 0x7f104c0c4950 2024-04-25T00:01:43.129 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.125+0000 7f1050bfb640 1 -- 172.21.15.157:0/1064734876 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f104c1592a0 con 0x7f104c0c4950 2024-04-25T00:01:43.130 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.126+0000 7f1050bfb640 1 -- 172.21.15.157:0/1064734876 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f104c06c4c0 con 0x7f104c0c4950 2024-04-25T00:01:43.130 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.127+0000 7f1042ffd640 1 -- 172.21.15.157:0/1064734876 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1044039040 con 0x7f104c0c4950 2024-04-25T00:01:43.130 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.127+0000 7f1042ffd640 1 --2- 172.21.15.157:0/1064734876 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f102c06be60 0x7f102c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:43.130 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.127+0000 7f1042ffd640 1 -- 172.21.15.157:0/1064734876 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5853+0+0 (secure 0 0 0) 0x7f10440ac580 con 0x7f104c0c4950 2024-04-25T00:01:43.130 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.127+0000 7f104a59c640 1 --2- 172.21.15.157:0/1064734876 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f102c06be60 0x7f102c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:43.130 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.128+0000 7f104a59c640 1 --2- 172.21.15.157:0/1064734876 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f102c06be60 0x7f102c06e320 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f103c009560 tx=0x7f103c009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:43.132 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.131+0000 7f1042ffd640 1 -- 172.21.15.157:0/1064734876 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f104c06c4c0 con 0x7f104c0c4950 2024-04-25T00:01:43.308 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.308+0000 7f1050bfb640 1 -- 172.21.15.157:0/1064734876 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f104c0ad7f0 con 0x7f104c0c4950 2024-04-25T00:01:43.978 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.978+0000 7f1042ffd640 1 -- 172.21.15.157:0/1064734876 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v31) ==== 178+0+0 (secure 0 0 0) 0x7f104407a840 con 0x7f104c0c4950 2024-04-25T00:01:43.978 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_data_ec' removed 2024-04-25T00:01:43.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.982+0000 7f1050bfb640 1 -- 172.21.15.157:0/1064734876 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f102c06be60 msgr2=0x7f102c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:43.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.982+0000 7f1050bfb640 1 --2- 172.21.15.157:0/1064734876 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f102c06be60 0x7f102c06e320 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f103c009560 tx=0x7f103c009190 comp rx=0 tx=0).stop 2024-04-25T00:01:43.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.983+0000 7f1050bfb640 1 -- 172.21.15.157:0/1064734876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f104c0c4950 msgr2=0x7f104c14ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:43.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.983+0000 7f1050bfb640 1 --2- 172.21.15.157:0/1064734876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f104c0c4950 0x7f104c14ac10 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1044009d90 tx=0x7f1044004a40 comp rx=0 tx=0).stop 2024-04-25T00:01:43.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.983+0000 7f1050bfb640 1 -- 172.21.15.157:0/1064734876 shutdown_connections 2024-04-25T00:01:43.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.983+0000 7f1050bfb640 1 --2- 172.21.15.157:0/1064734876 >> [v2:172.21.15.169:6832/662378289,v1:172.21.15.169:6833/662378289] conn(0x7f102c06be60 0x7f102c06e320 secure :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f103c009560 tx=0x7f103c009190 comp rx=0 tx=0).stop 2024-04-25T00:01:43.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.983+0000 7f1050bfb640 1 --2- 172.21.15.157:0/1064734876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f104c0c4950 0x7f104c14ac10 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:43.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.983+0000 7f1050bfb640 1 --2- 172.21.15.157:0/1064734876 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f104c14f3a0 0x7f104c1ee6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:43.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.983+0000 7f1050bfb640 1 --2- 172.21.15.157:0/1064734876 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f104c0afd00 0x7f104c0c4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:43.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.983+0000 7f1050bfb640 1 -- 172.21.15.157:0/1064734876 >> 172.21.15.157:0/1064734876 conn(0x7f104c066ea0 msgr2=0x7f104c067280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:43.986 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.986+0000 7f1050bfb640 1 -- 172.21.15.157:0/1064734876 shutdown_connections 2024-04-25T00:01:43.989 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:01:43.989+0000 7f1050bfb640 1 -- 172.21.15.157:0/1064734876 wait complete. 2024-04-25T00:01:44.006 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-25T00:01:44.007 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T00:01:44.007 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:01:44.007 INFO:tasks.ceph.mds.a.smithi157.stderr:2024-04-25T00:01:44.007+0000 7f9df297c640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-mds -f --cluster ceph -i a (PID: 38136) UID: 0 2024-04-25T00:01:44.008 INFO:tasks.ceph.mds.a.smithi157.stderr:2024-04-25T00:01:44.007+0000 7f9df297c640 -1 mds.a *** got signal Terminated *** 2024-04-25T00:01:50.108 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T00:01:50.109 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T00:01:50.109 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:01:50.109 INFO:tasks.ceph.mds.c.smithi157.stderr:2024-04-25T00:01:50.110+0000 7f8b366ec640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-mds -f --cluster ceph -i c (PID: 38137) UID: 0 2024-04-25T00:01:50.110 INFO:tasks.ceph.mds.c.smithi157.stderr:2024-04-25T00:01:50.110+0000 7f8b366ec640 -1 mds.c *** got signal Terminated *** 2024-04-25T00:01:50.425 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-04-25T00:01:56.211 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T00:01:56.211 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T00:01:56.211 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:01:56.212 INFO:tasks.ceph.mds.b.smithi169.stderr:2024-04-25T00:01:56.212+0000 7f148ae4b640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-mds -f --cluster ceph -i b (PID: 38459) UID: 0 2024-04-25T00:01:56.212 INFO:tasks.ceph.mds.b.smithi169.stderr:2024-04-25T00:01:56.212+0000 7f148ae4b640 -1 mds.b *** got signal Terminated *** 2024-04-25T00:01:56.939 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-04-25T00:02:02.313 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T00:02:02.313 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T00:02:02.314 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:02:02.315 INFO:tasks.ceph.mds.d.smithi169.stderr:2024-04-25T00:02:02.315+0000 7fcbfaa77640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-mds -f --cluster ceph -i d (PID: 38460) UID: 0 2024-04-25T00:02:02.315 INFO:tasks.ceph.mds.d.smithi169.stderr:2024-04-25T00:02:02.315+0000 7fcbfaa77640 -1 mds.d *** got signal Terminated *** 2024-04-25T00:02:08.415 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T00:02:08.415 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-25T00:02:08.416 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-25T00:02:08.416 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:02:08.417 INFO:tasks.ceph.osd.0.smithi157.stderr:2024-04-25T00:02:08.416+0000 7f9af40bf640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-osd -f --cluster ceph -i 0 (PID: 34499) UID: 0 2024-04-25T00:02:08.417 INFO:tasks.ceph.osd.0.smithi157.stderr:2024-04-25T00:02:08.416+0000 7f9af40bf640 -1 osd.0 35 *** Got signal Terminated *** 2024-04-25T00:02:08.417 INFO:tasks.ceph.osd.0.smithi157.stderr:2024-04-25T00:02:08.416+0000 7f9af40bf640 -1 osd.0 35 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:02:13.427 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:02:13.430 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:02:13.451 INFO:teuthology.orchestra.run.smithi157.stderr:error: state file /var/lib/logrotate/logrotate.status is already locked 2024-04-25T00:02:13.451 INFO:teuthology.orchestra.run.smithi157.stderr:logrotate does not support parallel execution on the same set of logfiles. 2024-04-25T00:02:13.452 DEBUG:teuthology.orchestra.run:got remote process result: 3 2024-04-25T00:02:13.453 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "src/gevent/greenlet.py", line 908, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks/ceph.py", line 189, in invoke_logrotate run.wait(procs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi157 with status 3: 'sudo logrotate /etc/logrotate.d/ceph-test.conf' 2024-04-25T00:02:14.517 INFO:tasks.ceph.osd.0:Stopped 2024-04-25T00:02:14.518 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-25T00:02:14.518 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:02:14.519 INFO:tasks.ceph.osd.1.smithi157.stderr:2024-04-25T00:02:14.518+0000 7f33b57c8640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-osd -f --cluster ceph -i 1 (PID: 34497) UID: 0 2024-04-25T00:02:14.519 INFO:tasks.ceph.osd.1.smithi157.stderr:2024-04-25T00:02:14.518+0000 7f33b57c8640 -1 osd.1 37 *** Got signal Terminated *** 2024-04-25T00:02:14.519 INFO:tasks.ceph.osd.1.smithi157.stderr:2024-04-25T00:02:14.518+0000 7f33b57c8640 -1 osd.1 37 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:02:20.619 INFO:tasks.ceph.osd.1:Stopped 2024-04-25T00:02:20.620 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-25T00:02:20.620 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:02:20.621 INFO:tasks.ceph.osd.2.smithi157.stderr:2024-04-25T00:02:20.620+0000 7fbb527f4640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-osd -f --cluster ceph -i 2 (PID: 34502) UID: 0 2024-04-25T00:02:20.621 INFO:tasks.ceph.osd.2.smithi157.stderr:2024-04-25T00:02:20.620+0000 7fbb527f4640 -1 osd.2 38 *** Got signal Terminated *** 2024-04-25T00:02:20.621 INFO:tasks.ceph.osd.2.smithi157.stderr:2024-04-25T00:02:20.620+0000 7fbb527f4640 -1 osd.2 38 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:02:26.721 INFO:tasks.ceph.osd.2:Stopped 2024-04-25T00:02:26.722 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-25T00:02:26.722 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:02:26.723 INFO:tasks.ceph.osd.3.smithi157.stderr:2024-04-25T00:02:26.722+0000 7fc5e948c640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-osd -f --cluster ceph -i 3 (PID: 34504) UID: 0 2024-04-25T00:02:26.723 INFO:tasks.ceph.osd.3.smithi157.stderr:2024-04-25T00:02:26.722+0000 7fc5e948c640 -1 osd.3 39 *** Got signal Terminated *** 2024-04-25T00:02:26.723 INFO:tasks.ceph.osd.3.smithi157.stderr:2024-04-25T00:02:26.722+0000 7fc5e948c640 -1 osd.3 39 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:02:32.823 INFO:tasks.ceph.osd.3:Stopped 2024-04-25T00:02:32.824 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-25T00:02:32.824 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:02:32.825 INFO:tasks.ceph.osd.4.smithi169.stderr:2024-04-25T00:02:32.825+0000 7f664986e640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-osd -f --cluster ceph -i 4 (PID: 34744) UID: 0 2024-04-25T00:02:32.825 INFO:tasks.ceph.osd.4.smithi169.stderr:2024-04-25T00:02:32.825+0000 7f664986e640 -1 osd.4 40 *** Got signal Terminated *** 2024-04-25T00:02:32.825 INFO:tasks.ceph.osd.4.smithi169.stderr:2024-04-25T00:02:32.825+0000 7f664986e640 -1 osd.4 40 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:02:38.926 INFO:tasks.ceph.osd.4:Stopped 2024-04-25T00:02:38.926 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-25T00:02:38.926 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:02:38.927 INFO:tasks.ceph.osd.5.smithi169.stderr:2024-04-25T00:02:38.926+0000 7f96be3b5640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-osd -f --cluster ceph -i 5 (PID: 34745) UID: 0 2024-04-25T00:02:38.928 INFO:tasks.ceph.osd.5.smithi169.stderr:2024-04-25T00:02:38.926+0000 7f96be3b5640 -1 osd.5 41 *** Got signal Terminated *** 2024-04-25T00:02:38.928 INFO:tasks.ceph.osd.5.smithi169.stderr:2024-04-25T00:02:38.926+0000 7f96be3b5640 -1 osd.5 41 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:02:39.705 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2024-04-25T00:02:45.028 INFO:tasks.ceph.osd.5:Stopped 2024-04-25T00:02:45.029 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-25T00:02:45.029 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:02:45.030 INFO:tasks.ceph.osd.6.smithi169.stderr:2024-04-25T00:02:45.030+0000 7f3ecc795640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-osd -f --cluster ceph -i 6 (PID: 34750) UID: 0 2024-04-25T00:02:45.030 INFO:tasks.ceph.osd.6.smithi169.stderr:2024-04-25T00:02:45.030+0000 7f3ecc795640 -1 osd.6 42 *** Got signal Terminated *** 2024-04-25T00:02:45.030 INFO:tasks.ceph.osd.6.smithi169.stderr:2024-04-25T00:02:45.030+0000 7f3ecc795640 -1 osd.6 42 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:02:45.411 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-04-25T00:02:45.411 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2024-04-25T00:02:51.016 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-04-25T00:02:51.017 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-04-25T00:02:51.131 INFO:tasks.ceph.osd.6:Stopped 2024-04-25T00:02:51.131 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-25T00:02:51.131 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:02:51.132 INFO:tasks.ceph.osd.7.smithi169.stderr:2024-04-25T00:02:51.132+0000 7fdc58eaa640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-osd -f --cluster ceph -i 7 (PID: 34752) UID: 0 2024-04-25T00:02:51.132 INFO:tasks.ceph.osd.7.smithi169.stderr:2024-04-25T00:02:51.132+0000 7fdc58eaa640 -1 osd.7 42 *** Got signal Terminated *** 2024-04-25T00:02:51.132 INFO:tasks.ceph.osd.7.smithi169.stderr:2024-04-25T00:02:51.132+0000 7fdc58eaa640 -1 osd.7 42 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:02:56.522 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-04-25T00:02:56.522 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-04-25T00:02:57.233 INFO:tasks.ceph.osd.7:Stopped 2024-04-25T00:02:57.233 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-25T00:02:57.233 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-25T00:02:57.234 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:02:57.267 INFO:tasks.ceph.mgr.y:Stopped 2024-04-25T00:02:57.267 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-25T00:02:57.267 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:02:57.268 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-25T00:02:57.268+0000 7f5900af7640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-mgr -f --cluster ceph -i x (PID: 34169) UID: 0 2024-04-25T00:02:57.300 INFO:tasks.ceph.mgr.x:Stopped 2024-04-25T00:02:57.300 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-25T00:02:57.300 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-25T00:02:57.300 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:02:57.308 INFO:tasks.ceph.mon.a.smithi157.stderr:2024-04-25T00:02:57.309+0000 7fe7e13e2640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-mon -f --cluster ceph -i a (PID: 34222) UID: 0 2024-04-25T00:02:57.309 INFO:tasks.ceph.mon.a.smithi157.stderr:2024-04-25T00:02:57.309+0000 7fe7e13e2640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-25T00:02:57.350 INFO:tasks.ceph.mon.a:Stopped 2024-04-25T00:02:57.350 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-25T00:02:57.351 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:02:57.351 INFO:tasks.ceph.mon.b.smithi169.stderr:2024-04-25T00:02:57.351+0000 7f4c2e856640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-mon -f --cluster ceph -i b (PID: 34164) UID: 0 2024-04-25T00:02:57.351 INFO:tasks.ceph.mon.b.smithi169.stderr:2024-04-25T00:02:57.351+0000 7f4c2e856640 -1 mon.b@1(peon) e1 *** Got Signal Terminated *** 2024-04-25T00:02:57.395 INFO:tasks.ceph.mon.b:Stopped 2024-04-25T00:02:57.396 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-25T00:02:57.396 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:02:57.437 INFO:tasks.ceph.mon.c.smithi169.stderr:2024-04-25T00:02:57.437+0000 7fa1594f6640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-mon -f --cluster ceph -i c (PID: 34165) UID: 0 2024-04-25T00:02:57.437 INFO:tasks.ceph.mon.c.smithi169.stderr:2024-04-25T00:02:57.437+0000 7fa1594f6640 -1 mon.c@2(peon) e1 *** Got Signal Terminated *** 2024-04-25T00:03:01.523 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2024-04-25T00:03:01.524 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2024-04-25T00:03:03.498 INFO:tasks.ceph.mon.c:Stopped 2024-04-25T00:03:03.498 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-25T00:03:03.499 DEBUG:teuthology.orchestra.run.smithi157:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-25T00:03:03.610 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi157.front.sepia.ceph.com 2024-04-25T00:03:03.611 DEBUG:teuthology.orchestra.run.smithi157:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-25T00:03:03.736 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi157.front.sepia.ceph.com 2024-04-25T00:03:03.736 DEBUG:teuthology.orchestra.run.smithi157:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-25T00:03:03.852 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi157.front.sepia.ceph.com 2024-04-25T00:03:03.852 DEBUG:teuthology.orchestra.run.smithi157:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-25T00:03:03.944 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi157.front.sepia.ceph.com 2024-04-25T00:03:03.944 DEBUG:teuthology.orchestra.run.smithi157:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-25T00:03:04.025 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi169.front.sepia.ceph.com 2024-04-25T00:03:04.025 DEBUG:teuthology.orchestra.run.smithi169:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-25T00:03:04.260 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi169.front.sepia.ceph.com 2024-04-25T00:03:04.261 DEBUG:teuthology.orchestra.run.smithi169:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-25T00:03:04.344 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi169.front.sepia.ceph.com 2024-04-25T00:03:04.344 DEBUG:teuthology.orchestra.run.smithi169:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-25T00:03:04.443 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi169.front.sepia.ceph.com 2024-04-25T00:03:04.443 DEBUG:teuthology.orchestra.run.smithi169:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-25T00:03:04.515 INFO:tasks.ceph:Archiving mon data... 2024-04-25T00:03:04.517 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672205/data/mon.a.tgz 2024-04-25T00:03:04.517 DEBUG:teuthology.orchestra.run.smithi157:> mktemp 2024-04-25T00:03:04.533 INFO:teuthology.orchestra.run.smithi157.stdout:/tmp/tmp.i2lbl0ffXz 2024-04-25T00:03:04.534 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.i2lbl0ffXz 2024-04-25T00:03:04.688 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 0666 /tmp/tmp.i2lbl0ffXz 2024-04-25T00:03:04.789 DEBUG:teuthology.orchestra.remote:smithi157:/tmp/tmp.i2lbl0ffXz is 336KB 2024-04-25T00:03:04.885 DEBUG:teuthology.orchestra.run.smithi157:> rm -fr /tmp/tmp.i2lbl0ffXz 2024-04-25T00:03:04.900 DEBUG:teuthology.misc:Transferring archived files from smithi169:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672205/data/mon.b.tgz 2024-04-25T00:03:04.900 DEBUG:teuthology.orchestra.run.smithi169:> mktemp 2024-04-25T00:03:04.916 INFO:teuthology.orchestra.run.smithi169.stdout:/tmp/tmp.xUn5rjr0vs 2024-04-25T00:03:04.916 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.xUn5rjr0vs 2024-04-25T00:03:05.062 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 0666 /tmp/tmp.xUn5rjr0vs 2024-04-25T00:03:05.160 DEBUG:teuthology.orchestra.remote:smithi169:/tmp/tmp.xUn5rjr0vs is 344KB 2024-04-25T00:03:05.259 DEBUG:teuthology.orchestra.run.smithi169:> rm -fr /tmp/tmp.xUn5rjr0vs 2024-04-25T00:03:05.274 DEBUG:teuthology.misc:Transferring archived files from smithi169:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672205/data/mon.c.tgz 2024-04-25T00:03:05.274 DEBUG:teuthology.orchestra.run.smithi169:> mktemp 2024-04-25T00:03:05.328 INFO:teuthology.orchestra.run.smithi169.stdout:/tmp/tmp.vLqqGu7FEk 2024-04-25T00:03:05.329 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.vLqqGu7FEk 2024-04-25T00:03:05.452 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 0666 /tmp/tmp.vLqqGu7FEk 2024-04-25T00:03:05.534 DEBUG:teuthology.orchestra.remote:smithi169:/tmp/tmp.vLqqGu7FEk is 344KB 2024-04-25T00:03:05.636 DEBUG:teuthology.orchestra.run.smithi169:> rm -fr /tmp/tmp.vLqqGu7FEk 2024-04-25T00:03:05.650 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-25T00:03:05.650 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T00:03:05.653 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T00:03:05.730 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T00:03:05.768 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T00:03:05.769 INFO:tasks.ceph:Archiving crash dumps... 2024-04-25T00:03:05.771 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672205/remote/smithi157/crash 2024-04-25T00:03:05.772 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T00:03:05.800 DEBUG:teuthology.misc:Transferring archived files from smithi169:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672205/remote/smithi169/crash 2024-04-25T00:03:05.802 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T00:03:05.832 INFO:tasks.ceph:Shutting down logrotate 2024-04-25T00:03:05.833 ERROR:teuthology.run_tasks:Manager failed: ceph Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks/ceph.py", line 1935, in task mon0_remote.run( File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks/ceph.py", line 257, in ceph_log logrotater.end() File "/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks/ceph.py", line 222, in end self.thread.get() File "src/gevent/greenlet.py", line 805, in gevent._gevent_cgreenlet.Greenlet.get File "src/gevent/greenlet.py", line 373, in gevent._gevent_cgreenlet.Greenlet._raise_exception File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/lib/python3.8/site-packages/gevent/_compat.py", line 49, in reraise raise value.with_traceback(tb) File "src/gevent/greenlet.py", line 908, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks/ceph.py", line 189, in invoke_logrotate run.wait(procs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi157 with status 3: 'sudo logrotate /etc/logrotate.d/ceph-test.conf' 2024-04-25T00:03:05.835 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T00:03:05.849 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks/ceph.py", line 1935, in task mon0_remote.run( File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks/ceph.py", line 257, in ceph_log logrotater.end() File "/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks/ceph.py", line 222, in end self.thread.get() File "src/gevent/greenlet.py", line 805, in gevent._gevent_cgreenlet.Greenlet.get File "src/gevent/greenlet.py", line 373, in gevent._gevent_cgreenlet.Greenlet._raise_exception File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/lib/python3.8/site-packages/gevent/_compat.py", line 49, in reraise raise value.with_traceback(tb) File "src/gevent/greenlet.py", line 908, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks/ceph.py", line 189, in invoke_logrotate run.wait(procs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi157 with status 3: 'sudo logrotate /etc/logrotate.d/ceph-test.conf' 2024-04-25T00:03:05.849 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T00:03:05.850 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T00:03:05.852 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T00:03:05.961 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T00:03:06.002 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T00:03:06.002 DEBUG:teuthology.orchestra.run.smithi157:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T00:03:06.005 DEBUG:teuthology.orchestra.run.smithi169:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T00:03:06.017 INFO:teuthology.orchestra.run.smithi157.stderr:bash: line 1: ntpq: command not found 2024-04-25T00:03:06.018 INFO:teuthology.orchestra.run.smithi169.stderr:bash: line 1: ntpq: command not found 2024-04-25T00:03:06.030 INFO:teuthology.orchestra.run.smithi169.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T00:03:06.030 INFO:teuthology.orchestra.run.smithi169.stdout:=============================================================================== 2024-04-25T00:03:06.031 INFO:teuthology.orchestra.run.smithi169.stdout:^- hv01.front.sepia.ceph.com 2 6 377 33 +897us[ +897us] +/- 582ms 2024-04-25T00:03:06.031 INFO:teuthology.orchestra.run.smithi169.stdout:^* hv02.front.sepia.ceph.com 3 6 377 35 +49us[ +58us] +/- 53ms 2024-04-25T00:03:06.031 INFO:teuthology.orchestra.run.smithi169.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 38 -50us[ -41us] +/- 75ms 2024-04-25T00:03:06.031 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:03:06.057 INFO:teuthology.orchestra.run.smithi157.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T00:03:06.057 INFO:teuthology.orchestra.run.smithi157.stdout:=============================================================================== 2024-04-25T00:03:06.057 INFO:teuthology.orchestra.run.smithi157.stdout:^- hv01.front.sepia.ceph.com 2 6 377 33 +874us[ +874us] +/- 582ms 2024-04-25T00:03:06.057 INFO:teuthology.orchestra.run.smithi157.stdout:^* hv02.front.sepia.ceph.com 3 6 377 34 +23us[ +23us] +/- 53ms 2024-04-25T00:03:06.057 INFO:teuthology.orchestra.run.smithi157.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 36 -32us[ -32us] +/- 76ms 2024-04-25T00:03:06.057 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:03:06.057 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T00:03:06.071 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T00:03:06.072 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T00:03:06.114 DEBUG:teuthology.orchestra.run.smithi157:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T00:03:06.220 DEBUG:teuthology.orchestra.run.smithi169:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T00:03:06.311 DEBUG:teuthology.orchestra.run.smithi157:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T00:03:06.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:03:06.339 DEBUG:teuthology.orchestra.run.smithi169:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T00:03:06.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:03:06.366 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T00:03:06.414 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T00:03:06.481 INFO:teuthology.task.internal:Duration was 2064.641697 seconds 2024-04-25T00:03:06.481 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T00:03:06.544 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T00:03:06.545 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T00:03:06.545 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2024-04-25T00:03:06.547 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T00:03:06.594 INFO:teuthology.orchestra.run.smithi157.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T00:03:06.632 INFO:teuthology.orchestra.run.smithi169.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T00:03:07.088 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T00:03:07.088 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi157.front.sepia.ceph.com 2024-04-25T00:03:07.088 DEBUG:teuthology.orchestra.run.smithi157:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T00:03:07.108 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi169.front.sepia.ceph.com 2024-04-25T00:03:07.108 DEBUG:teuthology.orchestra.run.smithi169:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T00:03:07.138 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T00:03:07.138 DEBUG:teuthology.orchestra.run.smithi157:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T00:03:07.151 DEBUG:teuthology.orchestra.run.smithi169:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T00:03:07.204 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T00:03:07.204 DEBUG:teuthology.orchestra.run.smithi157:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T00:03:07.217 DEBUG:teuthology.orchestra.run.smithi169:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T00:03:07.350 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T00:03:07.380 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T00:03:07.380 DEBUG:teuthology.orchestra.run.smithi157:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T00:03:07.432 DEBUG:teuthology.orchestra.run.smithi169:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T00:03:07.470 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T00:03:07.492 DEBUG:teuthology.orchestra.run.smithi157:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T00:03:07.495 DEBUG:teuthology.orchestra.run.smithi169:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T00:03:07.525 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern = core 2024-04-25T00:03:07.539 INFO:teuthology.orchestra.run.smithi169.stdout:kernel.core_pattern = core 2024-04-25T00:03:07.555 DEBUG:teuthology.orchestra.run.smithi157:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T00:03:07.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:03:07.597 DEBUG:teuthology.orchestra.run.smithi169:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T00:03:07.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:03:07.611 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T00:03:07.641 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T00:03:07.642 DEBUG:teuthology.misc:Transferring archived files from smithi157:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672205/remote/smithi157 2024-04-25T00:03:07.642 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T00:03:07.675 DEBUG:teuthology.misc:Transferring archived files from smithi169:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672205/remote/smithi169 2024-04-25T00:03:07.676 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T00:03:07.707 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T00:03:07.707 DEBUG:teuthology.orchestra.run.smithi157:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T00:03:07.714 DEBUG:teuthology.orchestra.run.smithi169:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T00:03:07.758 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T00:03:07.790 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T00:03:07.790 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T00:03:07.863 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T00:03:07.863 DEBUG:teuthology.orchestra.run.smithi157:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T00:03:07.866 DEBUG:teuthology.orchestra.run.smithi169:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T00:03:07.878 INFO:teuthology.orchestra.run.smithi157.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 00:03 /home/ubuntu/cephtest 2024-04-25T00:03:07.880 INFO:teuthology.orchestra.run.smithi169.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 00:03 /home/ubuntu/cephtest 2024-04-25T00:03:07.881 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T00:03:07.891 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T00:03:07.921 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: CommandFailedError: Command failed on smithi157 with status 3: 'sudo logrotate /etc/logrotate.d/ceph-test.conf' 2024-04-25T00:03:07.921 INFO:teuthology.nuke:Checking targets against current locks 2024-04-25T00:03:07.946 DEBUG:teuthology.nuke:shortname: smithi157 2024-04-25T00:03:07.946 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T00:03:07.966 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi157.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672205', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 23:22:11.271807', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHAfRS1+kKdGxkV0ecDPkMU4fbI8Zk7hbDuWL3iDXy2xV3OMdaboLf/Y1BX9iPjRBJLx5eWgCssb0hZqOoSmMlI='} 2024-04-25T00:03:07.973 DEBUG:teuthology.nuke:shortname: smithi169 2024-04-25T00:03:07.974 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T00:03:07.997 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi169.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672205', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 23:22:11.270314', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGUNF8X44qTR0U8iy8BEnPwE+8Z0hrRptOpkJu67oUJaC9XZZroz6yT9WgSl99I8qyu5Rov21j+zdmWFZ0gcnvI='} 2024-04-25T00:03:08.027 INFO:teuthology.orchestra.console.smithi157:Power off 2024-04-25T00:03:08.027 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T00:03:08.052 DEBUG:teuthology.orchestra.console.smithi157:power off output: Chassis Power Control: Down/Off 2024-04-25T00:03:08.053 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T00:03:08.069 INFO:teuthology.orchestra.console.smithi169:Power off 2024-04-25T00:03:08.069 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T00:03:08.078 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is on 2024-04-25T00:03:08.092 DEBUG:teuthology.orchestra.console.smithi169:power off output: Chassis Power Control: Down/Off 2024-04-25T00:03:08.092 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T00:03:08.119 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is on 2024-04-25T00:03:12.079 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T00:03:12.120 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T00:03:12.190 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is on 2024-04-25T00:03:12.230 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is on 2024-04-25T00:03:16.194 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T00:03:16.231 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T00:03:16.306 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is off 2024-04-25T00:03:16.343 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is off 2024-04-25T00:03:16.407 INFO:teuthology.orchestra.console.smithi157:Power off completed 2024-04-25T00:03:16.444 INFO:teuthology.orchestra.console.smithi169:Power off completed 2024-04-25T00:03:16.548 INFO:teuthology.run:Summary data: description: fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu/{overrides ubuntu_latest}} kclient-overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health} tasks/kernel_cfuse_workunits_dbench_iozone} duration: 2064.6416969299316 failure_reason: 'Command failed on smithi157 with status 3: ''sudo logrotate /etc/logrotate.d/ceph-test.conf''' flavor: default owner: scheduled_teuthology@teuthology status: fail success: false 2024-04-25T00:03:16.548 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T00:03:16.628 INFO:teuthology.run:FAIL